In Profundis Progress (6/30)

Ah, progress! Basic support for larger tiles is in now. In fact, I might make the tiles much larger but just display scaled-down versions during the game, which would allow for smooth scaling of the playfield between scales. I used a trick like that fscaling smooth-scaling map screen in Mayflight (a particular favorite effect from that game). I'm not sure I can do this with Pygame however -- one of the reasons I tried moving to pyglet is it can blit images while scaling them in transit. Pygame only scales through the creation of a new surface, which must then be blitted to the destination, which might be too much overhead for me.

One benefit of using larger tiles is the game doesn't have to blit so many images, which is worth another modest framerate increase. The disadvantage, of course, is that less of the world fits on the screen.

In design news, I've started planning out in detail how the random substance properties will work. There are planned to be three kinds of substances: liquids, gases and solids. Liquids are flowing materials like, but not limited to, water. Sand is also a "liquid," for instance. Gases hang in the air and slowly spread out. Both may or may not end up having pressure support, if I can figure out a relatively inexpensive implementation (which is one of the reasons I obsessed over finding ways to speed the game up for a while). The third type, solids, are basically stone walls, which are more reactive in their properties.

So what do I mean by a random substance? The idea is that, at the start of the game, in addition to substances with fairly obvious attributes, there would also be some with unknown properties that must be deduced through observation and experimentation. From the promo video, this is like throwing a torch into a pool of unknown liquid to see if it's flammable. My idea is that the first world the player explores in a campaign would have little or no random substances, but each one after that would have a little more randomness. Another idea, which I mentioned in an interview, is that the planets explored by the player would be divided into solar systems, and each system's would have its own "elements." So, different planets would have different maps and layouts of substances, but the behaviors of those substances would be the same throughout.

If I do this right it could be one of the most interesting aspects of the gameplay. I find that many of my favorite roguelikes are the ones with randomized equipment, which turns figuring out what your stuff does into a logic game. However, it is also true with many of those roguelikes that the identification game isn't as interesting as it could be, or is only a real requirement in the early phases of the game, as once the player learns what everything is there is nothing left to learn.

Anyway, all this is still some time off in the future. I'm going to work on the graphics a bit and fix up the platforming engine before then, I think.

In Profundis Progress (6/29)

Current work is devoted to increasing the tile size and consolidating the tiles into atlases on disk, for ease of editing. Along the way I'm putting in the beginning of different stone types, which will have random properties like gases and liquids.

In Profundis Progress (6/27)

A bit of real progress!  Profiling revealed that, anomalously, most of the processor time was being taked up by one particular lambda.  I've avoided lambdas in the code mostly, partly because I knew the various machine code compilers don't tend to optimize them well.  It didn't help at all that this lambda was being called in the coordinate sort function as a key.  Profiling revealed that this lambda was being called millions of times over the course of a short run of the platform engine.

I replaced it with a class method, and the result is a good 4-5 fps increase.  Further optimization is needed I think, but it's a solid improvement!

In Profundis Progress

So I made a backup copy of the code and spent an hour or two trying out Cython.

Cython (which is different from CPython) is another compiler for Python.  Unlike Psyco or PyPy, it isn't a JIT compiler; it turns Python code into C, and it works on the module level, which requires some elementary reorganization to use.  Not only does it promise some basic speed increase just from the conversion, it is possible to supply some explicit type information to optimize the resulting code still further.  Since that type info is added to the language in the form of new keywords, that has the disadvantage of making the code no longer runnable as pure Python.

The big surprise with Cython is: I got it to work.  There are no Python features in my code that it doesn't support.  This, by itself, is a surprise, and after my latest failures in getting outside code working it gives me a warm feeling to use one of these tools and not be faced with any error messages that I can't fix myself.

Unfortunately, I didn't get any observed speed benefit from it; the framerate actually dipped a little bit.  Still, the relative lack of impossible-to-fix errors is encouraging, and I might play around with this a bit more later.

In the meantime, the thing I can do to improve performance the most is probably to either optimize the cell sort or switch to a different cell calculation selection mechanism.  That's what I'll be working on tomorrow.

In Profundis (6/26, 2)

Profiling has revealed that it's not so much the sort that's taking all the time as the lambda function that determines distance from the player's location.  Hmmm.

In Profundis Progress

Actual work has been light the last few days, as I investigate a couple more possibilities towards speeding it up: eliminating the sort portion of the update loop (mentioned last night), checking into Cython and PyPy in terms of compilers, and trying to get Python's profiling tools to obey me -- I find they are not as simple to use as the documentation on the Python site implies.

In Profundis Design (6/26)

Recently the queue for processing cells changed.  Before it was a simple nested pair of for loops that went through every cell in a rectangular region slightly larger than the screen.  After it uses a substantially more complicated system, where the changed cells from the previous frame are added into a list, the list is sorted by the distance to the player's location, and the first few hundred closest cells are handled, with potential changes from those cells added back into the main list.

This is good for some things, and bad for others.  It makes the preliminary world simulation (two of the generation steps) much faster, and means, if there's nothing much happening near the player, we automatically simulate progressively further areas of the map.

However there is a problem, and the more experienced programmers probably already see what it is.  It's that sort.  Before the change it didn't matter much how large the world was; if it was 2x2 or 100x100, it took the same time to calculate.  The sort operates on every change-flagged space on the map, and it makes a function call for every flagged cell on every frame.  That's roughly 7,000 cell coordinates in a newly-generated terrain world, every frame.

You can actually see for yourself, a bit, how much time the sort takes up.  When you start playing in the prototype, if you turn on the debugging HUD you'll be told the framerate.  On my low-end, single-core laptop, I generally get framerates in the mid-to-high 20s.  The simulation tends to find an equilibrium over time though, and as water pools fewer nearby cells are calculated each frame, meaning the range that gets calculated expands, ultimately decreasing the size of that sort list.  The result is, over time, framerates slowly increase.

This is my primary development concern at the moment.  I'd like to increase framerates by a bit more so that I have more processor time to do effects (maybe related to making fluids less blocky), increase the complexity of the simulation, and perhaps draw a background layer.  Right now I'm getting a handle on Python's profiling tools to make sure my suspicion is correct, but even if it turns out not to be THAT bad I can't help but think I could design this better.  Maybe going back to the rectangle, but increasing its size as visible activity decreases.