In Profundis design (12/30)

So, here is how tools that have a visible extent in the game world should work. Maybe someone can suggest something if they spot a potential pitfall. This is the system used for ropes and spanners, and will probably be extended for other things.

The tool itself exists in one cell of the world. That cell "is" the tool for purposes of recollecting and destroying. A rope hanging from a piton "is" the cell containing the combined object. Spannners, portable, extendable platforms (not what are sometimes called wrenches -- maybe I should find a better name) have a central point from which they extend, and that point "is" the spanner. Make sense?

Once placed, "shadow" objects get placed through the CA system. I call them shadows because they are "cast" by the primary objects, but to the CA they can still have interactions.

On the primary object's turn, it checks in the spread directions (down for rope, left and right for spannners) and sees if it's empty and suitable for spreading. If so, it places the appropriate object (rope tile, extended platform for spannners) in the place. It also sets a "strength" value on the tool.

When a shadow object's turn comes up, it first checks in the direction to grew from, to see if a matching source object or stronger matching shadow object is there. If it is, and has a strength greater than 1, it tries to spread itself, and if it can produces a clone of itself at one point less strength. If for some reason it doesn't have a matching object, it removes itself from the game world, the "source of the shadow" has gone away, it isn't casting anymore.

Advantages to this system:
- Quick. The processing required for each cell remains light. While tools will probably not become very common objects in the game world, it's important that, if the player were for some reason to buy lots of them and string rope everywhere, the game engine could handle it. This avoids a particular Minecraft problem; there, liquid is handled in a similar source-and-shadow manner, but when a source tile is removed, on slower systems the computer becomes very laggy for a while, while dependent fluids are recalculated. (When I shut off or on the water flow in my multi-story mob grinder, which has many forks, long cascades of liquid, and collecting streams at the bottom leading to the output trough, it's several seconds before I can do anything else.)
- Self-correcting. Every shadow object checks for a source every turn; if the source disappears for some unexpected reason, it does to, and so thus do objects that depend on it.
- Fun to watch. Multi-tile objects don't appear instantly, but over a short period of time as the shadows propagate through he world.
- Has a certain elegance.
- Respects 'c', which is important for avoiding subtle CA calculation problems. 'c', or the "speed of light," is a bit of cellular automation jargon -- it refers to the maximum rate at which changes may propagate through the system. In Profundis' speed of light is one tile/frame. At the moment, no tile can affect the game world more than one immediate space in each direction. If placing a rope instantly placed all the rope tiles into the world, it would break the speed of light. Currently, In Profundis calculates its world in a square spiral pattern emitting from the player's location, skipping every other square, but starting on an even or odd square in successive turns. The result is that a checkerboard-like layout of tiles is calculated every turn. Since no tile can produce and effect that extends out further than one tile orthogonality, we can be sure that changes aren't propagated too fast, producing results like instantly-falling water. This doesn't matter so much for example tools, but might be important for other tools later.

Disadvantages:
- Inherently tile-based. I was worried before about the game's strict grid-based presentation; I've brainstormed ways to effectively implement sloped rock tiles that still handle water effectively for instance. I've spent too long trying to figure that one out however. The project has gone on long enough, and I think now I'm not going to worry about the limitations of the grid-based CA. It certainly hasn't hurt Minecraft any.
- May result in circular, self-dependent shadow objects if those objects aren't designed well. Could a situation arises where multiple-placed primary objects interact in a way that a self-reinforcing system of shadows could set up? Not in the simple situations I have described, I think, but more complex objects might have branching shadows, and if somehow a shadow object can produce another shadow of greater strength than itself this situation could arise, where all sourcing primary objects could be removed but surviving shadows keep each other refreshed.
- Has a certain lack of realism. Imagine a case where a spanner platform extends several cells out into the air, and a boulder falls through the middle of it. When it falls through the platform (spanners aren't strong enough to support such weight) the next platform over will no longer have a source, and will vanish the next frame. But the boulder will then continue to fall, and so the spanner it destroyed will be recast the next frame. The result is a one-tile gap that moves down the spanner like a wave, a behavior that isn't mirrored in real life. Similarly, what if gravity affects spanner sources? If the source is unsupported, it will effectively create a V-shaped platform shadow through the world as it falls, as it created new platforms around it as it descends, while those left behind it by its fall both spread and disappear outward through the route from where the spanner passed.

Still though, I think the mechanism solves more problems than it causes, especially if I keep tool designs under control, and it may be applicable to simulating non-tool objects as well. Remember, it's always best to keep CA effects as simple as possible; the more work you do on each cell, the more likely the whole simulation may, in some situations, start to drag.

In Profundis progress

Yep, I'm still alive, although right now I don't quite feel so definitely on the "alive" side, due to a cold that's lingered around a few days. It's been a hectic month overall. Distractions abound. I presented my final paper for my Master's degree and it went very well. A cousin unexpectedly died in a motorcycle accident. And I've been trying to work in bits on a prototype for a friend's game. I haven't forgotten about In Profundis though. Hell no.

In Profundis Progress: Tools, Inventory and personal coding deficiencies

This is about the place where my general lack of experience with modern (that is, post Commodore) development is biting me hard. Something I've been worried about is the representations of user-placed items in the world. There are three general possibilities here, and I've messed around with each one to some degree. Possibility one is to implement them as sprites, that is, as moveable objects that can move independently of the cellular grid, like the player can. This is what I tried first, but after a bit of that I realized that it really wasn't a good solution. Part of what makes tools interesting is how they can interact with the game world, and every object that isn't part of the basic world model, even if it's grid-aligned, complicates the code. However, it gives rise to the problem with possibility two, which is, define special tiles for in-world tool objects. It looks weird when water flows around a piton as if it was a full-block object. And a lot of tools are like this. It's the existence of liquids and gases, which are just cheat analogues for sub-grid-level phenomena, that makes this an issue. The nature of the game is such that, if push came to shove, I could probably get away with doing it that way anyway, but instead I'm opting for possibility three, which is to give each cell a tool field that can contain its own object. This way liquid can flow past a piton without overwriting it or flowing around it, and objects in lakes can rest there without making an air bubble around them. Now the better programmers out there are probably wondering why I'm spilling words on such a decision. It's because, really, this is the most complicated project I've ever worked on. Mayflight's world builder was more complex, but its world simulation was just a platforming engine. I'm somewhat out of my depth here. At least writing this in Python means I don't have to muck around with (overt) pointers and worry about the myriad bugs those can introduce. I visit anna anthropy's blog pretty frequently, and I've been known to chuckle to myself when I read about her aversion to coding. I really can't imagine doing a whole game using, say, Game Maker's abhorrent drag-and-drop scripting functionality -- I always tie each event to a text script. But really, I'm just one step further along the path than she is, and there's a dozen ahead of me. What is funny is that I was a quick study back in programming classes, but there's something about pointers that screws me over. It's not that I don't understand them -- far from it, I have a strong processor-level understanding of the things. I did a fair bit of assembly programming back on the C64. But something about the way C-style languages approach them trips me up hard. I'm constantly confusing where I need to use * and where I need to use &. I know that * is a pointer to some value, and & is the value that a pointer points to, but I don't seem to have it internalized to the point where I can write code with those symbols confidently. I wouldn't be surprised if I had some form of dyslexia tripping me up there. The language is such that you must know what is a value and what is a pointer to a value, but I nearly always get them mixed up. This, coupled with the fact that I was continually stymied by import errors, was probably what sabotaged my attempts at restarting In Profundis in C++. Game Maker is really not that different from C in syntax, but its garbage-collected, reference-counted scripting was the breakthrough feature that got me productive with it. I'm just blabbing here really; I felt guilty that it had been a little while since I made a post here. Anyway, working in inventory and tool simulation now. I hope to start putting in the backer treasure data before long, so that'll be fun at least.

In Profunds ideas: Flags

So here's something big that I'm playing around with....

In Profundis gives you a limited inventory for carrying around equipment. You tend to use up resources while you're exploring the game world, placing pitons, hanging ropes, and so on. Eventually you have to travel back to base and resupply.

A concern that I have for the game is that this travel back and forth, between base and the frontier of exploration, could get annoying, or worse, boring. But the resource and time costs of getting around the game world are the primary obstacle to exploration. How do I reconcile the two?

My idea is a special inventory item: flags.

After you explore for a while and figure it's time to return to base to resupply/get some rest/buy new stuff, what you can do is plant a flag at a place you'd like to return to later.

After this is done, the challenge is this: return to base without using any other items in your inventory, in order to "establish" the flag. It's like a mini-game in a way. It is okay to use items you've already placed, so you can sort of pave the way for your return trip. If you're forced to use an inventory item the only thing lost is the establishment run. In that case the flag remains where it is, and you can try it again just by returning to it.

The reason this has to be done is because the game is recording how long it takes the player to do it, in order to figure out a realistic travel time, and using resources along the way that might not be available later would spoil the experiment. (It's possible that this might just be "shallow travel" equipment, like jetpacks, rather than things that change the game world to make it easier to traverse, like pitons and spanners.)

After you've established the route, you can later "instantly" warp between base and the flag. Instantly is in quotes because this is only immediate to the player -- the game clock is set ahead by the amount of time it took to make the trip. Since the expedition is charged rocketship rental by the amount of time spent on the planet, this is a substantive cost, but it's your character who has to pay it, not you. The character is still making the trip, in effect, on autopilot. (In fact, I'd like there to be a "montage" of still shots of the trip, that takes a few seconds to play out, whenever warping.)

Continuity of the game world is very important I think. The problem with teleporters is that they basically make a mockery of the spatial extent of the world. I consider this bad because, as the player makes longer and longer expeditions, the increasing time required to get from base to the exploration frontier is, itself, an increase in difficulty. Exploring the other side of the map takes more time, which incurs more spaceship rental costs, which means the player can't afford to waste as much time.

Some of you are no doubt reading this and thinking something along the lines of: WTF? I think I can understand that. But I look at it this way: I've played hundreds, if not thousands, of video and computer games, and I've never known a game to do something like this. Maybe there's a reason for that. But it's a very interesting game mechanic to think about, and I think that will transfer to play. What do you think?

In Profundis Problem: Continually flowing water

One of the reasons there was a bit of a delay in work on the game was because I was mulling over for a while solutions to certain problems with the game as envisioned.  Here I explain one particularly annoying problem that I'd really like to see fixed, but currently don't have a very good solution for.  Maybe one of you might have an idea.  At the very least, maybe some of you might find thinking about this interesting.

In Profundis uses a cellular automation system for flowing water, yet because of the closed nature of the game world, most flow is ultimately user-initiated.  Liquid tends to pool and collect and remain as low as it can get, and remain there collected.  It doesn't actually end up flowing much except at the start of the simulation.  Once it gets to the lowest point it can reach, there it remains until something disturbs it.

This is not particularly interesting from a gameplay perspective.  What would be more interesting is if water could enter and leave the map, or at least flow through two locations in the world, repeatedly and endlessly.  Then the player might have to find some other way around it, or maybe divert the liquid to give himself a safe path.

The problem is that, because of the size of the game world, we can't compute the whole thing every frame and keep within an acceptable speed.  Some distance outside of the visible screen we have to abandon calculations.  This causes liquid to "pile up" when it flows outside the border.  If we recycle liquid between two "portals," unless both of them are within the calculation region at the same time, the flow will stop.  And as the player scrolls around the map the calculation region changes, meaning while approaching the area there will almost always be a time when one portal is in the region and one is outside.  If they are far enough apart, there could result in instances where the player is suddenly beset by a rush of liquid that has "backed up" because of the simulation border, then suddenly uncorked by the scroll.  (This could actually be interesting from a gameplay standpoint, but it goes against the philosophy of the game.)

I'm not sure there is a viable solution to this problem, but here are some possibilities:
* One thing I could do is consider the region between the two portals an "all-or-nothing" region for calculation; if one cell of the rectangular region between the pair is calculated, then do the whole thing.
* The larger the simulated region, the less evident the problem becomes.  If we work on enlarging that area (maybe by slowing down the rate at which fluids flow, which I've been considering for a while), the problem will become less visible.
* It's possible that the problem won't be perceivable to the player most of the time.  Because his range of vision is limited, even on-screen backups won't be too apparent so long as they're outside his field of view.
* The real-world solution to this would be a evaporation/condensation cycle, but that would probably require a scale larger than what we're looking for here.

In Profundis Progress (10/30): Tools and Flags

Still working on the in-world representation of tools.  I've decided that they need to be simulated on their own "layer," that is to say, a given space can contain both a tool or other object and more substantial world simulation elements like water and gas.

What do I mean by a tool?  Tools are objects the player can interact with, place in inventory, but also change the world to his liking.  Changing the game environment is a major theme of the game; the pitons and ropes you place to get around don't go away over time unless something removes them.  Since you can only carry so much at a time, exploring is an iterative process of placing equipment, over multiple expeditions, letting you get further each time.

One thing I'm worried about is that the process of traveling back and forth in the game world, between base for resupplying and the edge of exploration, could end up being boring.  One solution to this problem I'm considering is the use of player-positioned checkpoints.  The idea is, the player could obtain "flags" that he could place in the field.  After planting a flag, the idea is to then "validate" the flag, by traveling back to the base without placing any more equipment.  If this is done, the flag is validated, and the player can then warp from the flag to the base at the cost of the amount of game time it took to make the trip.  Likewise, the return trip can also be validated for reverse travel.

This system isn't perfect; after a flag has been validated going one way or the other, it is possible for the terrain between the two to change and make the route impassible.  This could be partially solved by keeping a list of the cells traversed in the route, and upon teleporting checking each of them to see if they've been modified since the route was validated.

In Profundis Progress (10/28)

I haven't been doing nothing on the project exactly, but I'll admit there's not a lot of actual programming work to show for it lately.  I've been thinking.  Thinking about things like this.  If this seems a little incoherent it's because I'm writing this as I work it out in my head.

One of the ideas of the game is that the player has to manipulate the game world to get around in it.  What you do changes the world, and changing the world is dangerous.  Like, putting a piton in a wall could crack the wall, causing liquid to drip through, possibly weakening the wall further, eventually destroying the rock face the piton is in.

It's not good enough to use a new cell type for each type of tool, because otherwise things like pitons and ropes will block fluids, which is no good.  Instead, each cell needs its own Tool field.  That adds a little overhead to the cell structure and cellturn(), but isn't too bad.

A rope can be tied to a piton.  The question is however, how to represent this rope? Do we arbitrarily plop it down when the piton is placed?  What if the piton is removed then, how will we know to remove the rope?  If multiple pitons are placed vertically so their ropes overlap, how will we know to remove the right ones?

Another thing is, I've learned a bit more about how Python handles recently, particularly I've cleared up in my head how it handles static class variables, which stands to simplify the code a bit and make it more maintainable, but requires going back and redoing some of my old work.

So work is continuing, and in fact should speed up now.  I find my energy for project work tends to come and go, based both on internal and external factors.  Updates should be frequent again, at least for a while to come.

Not forgotten....

Other projects and such have been vying for my time lately, and I've kind of let them for a little while because I'm wondering if what I have so far is "fun."  It has the capacity to be fun, I recognize, the question is if the world builder can take advantage of the potential to create interesting situations.  That is a hard problem; I've got some ideas in that direction, but there's

There's also the matter of some weird bugs that have cropped up in the fluid code with the random fluids.  Notably, when walkable fluids float atop swimmable ones, it's kind of wonky.  The platforming routines need a lot more work.

There's a thousand things left to do, but I'm working on them!  I am sorry it isn't going as quickly as I hoped.  I'm afraid I might not be as good a programmer as I had hoped -- the project is reaching the edges of my capacity to keep it all in my head at once, and that slows things down.  Please bear with me in this difficult phase of the project.

In Profundis Progress 9/29

Ah, been slow going for a couple of weeks I'm sorry to say.  I keep getting caught up by stupid bugs.  This time, it was an out-of-bounds error that turned out to be caused by some debug code I had put in and forgotten about.

In positive news, a change to the way the game decides which cells to calculate has increased the range it looks at while simultaneously bringing a framerate improvement.  The change was actually suggested by someone in the comments to Gamasutra article I wrote about cellular automation; specifically, not to calculate every cell around the focus, but just those in a checkerboard pattern around it.  Reverse polarity of the checkerboard every turn.  Calculating fewer cells means I can calculate a further range, but the result isn't a noticeable decrease in fluid flow speed because I was already ignoring cells after they flowed one space until the next frame anyway, so as to prevent moving objects from teleporting.  I think even now fluids might be flowing a bit too quickly, so I might ease up on spending so many cycles on calculating.  Having a better frame rate means I can put in more gameplay features and/or graphic effects, so it's a very good thing.

In Profundis Progress 9/17

ARGH

There is a bug in the game, an infuriating one that has got me, at the moment, stumped.  It has to do with the system the game is using to keep track of cells that potentially need updating.  In one specific circumstance, that being when fluids hit the edge of a platform and flow down off the side, some of it is often refusing to remain mark active long enough for all the fluid to flow off the ledge; it just pools up there, unsupported.  If something happens next to it it causes it to "wake up" long enough for some more of it to flow away, but only for one frame.

I'll figure it out eventually.  Just wanted to expose you all to my current personal hell.

In Profundis Progress 9/15

The last time I posted a picture of a game map the region styling code wasn't working properly. Have a look at a random game map now:

Looking at this, I feel excitement for the project building again.  For randomly generated platforming terrain, I think this is very good.  Keep in mind that fluids aren't even generated in the random maps at the moment, and there will be gasses in the final version too.

Here are some in-game images:

1. This is just outside the "base" area.  When done, the player's rocketship will be in this area, which serves as base camp.  Notice the character design has changed a bit: her head is more realistically proportioned, and there's now a spaceman's bubble around her head.  Also notice the faint colored backgrounds; these are colored from leftover data from terrain generation, so they tend to suggest the platforms are extending from background features.  (Well, that's how I describe it.  Maybe there's a better way to explain it, but I haven't thought of it yet.)

2. The colors of the walls will be important; they represent different types of stone.  Unfortunately they're fairly blocky at the moment.  I've thought a lot about how to remedy that without spending extra rendering time for edge effects, haven't come up with a good solution yet though.

3. A tunnel.  Currently the background being drawn for out-of-sight areas so long as the player has seen them before; this is a bug.

4. I put some liquid into this image.  Note that the blue substance here looks and flows like water but it isn't; it can be stood upon!  This is a property of this liquid, generated randomly.  I'm not sure what the gameplay consequences of this will be however; what if the player falls into an underground sea, but floating atop that is a layer of "solid" liquid?  Should "solid" liquids always be heavier than swimable types?  I'm currently thinking that, in most cases, it should.

In Profundis Progress 9/13

More news!  I've put in some background graphics now.  They're faint, indistinct and essentially huge pixels, but I think it kind of fits in with the art style.

It takes me a fairly extensive amount of work to create a demo video, since I have to record it, encode it, upload it to YouTube, get links to it, etc., but I hope to have something new for you guys to see soon.  Current work is focused on fleshing out the region terrain templates and getting fluids generating again.  Slowly, bit by bit, it's coming together.

In Profundis Progress: 9/11

Not a lot of progress the past week unfortunately, DragonCon got in the way of things and morale is kind of low right now. Have managed to pull myself out of my funk enough to start work on inventories and object simulation though, and done some of the art for those objects. Just generic stuff so far though, haven't gotten any of the backer-sponsored items started yet, but those are in sight. It'll be nice to get some of those things off the list.

In Profundis Progress 8/29: The map!

We have maps now!

This is a large image; the above is just a thumbnail.  It's not the complete map; it's one part of it, roughly two-thirds the whole thing I think.  This should give you some idea of the scale of the game.  It's so large that I'm actually starting to think that maybe multiple worlds in a single game is overkill; there is a lot of room in this map to hide things, and it wouldn't feel arbitrarily limited to not go on to a new planet after searching an area this large.  In fact, it might grow tedious if, in a single game, players had to explore many maps this big.  By this way of thinking, each game would be a single world, with a single map.

This isn't giving away any secrets because, of course, the world is randomly generated each play.  Note there are no liquids in the above map because they currently aren't being generated in world generation; putting those in is on the list of the next things to do.  Still lots of work to do of course, but figured I'd give you guys an idea of how things were progressing.

This also shows the different types of stone in the game, which serve the important functions of giving players another thing to learn about and giving different regions of the map their own visual identity.  Not pictured here is the updated character art, which I'm much happier with; it's kind of a mix between the previous art and the silhouette I used for Mayflight.

In Profundis Progress 8/21

I think I'm finally coming out of the woods regarding world generation.

DragonCon is in a week and a half.  What I would like is to make another prototype available to play around with before then.  It'll be difficult, but just barely possible I think.  We'll see anyway.  More before long I hope....

In Profundis Progress 8/17

Been a few days since the last post.  This stuff isn't really very interesting, but just to fill you in....

Still working on world generation.  The key now is setting up the materials, and the "personality" of each region, and getting the materials to generate properly in each region, and giving the materials their own properties that can affect the player and each other.  This is taking a long time; I seem to have a mental block concerning the algorithms necessary for the kind of random generation of properties needed here.  I'm making lists of substances, and lists of properties, and tying the properties to the substances, and lists of regions, and lists of substances in those regions, and deciding which substances should go in which region by depth and difficulty... and it's not something I can easily hold all in my head at once.  In other words, I'm having organizational problems.  I think it's time to sit down and just plot this part out, then implement that from off the paper.

In Profundis Progress 8/11

Gah.  World generation work continues, although haltingly.  I had a couple more annoying bugs.  I seem to remember programming being easier than this.  I continue to forge ahead.

The feature I just added, and took me so long, is world regions.  That is to say, each world won't be a homogenous area of similar terrain, but will be "themed."  This allows the game to tweak area creation by contiguous areas of the map in an intelligent way.

Man, when I see Tarn Adams chugging away, adding new features daily, I get intensely jealous.

In Profundis Progress 8/7

It's still rolling along.  Here I'm going to detail some of the finer steps in the world creation algorithm I'm working on.

The steps (partly restated from last time):

1. Define random material properties.
2. Make an array of sectors (each representing a 12x12 area of world tiles).
3. Build a maze in that array, with each element holding which adjacent elements it's connected with in bitfields.
4. Perform processing on that maze (this is what this post is actually about -- see below).
5. Take maze, indexed into heightmap influence templates, and apply that to an array matching the size of the world.  (This is what I talked about last time.)
6. Turn heightmap into wall cells in game world array.  Along the way, apply any special codes in the templates that produce special features - <i>ignoring liquids</i>.  (This part is new.  I'll have more to say about it later.)
7. Perform "flow experiments" to find good liquid flow paths through the world and create "wormholes" (actually, grates) that moves liquids from lower to higher points.  I'm handling this process by putting in some simulated liquid into the world and running some turns to see what it does, then cleaning up the liquid afterwards.
8. Apply the liquids skipped back in step 5.
9. Later processing, tile smoothing, sediment laying, etc.

Mazes are intrinsically interesting structures to explore, but have some problems, especially related to their use in random world generation.  A proper maze has only one way to get to any location in the maze, meaning if there is a difficulty in getting through a section, it makes it correspondingly more difficult to get to any location beyond that section.  By putting in more ways to get to a place, the gameplay experience is made more robust.  At the same time it actually simplifies navigation, due to the existence of tricks like the "left/right hand rule," a.k.a. the wall follower trick.  (More on maze solving algorithms at <a href="http://en.wikipedia.org/wiki/Maze_solving_algorithm">the Wikipedia page</a>.)

My idea is to put loops in the maze that allow the player to find ways around trouble spots.  (It's possible that the other ways have troublespots as well, but we're not trying to hand them the victory here.  They will have to sweat sometimes.)  I used a system like this in Mayflight, but I didn't care much about where the loops went.  This often meant the loops were trivial, like connecting two already-connected rooms.

I had already come up with a way to solve this problem many years ago however in an old Commodore program.  The solution is to make an intermediate pass and record each spots' minimum distance, traveling through the maze, from a random starting point.  Then look through the maze and find the adjacent spots with the greatest difference in distances from that point, and connect them.  To make additional loops it's then best to refigure distances, taking the new path into account, from a new random point.  The result (if memory serves) is a more satisfying maze to explore, with longer paths and more work to do to find alternate access routes.

In Profundis Progress 8/3

Still working on the world builder.  Here, I'll lay out the scheme I'm planning for the maps produced by the game, and what I'm aiming for concerning the overall metagame.  I am open to comments and suggestions here; I have a strong view of what the game should be about, but the specifics I'm still thinking about.

1. To start with, the game divides the world into <i>sectors</i>, which are 12x12 tile units.
2. Then the world runs a maze generation algorithm on those units to create a "true" maze, that is, one with no dead-ends.
3. It's not implemented yet, but eventually some more high-level processing will be done on this maze, to create loops providing alternate access to some areas, and also divide it into <i>regions</i>, each of which will have different generation properties, such as large block obstructions, seas with a specific type of fluid, narrow tunnels, etc.  Right now, I'm defining the <i>templates</i> for each type of region.
4. Once regions have been set, we change the sectors contained within each region to that region's template. Most of the templates are laid out in lists according to a 16-unit scheme, indexed by the bits connecting from that sector or'd together.  (Up=1, right=2, down=4, left=8.  Embarrassingly, confusion between two different bit schemes I was using in two different places caused a mysterious world creation bug it cost me a day in tracking down.)  Since each region type has the same kinds of templates laid out in the same order, it's a simple matter to drop them in as replacements.
5. Additionally, any sectors left out of a region can be randomly "tweaked," just changed to one of the possible templates matching its connections.
6. Once all of the templates are set and decided on, it's time to <i>render</i> them out to the game map.  Each space in each template has an integer value and (optionally) a code.  A random number in the range [-8,8] is generated for each space in each template, matching its location on the gamemap that added to that template.  The integer amount is added to each of the cells surrounding its corresponding location on a matrix <i>heightmap</i>, of the same size as the gamemap.  The result is, places that templates say should have walls spread out "wall influence" from their locations, and places that should have empty space spread out "space influence."  Wall influence is represented by positive numbers, and space influence by negative numbers.
I realize this is hard to picture.  I wrote a column on this algorithm for GameSetWatch some time back (scroll down to "room building").  As that column notes, the result looks like what happens if you apply a blur filter to a field of noise in a paint program.  Each random pixel is "spread out" across adjoining pixels.  The spreading-out actually introduces some order into the random field.
7. Finally, we go through each cell of the heightmap.  Cells that are 0 or less produce spaces in the corresponding cells of the game map, and cells that are positive produce walls.  That is, unless the template had a special code for that space; those codes can override the heightmap and produce "hard" spaces or walls, or other special tile types, or instead they can work with the heightmap and produce fluids or gases if the space is ordinarily empty.

One advantage offered by the gameplay (eventually) is that, since the player has equipment that can dig through stone, it isn't strictly necessary to ensure every sector of the maze turns out to be accessible through space, although neither is it particularly friendly if large sections of the maze are commonly locked off requiring a pick to access.

In Profundis Progress 7/31

Work on the game has been rather hard as of late, but I have basic level building working now. What this means is, the game will now construct a roughly naturalistic-looking maze to explore. Getting around in the maze is for when equipment is implemented, which is moving closer.

Although the game has maze-like caverns now, they aren't very interesting yet. Next is to flesh out the world builder a bit. This is a multi-stage process involving dividing the maze up into regions, styling each region according to a generation theme, tweaking passages, making the ends of the maze into places worthy of discovery, and making loops in the maze so as to give players multiple ways of access in case one proves too difficult.

More soon. I intend to post a new video once caverns become more interesting looking.

In Profundis Progress 7/29

Still working on generation. Spent today tracking down a mystery bug that turned out to be caused by my misunderstanding of numpy's workings. Hopefully more to show you guys soon.

In Profundis Progress 7/24

Been a few days since the last report.

The big thing I've been working on has been random substance support. Part of this means retrofitting the fluid physics code to generalize a number of prior assumptions. At the moment there are five fluids: water, sand, oil, "fire" (which doesn't burn anything yet, and burns out almost immediately) and "sediment" (a kludge to create smoothed rock floors). Under the new system, water and fire, and maybe sediment, will be the only hard-coded substances. (And I'm considering towards removing water's privileged position.) The others would be possible options that could be randomly included in a world, or could be "mutated" so that it works with different physics, or there could even be entirely random fluid types, made of different properties like density, flow mode and interaction types. Gas and stone would be similarly scrambled, although their workings would be less complex than fluids.

A second thing I've broken ground on has been the world generation. At the moment the "Terrain" world generation has been surprisingly interesting, but it's time to construct some actual play territory now, bringing in a semblance of game design to what up until now has looked more like a tech demo.

I've started on building the world "pieces" that the game will piece together to make worlds. The terrain generator is a further elaboration of the stuff I did for Mayflight, which is in place unfettered as the current generator. So, the "world pieces" I'm talking about are more accurate described as influence templates.

One problem that lies in this direction is that while the Mayflight terrain generator is awesome at producing nicely corrugated terrain, it's not so great at producing solid, pre-set puzzle areas. I like of like that though. I want the game to be more about making terrain which might sometimes be too easy or too hard that taking a strong design hand and making a bunch of pre-made situations, situations that might be interesting in themselves but by their very nature are still finite in number.

In Profundis Progress 7/21

I've got gases partly reimplemented, they aren't displaying properly at the moment though.

I'm also putting in some of the infrastructure for randomized material properties. To explain:

One of the basic ideas of the game is that you encounter liquids, gases and even walls with unusual properties. They might be bouncy, or sticky, or flammable, or explosive, or poisonous, or have any of a number of other possible attributes.

Here is my plan for implementing this.

First, there is one of each type of substance (fluid, gas, stone) that is not random, but is otherwise boring. For fluids this is water, for gas this is "breathable," for stone this is "inert." My idea is that this substance is the most common of its type in each world, except maybe in the weirdest areas, just to keep down player confusion. (If everything you touch immediately explodes it doesn't make for a very interesting game.)

There are also a number of other predefined substances. Sand and oil are the two major types of these we've seen to date, both holdovers from World of Sand.

Each world also bears a difficulty value, which increases as the player explores more worlds. As difficulty goes up, at world generation time the game will randomly switch out more of the stock substances. So a game might not have sand, but another fluid. It might clump like sand, or it might slide around the ground, or it might slowly eat through stone, etc. Higher difficulty values also make it more likely that dangerous attributes are selected. Later on it would be possible for each substance to actually have two such attributes.

My problem here is: will this make for interesting gameplay? This is the biggest question of the design. My hypothesis is that it will if the player is given a limited number of ways to figure out what substances do, akin to the rogue like scroll of identify and Nethack's many ID tricks. I think this should be balanced so that players can't test out *everything*, just some things, forcing him to put up with whatever special attributes other things turn out to have.

Even if it's interesting, will it turn out to be enough to carry the game? The exploration aspects, the navigation and treasure seeking, and the resource management game of making efficient use of your equipment, all of these things together are what I consider to be In Profundis. If that turns out to be fun by itself then we can make an initial release and add in stuff like fighting/communicating with aliens later. If it doesn't, well then we might have to add in other actors earlier rather than later.

In Profundis: New Video

The work on the graphics engine has borne a bit of fruit.  Have a look:

http://www.youtube.com/watch?v=JBdauy4GaHU

Notes:
This is the first time I've shown off the engine using larger tiles.  Of course the graphics for walls are fairly plain right now.  What isn't shown is that there are now up to eight kinds of stone, which will have randomized properties.

The big thing that's visible here is that sand and water now has a smooth surface.  In particular this looks nice when a single level of water is moving around at the surface of a pool.  At the moment this is only cosmetic, and a bit glitchy at that.  Notice how, when water meets up with sand at about the same level, the sand angles down but the water stays level, and so it looks like it's being held up by air.

Also, the known "map" areas out of sight but still onscreen are now displayed using outlines.  There is some intelligence there that outlines the edges of the region but not the tiles in the middle.  It's not perfect yet; you can see some dots in regions where aborted walls are drawn, and the line isn't complete at the right-hand sides of slopes.  Not to mention, in some cases where sand is underwater, a black corner is drawn on the sand.

The update engine now uses a hybrid of the first system, which was a simple for loop, and the second system, which kept a list of all the tiles that might change this turn and iterated through them.  Now it iterates through the cells in a spiral pattern from around the "focus," which is either the player's location or the cursor.  It keeps a "dirty" flag on cells that might need updating next frame, allowing the game to avoid looking at every cell with complex interactions, like fluids, just to determine that they don't have anything to do this frame.  There are a couple of bugs with this too at the moment: sometimes the game forgets about liquids on "steps" and they are left to just pool there, and in edit mode placed fluids don't do anything until an adjacent space is modified.

I've also started planning out how gasses will be returned to the game.  They probably won't be updated as often as liquids, which befits a more diffuse way of spreading.

In Profundis Progress 7/15

1. Considering redesigning the character some more. It's not that it's not okay, but it's more like, we already have a CA-type game with graphics of sort of that style (Terraria). I'd like to distance this game as much as possible from that.

2. Abandoning the "coordinate list" more of keeping track of updates. The sort is just too much of a performance penalty. Someone MeMailed me a while back and suggested using a quad tree to keep track of which cells needed to be updated, but I frankly don't see how I would get substantial advantage from that. I will admit I am no expert on the structure though. It seems to me that any advantage gained from that wouldn't be greater than just iterating through them and using update flags to rapidly skip by any cells that don't need calculation.

In Profundis Progress 7/13

Some heartening progress over the past couple of days! I've somewhat successfully gotten the graphics rendering to display more smoother graphics for liquids without resorting to making hundreds and hundreds of tiles covering every possible liquid state. I'm not sure as to what the performance hit is for it yet, more news soon.

In Profundis Progress 7/11, pt 3

I think I've discovered the best way to smooth over the liquid tiles. It does require a fair amount of extra tiles to be generated, but it shouldn't be any slower than the current system.

I did a bit more looking into the state of Python. There are some very promising avenues on the horizon for it, that should be of interest to other Python game developers:

1. There is an optimizing compiler for Python coming up called, in the style of Monty Python references, Unladen Swallow. It's developed by Google so it has some strong support behind it, and the word is that eventually it will make it into the stock CPython distribution. That, coupled with the existence of Cython, make it clear that there is strong hope for dropping Psyco in the future.

2. There is some murmuring in the community about creating a version of Python for iOS. That's a target platform for me. I don't know what the performance will be like (word is that it's not bad), and anyway the "gold rush" phase of iOS development seems to be over, but it's still a bit of a vanity thing for me to make an iPad version. We shall see.

More In Profundis Progress 7/11

Additionally, I reimplemented an extra little quirk of vision: you can no longer see through sand or sediment more than a full layer (meaning, for example, you can no longer see the stone floor beneath sand dunes), and full tiles of other liquids halve vision range through them.

In Profundis Progress 7/11

Today, I'm working on a bit that will simplify liquid tiling. Instead of drawing each tile and referring to them all one at a time from the source, I'm just loading in a mask for the liquid, colorizing and saving it for later use at startup. This means, instead of having 16 tiles for every liquid in the game (8 for visible, 8 for map), I just have to have 16 tiles, period. The game makes the colors itself.

This is important because one of the biggest complaints with the videos was the blocky graphics. Smoothing them out is not as simple as some people seemed to think it would be. The easiest/fastest way to do it would be to create tile graphics for every possible slope of liquid surface, but with at least 8 possible liquids in a world, maybe more, that quickly runs into many, many tiles, which would take a long time to draw. This way, I just have to draw a basic set of tiles and colorize them ahead of time. The biggest drawback is that adds up to a lot of tiles; over 1,000 by my reckoning per liquid, but compensating for that is that each is only 32x32 pixels.

Well, we'll see how it turns out.

In Profundis Progress 7/9

Current work on the game concerns mainly refining the platforming. One of the complaints people had upon seeing the videos was that the world looked blocky, especially falling water. While I haven't decided on the best solution to illustrating falling liquids yet, the platforming change should make sloped surfaces a little more feasible. Emphasis on "should;" it's going to take some work regardless. Sloped surfaces are deceptively tricky to implement in the way one would expect (which is largely based upon Mario 3) in a tile-based world.
Also, I've switched the internal representation of the world fro a Python list-of-lists to a numpy array. The result is a very slight performance boost coming from reduced overhead, but more importantly it simplifies the code a bit, and should be easier to optimize in Cython.
Both these things took rather longer that you might expect, from tracking down bugs and charting out code. Still, progress is such that morale is up at the moment.

(EDIT: Fixed date, which was incorrect [again] due to confusion over the current Peanuts Roasted date.)

In Profundis Progress (7/7)

More bug fixing in the wake of the switch to more integer-based math for platforming. Internally actor coordinates are switching from pixel-based math, with decimals handling sub-pixel movement, to 16 subdots per pixel. This means tracking down a few places where values are translated between pixel-level representations and cell-level.
The next thing I'm working is making platforming as bulletproof as possible. Sorry if it seems like it's going slow, I am but one person. It is happening though.

In Profundis Progress 7/6

The past two days of work have been oriented towards ripping out as much of the floating point math as is convenient from the platforming engine, and switching it to integer.  It's a general code improvement thing; relative to the amount of time the game spends updating the world state, the performance gain from removing floating point math is slight, but it'll make collisions simpler to detect and handle by not having to worry about piddling small decimals riding on the value.

In Profundis Progress 7/3

I had a bright idea.  What if we didn't draw every tile every frame?  What if we just scrolled the image from the last frame, and drew in tiles that changed, those scrolling in from the edge, and those around actors?

Today I tried this out.  And you know what?

It really didn't make all that much difference, while it cluttered up the code with drawing flags and checks to see if the tile scrolled in.  I think the scrolling, even though handled by SDL, end up taking almost as much time as it took to just draw the tiles anew each frame.  D'oh!

In Profundis Art: Character Idea

To replace Placeholder Guy as protagonist, here's a sprite I've worked up.  Tell me what you think:

Points of interest:

It's a girl, but not ostentatiously female.  In the final game there might be a gender choice, depending on how much time it takes to make alternate versions of every frame of animation.

Shoes are a bit oversized.  I like to think of them as "clompy boots," in the style of the Iron Boots in Ocarina of Time.

Realistically proportioned body, with a big head. 

Orange backpack, ala Earthbound, explains freakish carrying capacity.

Clothes functional, a bit baggy.

Dammit Blogsy

Sorry about the Peanuts post here.  Been trying out a blogging client, and it makes it easy to get blogs mixed up.

In Profundis Progress, DIrection

So, here is where we are:

I have changed the graphics loading mechanism to use "atlases," large images separated into tile blocks, instead of the many individual images used before. This will make improving the graphics a lot less annoying. The tile size has also been increased to 32x32, which fits less onscreen at once but gives a better view of the world. We'll see if it's a good change with play testing, it won't be hard now to change it back.

The two most promising leads for performance improvements are Cython, a Python-to-C converter, and OpenGL. In the first case, that is something best saved for much later. In the second, the PyOpenGL library has a bug that prevents its installation under Python 2.6, so that's going to have to wait for now.

Today is mostly devoted to graphics work. I'm considering character designs for the main guy. I might temporarily import Aurora from Mayflight to fill the role, but she's 16x16 anyway. She'd make a good placeholder though.

In Profundis Progress (6/30)

Ah, progress! Basic support for larger tiles is in now. In fact, I might make the tiles much larger but just display scaled-down versions during the game, which would allow for smooth scaling of the playfield between scales. I used a trick like that fscaling smooth-scaling map screen in Mayflight (a particular favorite effect from that game). I'm not sure I can do this with Pygame however -- one of the reasons I tried moving to pyglet is it can blit images while scaling them in transit. Pygame only scales through the creation of a new surface, which must then be blitted to the destination, which might be too much overhead for me.

One benefit of using larger tiles is the game doesn't have to blit so many images, which is worth another modest framerate increase. The disadvantage, of course, is that less of the world fits on the screen.

In design news, I've started planning out in detail how the random substance properties will work. There are planned to be three kinds of substances: liquids, gases and solids. Liquids are flowing materials like, but not limited to, water. Sand is also a "liquid," for instance. Gases hang in the air and slowly spread out. Both may or may not end up having pressure support, if I can figure out a relatively inexpensive implementation (which is one of the reasons I obsessed over finding ways to speed the game up for a while). The third type, solids, are basically stone walls, which are more reactive in their properties.

So what do I mean by a random substance? The idea is that, at the start of the game, in addition to substances with fairly obvious attributes, there would also be some with unknown properties that must be deduced through observation and experimentation. From the promo video, this is like throwing a torch into a pool of unknown liquid to see if it's flammable. My idea is that the first world the player explores in a campaign would have little or no random substances, but each one after that would have a little more randomness. Another idea, which I mentioned in an interview, is that the planets explored by the player would be divided into solar systems, and each system's would have its own "elements." So, different planets would have different maps and layouts of substances, but the behaviors of those substances would be the same throughout.

If I do this right it could be one of the most interesting aspects of the gameplay. I find that many of my favorite roguelikes are the ones with randomized equipment, which turns figuring out what your stuff does into a logic game. However, it is also true with many of those roguelikes that the identification game isn't as interesting as it could be, or is only a real requirement in the early phases of the game, as once the player learns what everything is there is nothing left to learn.

Anyway, all this is still some time off in the future. I'm going to work on the graphics a bit and fix up the platforming engine before then, I think.

In Profundis Progress (6/29)

Current work is devoted to increasing the tile size and consolidating the tiles into atlases on disk, for ease of editing. Along the way I'm putting in the beginning of different stone types, which will have random properties like gases and liquids.

In Profundis Progress (6/27)

A bit of real progress!  Profiling revealed that, anomalously, most of the processor time was being taked up by one particular lambda.  I've avoided lambdas in the code mostly, partly because I knew the various machine code compilers don't tend to optimize them well.  It didn't help at all that this lambda was being called in the coordinate sort function as a key.  Profiling revealed that this lambda was being called millions of times over the course of a short run of the platform engine.

I replaced it with a class method, and the result is a good 4-5 fps increase.  Further optimization is needed I think, but it's a solid improvement!

In Profundis Progress

So I made a backup copy of the code and spent an hour or two trying out Cython.

Cython (which is different from CPython) is another compiler for Python.  Unlike Psyco or PyPy, it isn't a JIT compiler; it turns Python code into C, and it works on the module level, which requires some elementary reorganization to use.  Not only does it promise some basic speed increase just from the conversion, it is possible to supply some explicit type information to optimize the resulting code still further.  Since that type info is added to the language in the form of new keywords, that has the disadvantage of making the code no longer runnable as pure Python.

The big surprise with Cython is: I got it to work.  There are no Python features in my code that it doesn't support.  This, by itself, is a surprise, and after my latest failures in getting outside code working it gives me a warm feeling to use one of these tools and not be faced with any error messages that I can't fix myself.

Unfortunately, I didn't get any observed speed benefit from it; the framerate actually dipped a little bit.  Still, the relative lack of impossible-to-fix errors is encouraging, and I might play around with this a bit more later.

In the meantime, the thing I can do to improve performance the most is probably to either optimize the cell sort or switch to a different cell calculation selection mechanism.  That's what I'll be working on tomorrow.

In Profundis (6/26, 2)

Profiling has revealed that it's not so much the sort that's taking all the time as the lambda function that determines distance from the player's location.  Hmmm.

In Profundis Progress

Actual work has been light the last few days, as I investigate a couple more possibilities towards speeding it up: eliminating the sort portion of the update loop (mentioned last night), checking into Cython and PyPy in terms of compilers, and trying to get Python's profiling tools to obey me -- I find they are not as simple to use as the documentation on the Python site implies.

In Profundis Design (6/26)

Recently the queue for processing cells changed.  Before it was a simple nested pair of for loops that went through every cell in a rectangular region slightly larger than the screen.  After it uses a substantially more complicated system, where the changed cells from the previous frame are added into a list, the list is sorted by the distance to the player's location, and the first few hundred closest cells are handled, with potential changes from those cells added back into the main list.

This is good for some things, and bad for others.  It makes the preliminary world simulation (two of the generation steps) much faster, and means, if there's nothing much happening near the player, we automatically simulate progressively further areas of the map.

However there is a problem, and the more experienced programmers probably already see what it is.  It's that sort.  Before the change it didn't matter much how large the world was; if it was 2x2 or 100x100, it took the same time to calculate.  The sort operates on every change-flagged space on the map, and it makes a function call for every flagged cell on every frame.  That's roughly 7,000 cell coordinates in a newly-generated terrain world, every frame.

You can actually see for yourself, a bit, how much time the sort takes up.  When you start playing in the prototype, if you turn on the debugging HUD you'll be told the framerate.  On my low-end, single-core laptop, I generally get framerates in the mid-to-high 20s.  The simulation tends to find an equilibrium over time though, and as water pools fewer nearby cells are calculated each frame, meaning the range that gets calculated expands, ultimately decreasing the size of that sort list.  The result is, over time, framerates slowly increase.

This is my primary development concern at the moment.  I'd like to increase framerates by a bit more so that I have more processor time to do effects (maybe related to making fluids less blocky), increase the complexity of the simulation, and perhaps draw a background layer.  Right now I'm getting a handle on Python's profiling tools to make sure my suspicion is correct, but even if it turns out not to be THAT bad I can't help but think I could design this better.  Maybe going back to the rectangle, but increasing its size as visible activity decreases.

In Profundis stuff

$10-and-up backers have had messages sent out informing them of how they can obtain a "playable" copy of the prototype I made the videos from.

More research is being done while I work on the game, in the continued hopes that I can break the code free from reliance on Psyco, a dead-end in terms of Python development.

In Profundis Question

Talking it over with Simon a bit, this business with chasing C++ may be taking too long.  I may go ahead and develop the Python version some more, do some releases of that, and see if it ends up being good enough.  In the meantime I'll read up on C++ and try to get the kinks out of my head concerning it.

The thing about that I'm most worried about is it might turn out to be a dead-end.  A reliance on Psyco almost certainly is; it's been dead in the water since 2009, and hasn't had an official release since 2007.  Going with its successor PyPy is perhaps possible, but I've so far been unable to get Pygame working under PyPy.  I'm pretty sure it works since PyPy has an important profiling tool that uses Pygame for graphics output, but that site is weirdly lacking on details for installing it.  The Windows installer for Pygame refuses to recognize PyPy as Python 2.7 and easy_install seems to work at first but in interactive mode refuses to import it.

Any of you have experience with PyPy?  I apologize this is all taking so long.  I never promised you an instantaneous rose garden, although I dare say you deserve something soon.

In Profundis Progress (6/22)

Did I make a mistake in picking C++?  I know it sounds like I'm dithering between languages, but coming to it from something as luxurious as Python makes C++ seem really... what's the word I'm looking for... Aspergery?

In any case, the development timeframe for this thing is such that I don't think I can become proficient enough in C++ quickly enough to make effective use of it.  But fast-enough Python code relies on either using an out-of-date library (Psyco ceased development with Python 2.6) or a special interpreter which is probably not compatible with the breadth of Python modules I require (pypy).  Further, packaging Python into executables seems to be generally hacky.

I just want to code in peace, and I can't seem to write anything right the first try in C++.  Someone in comments a while back suggested Java.  Maybe I should consider that.

In Profundis Progress (6/21)

Still trying to muddle through.  I have a question for you more experienced C++ programmers.

The Python version is implemented procedurally.  That is my way of saying, there are multiple loops.  In this way, program state (title screen, menus, gameplay, edit mode, etc.) is basically kept in the instruction counter.

It's a comfortable way to program, basically how I would implement things back on the old Commodore 64, but is hard to convert to an event-driven paradigm, which is why the pyglet and, maybe, the C++ versions are bothersome to develop.  (Well that and the fact that C++ is maddeningly finicky.)  I'm trying to design this one using a master event loop that switches off depending on the program state.  Which is not how I ordinarily would do this, but it seems to be what is expected from a more professional program, and when it comes time to make an iOS version would potentially pay off there.

Or will it?  I'm making assumptions that might not be borne out by practical considerations.  It is an alien way for me to code, a method that requires a bit more forethought than I'm used to.  In the past, I've coded in a kind of iterative way: I know generally where I'm going, I think out what will be necessary, then I try to get a bit of the way each time.  This seems like something more architectural.

Or is it?  Does this make sense to any of you?  Is there a better way to do this that I haven't considered?

In Profundis Progress (6/18)

I continue to bash my head bloody against the brick wall of C++.  I have an annoying problem where I try to define a variable in a header, and other files that include that header complain about multiple definition errors despite the presence of a header guard.  This is annoying for me since one design pattern I frequently use is to use one file as a place to store essential global values, like gravity and friction values, for easy reference/changing later, but if I just use that file for those variables the linker balks.

I asked a programmer friend for help and his suggestion managed to resolve the problem (basically to move the values into a different source file), but the reason he gave that it works uses a development metaphor that is nonsensical to me, I don't grasp its context, and so I'm resigned, for the time being at least, to doing it just because it works, not because I understand it, and that makes me nervous.

As for the stuff I'm implementing, I'm working on the data structures used for holding graphics data right now.  Fun!

In Profundis Progress (6/17, #2)

I hate C++ a little less.  It still has much to atone for, however.

In Profundis "progress" (6/17)

Grar, I hate C++!

In Profundis progress (6/15)

After fixing the sight bug, I broke ground on the C++ port.  Getting in the way is the fact that although I'm generally familiar with C-style languages I've never done a major project in it or C++ before, meaning I'm going through that phase where every file has 20 different errors, sometimes due to a problem halfway up the page from where the compiler caught it.  In other words: uphill climb.  Still, going faster than I expected.

I am considering releasing the Python version as a curiosity for the alpha-level backers to mess around with.  It is kind of fun to plop down liquids and sand in edit mode, but it looks real primitive and isn't a real game at this point.  What do you guys think?

In Profundis Progress (6/13)

The source has been reverted back to the Pygame version.  The spectre of performance continues to hang over my head though.  I am beginning to plan the switch to C++.

Back to normal design issues, the switch a couple weeks ago to keeping a cell change list broke vision, since visibility propagation was handled through the CA engine.  Today I worked on fixing that.  Another bug has come to light (ha ha) in the process, which is causing some out-of-vision cells to show incorrectly. Fixing that is the immediate objective.

In Profundis Progress

Ah, the problem that has been holding me up for almost a solid week now has been tracked down.  It was a lot of sweat to discover this fact so I'm going to use very large text.  It is this:

PYGLET IS NOT COMPATIBLE WITH PSYCO

When I tried attaching keypress events to the window they became unreliable.  Removing the call to the psyco compiler causes them to become reliable again.

I feel like I've wasted a week but no matter, I know there isn't some bizarre phantom bug in my code now. 

Oh, if you only knew the effort I spent on this.  The energy I've poured into this.  On looking at every part of my code, of creating test situations, of slowly going between the test situations and my code point for point and finding where the differed, of creating MORE tests.

I feel like curling up in a ball on the floor right now.  But at least the problem has been solved.  In Profundis is special in that it's not just the display that's the biggest processor cost, the program does some heavy calculation behind the scenes.  This means I can't abandon psyco, and that means I must abandon pyglet.


So long, pyglet

In Profundis creaks along

It's been a while now and I still don't have a lot visible to show for it.  Honestly, the switch to pyglet has been going more slowly than I expected.  The thing is, to get the most performance out of it I have to redesign the main loop to be entirely event-driven, and that requires learning pyglet's callback system, the examples of which most frequently use decorator, which are proving unfriendly to circular imports....

Bleaah.  It is coming along, it's annoying frustrating work but it's happening.  And I'm doing it for more reasons than just performance -- being fully event driven will make the design of the thing a lot more professional, and make it easier to adapt to mobile platforms, which mostly use entirely event-driven paradigms anyway.  And pyglet is capable of some special effects, which might help me out in ways related to the oft-commented blockiness of the water effects.

So it's coming along.  I figured you all deserved a status report regarding the game.  This is getting reposted on Kickstarter so it'll go out to everyone.  You should also all know that the game has a placeholder website not at http://www.inprofundis.com/ -- it's just a barebones Wordpress thing at the moment.  I'm going to continue to use the Coin Door Interlock blog for now though for development reports, though.

More news as it happens....

In profundis Progress (6/1)

More boring stuff.  The pyglet changeover continues.

Someone in comments in the last one asked if I was sure that changing to pyglet would be a net performance improvement.  I don't know for sure; he is right, the reputation I have of it is largely anecdotal.

However it does look like, as far as rendering goes, pyglet is rather more flexible.  Or at least the way I was doing it before was less flexible.  Either way, I'm taking the opportunity to change the rendering system a bit.  Which is, again, not much to show.  (Well this is a development diary, heh.  A lot of development, especially in the early phases, looks like this.)

In Profundis Progress (5/28)

Work on switching over to pyglet continues.  Also redoing the graphics for performance improvements.  Not particularly interesting I know.  Right now most of the work has been infrastructure, gruntwork, and preparation for... the future....

In Profundis Progress (5/28)

Work continues, although it hasn't been straight development work. I've been reading up on the Pyglet graphics framework and it looks like I should be adapt the code to it without a huge amount of trouble. This means better performance, meaning it'll work on more people's computers without stuttering, which has been a concern. My development machine is a very cheap laptop, but that's on purpose; if it works on this, I reason, it should work on most machines.

I've also gotten the hosting and domain registration for the game's website, which is where the alphas and betas will go up. It's just a placeholder page for now, but when it's ready you can point your browsers to http://inprofundis.com/.

Website coming soon

I registered the domain name and hosting today.  Worried about the cost, but I don't think free Google hosting will suffice for password-protected file distribution.  Or will it?  More to come.

In Profundis Progress (5/22)

I'm back baby!

So, here is the thing that's been consuming my development time lately.  Cellular automation for use in real-time games frequently has to be heavily optimized.  When you're cycling though every cell of a hundreds by hundreds array every frame, then even the overhead from those cells in which you just check contents and decide nothing needs to happen is significant.

My current strategy for negating this is to not use a big loop that iterates through every cell.  Instead, I keep a list of every cell that _might_ change this frame.  I calculate those and, any that do change, I add adjacent cells to the list for the next frame.

It's much MUCH faster, but it's worth noting that this is entirely because, for most of the game world, large portions are static between each frame.  Stone is largely inert, so we don't need to worry about it except how other things react to it -- they are the initiating factors in that, so their routines handle the work.  Empty spaces (that don't contain weird gases) are similar.  These two cell types are 90% of the world.  By removing them from consideration, it's worth a huge performance gain.

A huge gain, but it turns out, not huge enough.  I remind you that, before, we kept performance under control by only updating the screen and a short distance outside of it.  Under this system we update the whole world, and factoring in all of the cells that CAN change, it's a bit too much to reliably handle.

There is a way around it.  We sort the update list, using one of the handy-dandy, heavily-optimized Python sort functions, with the sort key being a simple Cartesian distance from the player or cursor.  Then we can just pop off the first few hundred or so every frame, and leave the rest for the next frame.  If the player never goes anywhere near them they won't get handled, but as soon as he does they'll enter consideration.

It's a cheat, but real-time game development is basically a convoluted series of cheats, a set of makeshift concessions to processor speed and memory limits.  Now that we have multicore machines and multi-gigabyte memory spaces this fundamental fact is sometimes obscured, but it doesn't change it.

In Profundis (lack of) Progress (5/19)

Been working on rewards so far and trying to work with Kickstarter's survey system to get necessary information for fulfilling them.  Will let you know when work on the game itself occurs.

The project make the goal

If I were the kind of person to say things like "woo-hoo," I'd be saying it now.

Maybe I should give it a try.  Here goes:

"Woo-hoooo!"

Hm, unfulfilling.  Especially since the hard part starts now.

But thanks everyone for believing in my idea, and the project.

In Profundis progress (5/13)

The biggest change, which is so big that I've been taking a break from development for a few days to consider what it means, is to the way the sim looks at which cells to calculate.

Originally, it just iterated through a loop through the whole world, from bottom to top, examining each cell and deciding what should happen there.  That took a long time.

More recently, it'd just been updating a frame around the currently visible portion of the world.  That's  much faster, but causes certain bits of weird behavior in some circumstances.  For example, if enough water pouring off screen in a place eventually it'll fill in the cells in the no-update region off the bottom of the screen, and then it'll pool on top of that.  If the screen then scrolled down somewhat, it'd suddenly all start falling as the "plug" dissipates.  Similarly, water flowing off-screen will eventually act as if it's reached a wall and begin pooling, but if you move forward it'll start flowing out again.

The new system is a rather more radical change.  Now the game has a list of cells to update, compiled originally by scanning through the world.  Cells that are mostly inert, like walls and empty spaces, don't make it into the list, and thus no cycles are spent on them.  When an operation is performed on a cell, the game adds nearly adjacent cells to the (initially empty) list of cells for the next frame.  Those calls are "activated."  Since the most distance away a cell can affect the world on a single turn is one space, we never have to add too many cells to the list.

If a space containing some possibly-active element, such as a liquid or boulder, does nothing this turn, it is left off the list for the next frame.  The simulation doesn't even consider it until something happens close enough to it that it might be upset.  Minecraft does something similar with its sand; sometimes you'll find a place where the world generates sand blocks that are unsupported, or even hanging in air


When the player does something that affects the game world under this system, the nearby cells have to be added to the update list in case there's any simulating to do.  That's what causes Minecraft's sand to suddenly realize that gravity exists and begin falling.

In Profundis progress (5/8, part 2)

Did some more work today on speeding up the simulation.  Before, the cells that were calculated were all of them within the borders of the screen plus a small area outside of it.  Current work is towards switching to a system where changed cells in a given frame mark cells adjacent to them for calculation in the next frame.  The idea is that changes don't happen randomly; they're always spurred by other changes.  So, there's no need to, say, check a boulder every frame to see if it should roll.  If it's stable on a given frame, it won't be checked again unless the space around it is disturbed.

Since most of the game world is fairly static each frame, the result is potentially much more efficient world calculations, meaning being able to simulate more of it each frame.  The biggest drawback is that gases are unsuitable for this system, since they'd be constantly spreading bit by bit regardless of whether the neighbors change.  Even so, it's a promising direction for development.

In Profundis progress (5/8)

Been going very slow due to frustrations with the Kickstarter project and trying to work on another article about it, to get up whatever little publicity I can in the few days until the project inevitably fails.  I don't think the game will be abandoned when the project fails to make its goal, but it is going to take me longer, much longer probably, to work on.  As such, it'll probably be longer between posts here unfortunately.

Gamasutra article up

The Gamasutra article offering a rough view of the cellular automation liquid algorithm in In Profundis is up.

In Profundis progress (5/3)

Pushing boulders now works.

Additionally, there is now a faint ghost cursor that accompanies your character as he wanders around the world.  Not like the one you've seen in videos of edit mode, an understated one.  It acts as a kind of context-sensitive prompt.  For the demo, the object it covers represents the action you will perform when pressing the action key.  Empty space next to a wall: insert piton.  Empty space away from wall: fire airgun.  Overlapping wall: pickaxe.  In fluid or sand: shovel out some material.

The cursor can be adjusted a bit; if you hold the up key, it moves into the air above your head.  If you crawl, it goes down below your feet.  People who have played the classic Tecmo puzzle game Solomon's Key will get the idea, regarding which blocks you can affect at a given moment.

In Profundis design ideas (5/1)

Work on the game has been slow this weekend (which is to be expected, since those are the days I spend 1. delivering pizzas, and 2. cursing fate), but I figured it'd be useful and perhaps interesting to discuss some of the ideas I have for the game.  These are all things I have been playing around with and may, or may not, someday implement.  Perhaps (or perhaps not) someday soon.

#1: Basic equipment

Okay this one is pretty set-in-stone, and in fact is the current feature I'm working on, hopefully to be finished before the Kickstarter project concludes.  In Profundis is intended to be a kind of game where just getting around is a large part of the challenge.  Your guy's abilities by himself are pretty much limited to jumping, crawling, swimming, pushing, and climbing -- and climbing only works if there is a cracked stone surface to scale, which are not usually common enough to rely upon.  Everything else you can do relates to your equipment in some way.

In the real game equipment will be handled from an inventory screen.  (I have persistent recurring visions of a roguelike-style inventory screen, but that would probably scare off a lot of players.)  Each item would be rated by weight, and you could take a certain amount with you on an expedition.  The various bits of kit you take with determine your abilities.  In the meantime, for testing, I'll just enable unrestricted use off all the items.  I intend to make this version of the game available as a playable alpha.

If you find that you've brought inappropriate stuff, you can go back and reload, but at a cost.  Living on an alien planet is not without its expenses.  Every expedition takes at least one day of game time, and thus a certain amount of funds.  The idea is that this cost slowly increases the more days you spend.  So, even if you seem to have inappropriate stuff with you, you still want to make what use of it that you can before you return.

The basic pieces of equipment planned are:

Pitons: These are basically simple metal spikes.  By themselves they are useless.  They are intended to be used with the Hammer or Air Gun.  Once inserted into a rock face, they can be used to climb.  Pitons remain in place after being inserted, even across many days, so as you spend more time at the site you gradually transform hostile terrain into something traversable.  This is a general theme for the game, how you reshape the cave to your purposes.  But pounding spikes into the rock can also cause cracks to form, and this is another theme: remaking terrain to suit your purposes is dangerous.  Sufficiently cracked walls can break, and expel whatever material was built up behind it into your face.  Once hammered in, these pitons are generally unsuitable to later reuse, but that's okay as pitons are one of the cheapest pieces of equipment you have.

You can also tie a rope onto a piton, either in your inventory or one that's already in the rock.  More on that below.

Hammer: One of two ways to insert pitons.  The Hammer is not consumed and doesn't wear out, making it a rarity among In Profundis equipment.  It can be used while climbing, so as long as you can get hold of the wall, you can continue pounding in new pitons on your way up or down the surface.  The problem is that you can't get around overhangs with just the hammer; pitons hammered into ceilings are not suitable for climbing.  For that you'll need to use something else.

Air Gun: Something else like this.  Powered by consumable cans of compressed air, this device can be used to shoot pitons out towards other surfaces.  I'm still figuring out how this will work; I'm thinking about a targeting system where you aim a cursor.  Firing a single piton is not very useful, but firing a piton with a rope tied to it is quite handy.  It'd be nice to use a full physics simulation for the ropes.  Nice, but no, I don't think it's going to happen; we're already consuming a lot of processor time with the cellular engine, and I don't think we'd actually get that much use out of swinging ropes, even if they would be bloody flash to see.  The way ropes are projected to work is, when the piton strikes the rock surface, the rope will unfurl downward ten units from where it hits.  If you can grab onto that rope (it's freely climbable at this point) and get to the end of it, you can tie another rope to the end, and in this way descend until you run out of rope.

Tradeoffs to the air gun: the rope weighs the piton down so the range, while not short, isn't very long.  It is long enough that, unless you have an unusually powerful light source with you, you'll have to rely on the automap to fire at the appropriate location.  And if something breaks the rope the rest of it, below that point, falls.

Air Can: These power the air gun.  There are other uses projected too, relating to the gas pressure system I've got planned, but in the meantime they'll just serve as gun "fuel."

Flashlight: Abstracted into an increased viewing range.  In the full game this will have batteries that can die.  Being without a source of light is a big problem; fortunately batteries have a good lifespan.  In the full game, if you run out of battery you can fall back on torches, but they produce less light and additionally carry other dangers.

Pickaxe: A very useful tool.  With this, you can smash tunnels into the rock, entering adjoining chambers or, in a pinch, digging a staircase back up to the surface.  But the tradeoffs are great -- you don't always have enough knowledge of what might be on the other side of that wall, the noise is considerable and prone to causing rockslides or alerting monsters, and worst of all, pickaxes are heavy and wear out over time.  Also, and importantly, it is impossible to get a good swing with a pickaxe while climbing.

That's all for now.  More ideas later!

In Profundis progress (4/30)

Have gotten noting done on the game over the past couple of days, partly due to working in the dreaded pizza mines (I have a sucky job), partly because of trying to think of new avenues to explore for pledges.

I'm thinking the best thing to do at this point is to get basic equipment working (pitons, ropes, pickaxe, airgun), without inventory management if need be (just allowing for infinite uses), and trying to make something playable on a rudimentary level.  If I can do this in a week, I can then maybe take it to TIGsource or Rock, Paper, Shotgun, and get mentions in those places before the project expires.

In Profundis progress (4/28)

Unfortunately the last few days progress has been fairly limited.  I tweaked graphics a little.  I did make a couple of Reddit posts, but that's about it.

Honestly, the Kickstarter thing has made me nervous beyond all reason.  I keep racking my brain for places to write to try to publicize the thing that might bring in some more backers, but so far I haven't come up with many that it seems likely would post a link to it.

The Kickstarter blog posted an entry recently saying that 90% of projects that make it to 30% funding then go on to get to 100%, but it doesn't seem awfully likely to me right now.

About In Profundis

IN PROFUNDIS (Kickstarter project) is a 2D cave exploration game in a very random world.  Not only is the cave highly unpredictable (as you can see in the below demo video), but it also has dynamic, flowing water, falling boulders, dangerous gases with random properties, and many other dangers besides.  It is somewhat realistic; your character's abilities are modest, more like what a real person could accomplish, but you can use equipment brought from base to help you get around: pitons, hammers, ropes, air guns, and more. 

Platforming demo:

In Profundis is inspired by the roguelike games in the high consequences for failure, the dynamic, mysterious nature of the randomized world, and the limited range of vision.  While some monsters are planned, the player's true foe is the cave itself.

If you went to an unexplored cave in the real world, how would you explore it it?  How would you explore it without getting yourself trapped, crushed, flattened, suffocated or lost?  What would you need to bring with you to ensure you could get out alive?

The Kickstarter page is here.  Please pledge today and help to make my game a reality!

In Profundis progress (4/25)

I've gotten the platforming part working acceptably for now.  I've added one more task to accomplish before making the new video and going on the ALL-OUT MEDIA BLITZ which honestly just includes posting to Reddit and submitting to Rock, Paper, Shotgun and maybe Tigsource.  But that's because it's one of the key features that I think will make In Profundis worth playing: the limited visibility routine.

When you can see the map around you on your screen filled it, the feeling is entirely different when you can only see the area right around you.  But just limiting what is shown to what you can see at a given instant, I think, is not fair to the player.

My compromise is a roguelike-style on-screen "map memory" that shows places you've seen before but can't now with a different graphics style.  The idea is to present those areas with a "hand-drawn" graphics set, like sketched with a marker, but for the time being they're displayed with schematic-style shapes with line shading.  The graphics aren't bad I think, but I need to work more on the visibility algorithm.  It works currently acceptably in real-time, in fact I'm often seeing a substantial time gain since I don't have to draw the parts of the map that haven't been seen yet.  But it's a little _too_ realistic I think; for gameplay's sake, I think it would be better if the player had a little extra visibility, maybe a little around corners.

Here's a screenshot, which is also the first public viewing of the little guy who's the placeholder I'm using for the player (he's crouched down in the middle of the screen).  Click for full-size: