Gonzo skill system trials

Rather than taking it for granted that my weird skill system idea will work as hoped, I'm taking a small detour to make a simple game testbed for it, so I can get some picture of what it'll work like in practice.  I am considering making this available to backers as a bonus, something to play around with.  That is, when it's written.

The past couple of days have been mostly spent designing it further, and looking into the capabilities of the IGraph software, which has a Python package interface, to this end.

Native AI, crawling

Slow going as I go through and try to figure out how to implement these things.  For native pathfinding, I'm considering a variation of the kind of pathfinding that ants use in the real world (and in fact is used in Will Wright's SimAnt).  Before that though is crawling.  Since the player is an entity in the world, although one composed of multiple tiles, crawling actually changes his shape.

In positive news, the interaction cursor works fairly well.

Crawling

The main character can now crawl.  It sounds like a minor thing, but now that the character is actually an entity in the cellular system, the system has to be able to resize entites, check to make sure there's room, and things like that.  The analogue in the platforming system is all those problems with handling wall embeds and ejects, which is one of the trickiest parts of programming a platformer, so trading these problems, which are now solved pretty much, for those, which I never could get to work right in the prototype's dynamic world with a sprite overlaid, is a good trade I think.

I'm going to have to modify the display engine now to display multiple possible actor states.  After that, more inventory work, then maybe actually putting in new entity types, like growing plants, the beginnings of an ecosystem, and other actors.

Multiple sized actors, menuing, interaction cursor

- Multiple-sized actors means, the creatures you interact with can be different sizes, to a single block, to human-size (three blocks high), to bigger.  This also means multiple-sized players, which will come in handy for things like crawling through low passages... or shrink rays?
- Menuing is working now.  The beginnings of inventories (both player and other) are in.  Item classes are progressing.
- The interaction cursor, which never did much in the prototype, is back, and has a wider array of movement than before and is more flexible, and easier to control.
- Up until now, world generation has been accompanied by flashing colors, just to indicate that something is happening and nothing has crashed.  Now it displays a log-style progress report.

Step by step, inch by inch....

Working menu

I got a menu working.  The difficult part of this isn't creating a menuing system it's making it work with the overall state machine, which affects display, setting up its own state machine, and so on.  I'm not real used to this frame-based, event-driven business, dangnabit.

Menu coding continues

Trying to track down some state bugs in menus.  Also, menus are appearing (in their clumsy current form), but the selector arrow doesn't appear to be moving.

Menus

I haven't done a whole lot of menus in a state machine driven kind of design before.  I keep trying to design something sensible and modern, and I keep noticing how my idea for approaching the problem are deficient.  The perfect is the enemy of the good, I know, but I'm just aiming for fairly acceptable at this point.  I'm going to describe what I'm designing right now, not just to stay public about things, but also because it'll help me keep it all straight.

At the moment the program tracks what state its in in a global variable called basis.  basis is actually an object that links to almost everything about the program's progress for easy saving and loading later, but it tracks overall state too.  Two of those states, and probably more in the future, involve menuing: the title screen, and the inventory screen.  Additionally, some of the menu entries may spawn additional menus according to what's chosen: does the player want to pick up an item, or drop it, or use it, or equip it?

So we actually keep a stack of menus; if we open a submenu, we create a new Menu object and push it onto the stack.  That part is fine.  But the code that actually does things with menus, that is still in the state handling routine that is called every frame, and so that's got to track which submenu it's in.  It seems like I should be able to design this in some better way (maybe the code for handling the menus should be part of the item itself?), and that possibility is distracting.

Hard drive replaced

Well it came down to it, and I had to replace the laptop's hard drive.  Fortunately full backups were already made, and it just took time to do it all.  A surprisingly large amount of time: I apparently have one of the few laptop brands that doesn't make the hard drive easily accessible, and I damn near had to take the whole thing apart!  Here's the step-by-step video I used to do it.  How tiresome.  Thanks a heap, Dell.

Recent progress

Still churning away, although admittedly slower than I like the past week and a half.  Most of that is working in the pizza mines, but it doesn't help that I'm in the part of the coding where I'm implementing the menu system.

One of the things I learned about doing over what I'm going to call the Hiatus Project is designing and coding state machines.  Most games these days are implemented so that they have to call a loop that draws the display every frame, and because this routine has to run regardless of what else in the program is doing, it means implementation of the main loop pretty much requires some sort of state machine, to do the displaying then continue working on whatever else the program has to be doing at that moment.  There are alternatives, but this seems to make the most sense here.  (Python has an alternative called co-routines that presented an alternate design strategy, but other than a generator used during map generation, I've avoided them, because they are a feature that Cython does not yet support, and I'm still aiming towards using that eventually.)

Well the menu system is part of that.  And another thing is ideally the menu should allow for being recursive -- if you've got the inventory open, for example, it'd be nice if there were submenus for determining what you can do with a specific item.  And what's more -- and this is where it gets a little weird -- it'd be nice if this happened while the sim is going on in the background.  That is, I am strongly considering having the inventory window open not pausing the game.

Why?  Well the thing is, while I'm not trying to develop an action game, it is still what I'm going to call a tension game.  Not about performing a sequence of controller or keyboard inputs as fast or accurately as possible, but more about, sometimes, having to decide what to do and get it out of your virtual pack in an expeditious period of time.

There will still be game pausing, of course.  And encounters with NPCs will necessarily pause the simulation portion of the game because it could be a subtle form of cheating if it didn't.  But the inventory window being open, I am considering, might not pause the simulation.

I'm still thinking through the ramifications of this.  What if something important happens while the window is open?  I'm considering popping up an alert message, or if it's particularly serious auto-closing the window.  Well, I admit it's a weird idea.  I might just forget about it.  But I figured I should say something about it.

Various things

Working on an assortment of things today, in addition to continuing to look into something to record the screen (I might eventually have to go with a gizmo that reads the computer's videoout), I'm working on the basics of the menu code.

At least things are moving along a lot faster now than they were!  Spirits are running kind of high regarding In Profundis, in fact.  Dum de doo....

Menus, more movement, and trying to record video

The thing I used to record the original intro was Microsoft Expression something-or-other, but that version also used Pygame and handled drawing in software.  Camstudio doesn't want to record OpenGL, and I've yet to find something suitable to replace it.

In other news, I'm thinking more about the skill system, native AI and pathfinding.  I'm considering using a system, if I have my algorithms correct, similar to that used in Will Wright's classic game SimAnt, where entities leave "trails" behind them that others can follow or ignore depending on what their priorities are.  But I'm still thinking about it.

While working tonight, the MST3K episode "Wild Wild World of Batwoman" was playing in the background.  It's not easy to code while cringing.  How on earth did that movie get made?

Screencasting

The program I used to make movies earlier I can't seem to find, and Camstudio doesn't seem to want to record OpenGL hardware surfaces.  Would anyone happen to know a good, hopefully free, screen recording program?

The Guy

I have a guy walking around now.  Movement is not smooth (in fact, he "steps" along a tile at a time), but that keeps with the conscious attempts to downplay platforming.  Despite this he can jump; it's too limiting not to be able to jump a bit.

If anything, he probably moves a bit too fast at the moment.

Player movement, adapting multitile objects for use as actors, planning native AI & pathfinding

The subject line says it all.

Multitile aggravations

I've gotten multitile objects working, and not becoming inconsistent as they fall or move around or anything.  It was a bit of a debugging minefield but they seem to be working now.  There are still some problems and inefficencies, but it's not something that should hold on moving onto the next item, which is, the player and movement.

Hacking away at multitile objects

Still working on it.  I think I'm getting close.  Ultimately, in addition to boulders and moving characters who are larger than a single tile, this will make possible Mr. Driller-style, large irregular blocks, that can support each other in interesting ways.  It is not entertaining work though, it has been dragging.  I continue to muddle through it.

I've been thinking a bit about objects being able to fall more than a single tile per frame.  Currently fluids, to make up for the fact that they are no longer slices in single tiles but full tiles themselves, can fall up to four spaces per frame.  I've thought a bit about generalizing that into a generic movement system, which would result in more realistic physics to a point.  It would not be a simple change though.  I think I'd be better served reimplementing character movement, NPCs, and starting on item and inventory stuff.

Back to the sim

I'm setting aside work on the skill system for now, I'm still not sure if will rightfully become part of In Profundis after all.  Multitile objects are back on the menu.

The tyranny of a weird idea

So the past couple of days I've been continuing to hack at the weird skill system I've devised, and....

And I'm not sure if it'll actually be used in the game.  It's interesting, I can't stop thinking about it, it's wholly novel and never seen before in any way in anything I've seen, and I've seen a lot of games, and...

...and it might be practically useless.  I've considered describing here in full, even posting my implementation for people to play with, which consists of a command line shell interfacing to a pickled Python dict database, but that would require spilling a lot of words just to describe it, and as a friend I tried to explain it to pointed out to me, it solves problems that not a lot of people see as existing.

I feel like I should make it available for people to at least look at, so it would turn up in web searches and not be lost to the world, but I'm unsure as to how.  This is my dev blog, I suppose it's the right place for it, even if it might not end up being completely related to In Profundis.

Uh-oh

My laptop was making weird noises a while ago and the machine was running slow.  There are only two things (besides speakers) in a laptop that could be making noises, and there was nothing in the CD drive.

The first thing I did after power-cycling it was zip up the whole project folder and stuff it into Dropbox, so that should be okay.  But checking on hard drive health indicates nothing wrong, SMART is perfectly okay, no errors found, etc.  If something DOES go wrong I have a backup laptop (it's slower but still capable) for the short run, and replacement internal hard drives aren't really expensive.  So it shouldn't result in any lasting trouble.  I hope.

An RPG-like skill system

Been working on an RPG-like skill system that may, or may not, end up being used for the player and NPCs in the game.  The specifics are difficult to explain, and when I've tried explaining them to people have gotten decidedly mixed results, so I'm going to pass on describing it for now.  It is a weird idea the implications of which I'm still trying to get a grip on.  It might not be satisfying to hear about things like this, but at least it is interesting work.

More on multitile objects, NumPy

Cellular automation works best on entities that are the same size as a single cell.  Once you start simulating things that are smaller or larger than that, things get complicated.

The smaller case is why, in the rewrite, I abandoned the previous tactic of simulating multiple fluid layers in a single tile.  As for larger, well, I'm implementing that in the form of multitile objects.

Everything that has an extent that goes into more than one tile is a multitile object.  That includes the player and other actors in the world.  It will also include things like boulders (which are going to be two-tile objects, at least) and even oddly-shaped solid objects that fall due to gravity.

There are a number of challenges to simulating these things.  For example, how to tell if one should fall or not?  You have to check beneath every tile in the object, but more than that, if the whole object falls every time any of its cells come up for a turn it'll end up falling much faster than other things in the simulation, and yet, each tile cannot just fall itself if the object as a whole is to keep together.  How about if something breaks apart a multitile object so it splits in two?  If you have two of the same kind of object next to each other, how will the system know they aren't part of the same thing?  And so on.  And there are some weird edge cases too, like two objects each "supporting" each other, so as a collective unit they both hang in mid-air.

Today's work has been in putting in more multitile object support.  In order to hurry lookup, we're using a weird system where a Python dictionary (known as a map or hash in other languages) is used to index them by their coordinates.  This trades off a little bit of speed for much reduced memory usage -- we don't have to store zero or None for tiles without multitile objects.

By the way, one of the greatest things about Python is the universe of third-party modules available.  Of these, one of the most prominent is NumPy, which is an optimized interface to a C-style array with several powerful optimizations applied to it.  In Profundis does use NumPy, but not, in the normal case, to hold the map at this time.  This is because the various overheads of Python don't interact with C well quickly; while arrays are held in a highly optimized state storage state in memory, Python converts each value into a Python object when read, and when stored back it converts it back, and this is an expensive operation.

What we do is, simply, use NumPy as a shortcut to creating the map grid, converting the whole thing into a list of lists easily in one method call (.tolist).  It's a bit wasteful in memory, but it's really a lot faster this way, the new emphasis on smaller regions saves some memory anyway, and compared to the large amounts of memory even inexpensive systems are shipped with regularly these days it's not projected to be a problem.

The system is set up that a flag can be set to use NumPy arrays for the world state instead.  This is because Cython can potentially make much better use of C arrays, so if we take the step of going with Cython as the implementation basis we've already got one foot in that direction, so to speak.

Not much today

Last night I tried out new verts for fluids but found the results poor.  Thinking about how to resolve that is most of what I've had the chance to do today.

What the game will be like

Today I've been spending time designing verts for fluids, so they don't look so much like bricks of water, which got me to thinking some more, through some process, about the direction of the game.

One of the little disheartening things that happened early on was when Terraria was announced, which immediately seemed to steal some of the niche that I was aiming to fill with In Profundis.  But it wasn't long before I started noticing that Terraria, while good for what it is, isn't really the kind of game I want to make.  And neither is Spelunky (although I think it's aces).  Not to try to bury either of those games, but I'm trying something different here, which is part of why I had a bit of trouble with direction some time back.  Let me see if I can explain what that is through two major differences.

* Terraria has an advanced crafting system, like Minecraft.  In Profundis will not.  I've always kind of looked down my nose at overt crafting systems, where you're collecting ingredients in order to make higher-order tools that are the things you really want.  There's usually too much trial-and-error, which is a purposeful waste of resources, to find out what the recipes are, unless you're just given them (in that case why not just give the item?) or you look up a FAQ (while I am a fan of Nethack, most games which demand looking up FAQs as an aspect of play are just broken).

* Terraria is a game with action combat.  In Profundis will have a kind of combat, the exact form of which I'm still working on.  It'll probably be a RPG-style separate screen, although it won't take the form of a traditional RPG battle screen.  This is because the other actors in the game are not necessarily your enemies.

In many games, the creatures you find are opponents to fight.  In Profundis is meant to be a game of exploration, and one of the things about exploration is the finding, and interacting, with other cultures, who may have alien viewpoints.  I can't think of a better way to present that than making them actually aliens.

Real-world explorers have had a somewhat unsavory history.  Often "explorer" has been basically a synonym for "exploiter."  And In Profundis is intended to be a game in which you are a treasure-hunter, which is often just another name for thief.  It is not my intent to make a game to wallpaper over the nature of the play.  In this game, you can do some pretty mean things; you won't be able to actually kill aliens, nor will they you, but you can hurt them, and you can steal things from them, as will they be able to do to you.  How you interact with them is up to you.

I am also aiming to not have the game presenting a false choice.  You are in a tight spot, having to find a good source of income to continue your explorations and do research.  You might be forced to steal to keep the game going, and that might hurt your relations with the natives.  But on the other hand, you might find a culture with a policy of giving visitors copious valuable gifts.  The plan is to have these things semi-randomized each game.  You might find a race that thinks you're doing them a favor by cleaning their territory of all that bothersome shiny trash.  You might find a race of aliens who seem to be giving things away, but actually expect trade, or favors, later.  You might find a race who views martial aggression as a sign of honor, and so if you attack them and take their stuff they respect you for it, looking down upon mealy-mouthed peacemakers.  And you might find a race of pacifists who just want to be left alone, and as their sign of this wave around their Surrender Sticks, which just happen to look like spears aimed at you.  Circumstances can make people act in ways which seem nonsensical to us, but they have their reasons.  And there might even be individual aliens with attitudes that don't match up with their comrades, just to throw you another curveball.

But if this is too random, then players will assume it's best just to shoot everyone anyway, assuming they're screwed no matter what they do.  So the idea is to have a list of 100 possible civilizations, and the player may have access to an item by which he can look them up and try to identify which each race he encounters might be, and what their attitudes are.  Of course, there will be potential for false matches.  And there will probably be other, completely random races, but most of them will be from the list.  The intent in this is to make the player work to figure out what is going on, in a way that, while not invulnerable to FAQs, still forces him to bring more of himself to the play than your typical time waster.

Well, that is my ambition.  I think the road towards that goal is clearer now than in the past, and I am running for it.  Who knows if it will work out that way.

A screenshot

Here's a screenshot taken from the current dev version in sandbox mode.  Fluids are still blocky, but stone is looking fairly nice.  There's an outline to stone that's hidden when submerged by fluids.

Note, for some reason the background in this image is showing as a light color instead of black, this is probably something to do with pyglet/OpenGL's screenshot function and the pre-frame screen clearing function not working as I expect.  Also note, unlike the project pre-rewrite, this is running full-screen, at 1366x768, without affecting performance.  It's currently running at about 10-15 frames/second.  I have an idea to speed that up further, but it might not be necessary, depending on how complex scenes get.

I'm not sure yet how to record videos of this.  The old Pygame/SDL version was slower, but by not relying on hardware acceleration it seems like it was easier to record screencasts.

EDIT: Ah, I see now.  The background color was showing up in the screenshot as transparent!  It actually doesn't look bad on Coin Door Interlock's dark background.

Two kinds of turns

One of the things I figured out while working on Employment Project during the hiatus was an alternate way of scheduling turns for slower cellular automation.  I don't know of anyone else doing this, so I figure I should mention it aloud in case others find it useful.  I am using it for slower effects in In Profundis, such as corrosion, erosion and gas diffusion.
In Profundis uses two systems for scheduling cellular "turns."  The first is a queue of active cells, which is continually updated each frame with the output from the previous frame.  This is based on the principles that:
- Each tile is mostly static.  Solid walls of stone don't require any processing, and big airy voids don't have a lot going on.  Similarly, bodies of water don't do anything generally unless disturbed, except on the surface.
- So, instead of performing a big loop iterating over every tile in the area, we keep a queue of tiles in a Python list, and iterate through that.  While processing, each tile changed adds its coordinates, as well as the coordinates of adjacent tiles that might have been "dislodged" by this event, into a new list.  Once the first list has been exhausted, we replace it with the new list for the next frame.

(Note 1: actually, we only give half the cells turns each frame, in a checkerboard pattern that alternates, so we don't end up with as many situations where turn ordering influences the sim.  It's not entirely successful at this, due to the vagaries of the ordering of the tile queue, but it's good enough.
Note 2: The list added to is actually a Python set, as it automatically excludes dupilcates.)

This vastly cuts down on the overhead of running a cellular system, and allows us to do it in ordinary Python at a reasonable speed, at the cost of being a little less certain about the outcome (since tiles that act earlier in the queue might create a situation that prevents tiles occurring later from acting -- if this were a problem we could use the bisect module to ensure the list is ordered), and for being less efficient if very large numbers of tiles change in a frame.  The first doesn't matter too much in practice, and the second would produce too chaotic a world to be interesting anyway.

I've talked about this before, I seem to remember.  What's new, however, is a second kind of turn, used for longer, slower processes.  Not every cell gets one of these "slow turns" each frame; indeed, about 1-in-64 of them do.  Since there aren't as many of these things happening, we can afford to look at every tile, and thus we can use this to simulate a wider variety of effect.  We'll get to that in posts to come.

Upcoming targets

This is more to remind myself of the next things to get done, but I'm making it public in order to be more transparent about this.

• Continue work on multi-tile objects, and creating/editing/simulating them in Sandbox Mode.
• Pausing.
• Implement the player character, implement the vectorized artwork I have in mind for him/her (user selectable), and movement.
• Start work on the Tool Layer, a second layer of the world that will hold items and player-placed objects and structures.
• Maybe the beginnings of NPC support.  (I have a longer post I'm working on for this, I have a specific idea of how native populations should be handled.)
• Now that terrain is interesting to explore again, I should put some more effort into varying it, making a larger percentage of a map explorable.
• Maybe start on multiple map exploration, and the larger structure of the world.
• Related to that: saving and loading.
• Player inventory.

 There, that's enough for the near future.  Onward to destiny!

Past three days

Little got done Tuesday and Wednesday.  Today I've gotten some stuff done for sandbox mode, with the most work going into the display of the edit cursor.  Yeah, pyglet still isn't extremely easy to work with, but at least most of the objects in the game are going to be drawn as part of the worldsim, and that's mostly working well now.

Note: I'm trying to post here a lot more now, not just to keep you guys appraised, but it helps me to stay on target.  I'm not just having to navigate the vagaries of the design of In Profundis here, but the contours and roadblocks of my own brain, some of which is difficult traveling.

More multitile stuff, reimplementing sandbox mode

Trying to post something here every day or two, to keep up that momentum.

Done more work on multitile entites.  I've been thinking, when a large entity moves, what happens to the things where it wants to move to?

Single-tile objects just swap positions.  Large objects could do the same thing, but it seems more realistic to try to move stuff out of the way.

 One recognized flaw with the current system for implementing falling multitile objects.  It's possible for two "convex" objects to be molded around each other so that both hang in the air, each "supporting" the other.  I'll probably handle this one by forbidding such objects.  There are other solutions, we'll see if it becomes necessary to go that far.

The sandbox is being reimplemented now as a debugging aid.

Multitile entities

To trade off no longer tracking fluid levels in a tile, instead watching "blocks" of water, in the rewrite we're simulating the grid with a little finer grain: what was one tile before is now effectively two tiles, stacked vertically.

This means that the boulders, from before, will have to have special support to handle being larger objects than the tile resolution.  Also, in the updated design, actors like the player will no longer exist as entities outside the grid, but in fact will be special types of tiles, which is a more "organic" representation that significantly changes a lot of subtle things.

The result is, I'm working on a new type of entity: the Multitile entity.

Multitile entites have one tile that's considered to be the "key" tile.  This is the one that gets turns, the only one that gets added to the cellular queue; the others are just along for the ride.  A previous project (one of the things I had to pick up for money, that has delayed In Profundis for so long) also used a cellular system that used multitile entities.  I learned a lot from that concerning how to simulate these kinds of objects, and the possible pitfalls.

So you see, I don't think all that time was wasted, I think it's a subtly wiser programmer that's working on the problem now.

GL_DEPTH_BUFFER

Turns out, if you want to use the Z coordinate to provide layering in OpenGL, you do have to explicitly use a depth buffer, it's not automatic, at least not in orthographic mode.  Just mentioning.

Fairly productive today

I switched the game's internal vertex format from two to three coordinates for vertexes.  The result is, I should be able to use the Z coordinate in the orthographic perspective to do tile priorities, which will allow for overlarge, overlapping tiles that don't cut each other off if they're on different planes.

"Solid" tiles, like rock, growing plants, and so on will use slightly-too-big tiles.  Fluids will too, but with a lower priority.  This should help to obscure (very slightly) the discrete, grid-based nature of the simulation.

Important note, that I've mentioned before but want to reiterate: the game is no longer going to be a platformer, but more of a real-time turn-based thing.
Reasons:
1. Platforming engines are difficult enough to implement without handling ejects and things like that in a dynamic cellular world.
2. That kind of action-based play I think will obscure the more thoughtful nature of the game.
3. I don't really want it to be, I want to try something new here.
4. I know roguelike implementation fairly well, and this helps to keep the game more easily achievable.
5. The baggage of having platforming was one of the things that bogged down the project before.  Let's keep it more achievable now, and not shoot for outer space in a solo project, the upper ionosphere is enough.

Adventures in pyglet: Quads

Got called off from the pizza mines tonight.  Should be worth some more time on the engine.

Switching the engine from rendering triangles to quads.  The way the new tile engine works is, it sets up a 2D array of vertex lists, all the same size, that represent the tilemap.  Deleting and creating new lists as needed, every frame, was too slow, so it leaves the lists in place and changes the vertex data when necessary, which seems to be fast enough, and flexible too; Z-order can be changed, I believe, by specifying a Z coordinate, which is important to the direction the art is taking.

The best way to do this would be to have one copy of each tile in the graphics card and just render each at need in the appropriate place, but difficulties in doing this seem to be a drawback to pyglet.

It's a somewhat confusing new way to do graphics for me, but I seem to be getting used to it.

Continuing with pyglet

After a lot of web searching and lying trying to beat the concepts into my head, I've finally gotten the basic tile engine working AND at a good enough performance.  Only thing is, now the HUD isn't appearing.  GAH!

EDIT: But wait!  I fixed it.  Maybe I can finally move on to non-mind-draining work, at last.

pyglet discovery

The current scheme is, a primitive shape (a triangle list) for each tile on screen.

My first attempt deleted the previous primitive and created a new one each frame.  Very poor performance.

Second attempt kept the tiles in place, but changed the vertices.  If the new tile has a different number of vertices from the first though, a resize operation has to be done.

It turns out that resize() is very expensive.  If I don't do it, and just make sure all the tiles have the same number of vertices, screen updates are much faster.

I'll have to design tiles around this, but the speed difference is encouraging.  I'm still looking into ways to improve performance.

What I was doing wrong

So, it seems in OpenGL there are modes the system can be in, with names like GL_PROJECTION and GL_MODELVIEW.  And each one is a "stack" of matrix transforms, through which the various objects you then create are converted on its way, each frame, into being displayed.

If you change the camera, you really want it to be in GL_PROJECTION mode.  What is more, if you don't call gl_LoadIdentity() before you actually change the camera, then it seems your camera's position is modified relative to its previous position instead of from the arbitrary origin of the coordinate system.

It seems like a simple mistake to make, and to fix, but my Google searches were not an appropriately-shaped key to unlock that knowledge.  Most of the tutorials I had seen had covered using OpenGL for 2D, but not for scrolling around, at least not in the way I was wanting to use it.  Because I didn't perform this necessary step I was getting strange behavior, like the camera skittering off when I tried to move it.

I had been partly misled by a tutorial that said you could set the projection system to gl_Ortho once, at setup, and leave it alone.  The thing is, the code ended setting the mode to MODELVIEW.  If you then change the camera after that, well, you get weird behavior.  Instead, I needed to change back to PROJECTION, reload the Identity matrix, and move the camera from there -- changing it back to MODELVIEW afterward.

Wheee.

Still working on OpenGL/pyglet

I had jerry-rigged tutorial code in place that *almost* did what I wanted.  But it was more fragile than I expected, and when I changed it to do the rest, it broke.

I've been having the disconcerting experience where, when in my web searching I find out something I've been doing wrong, I go in and fix it, and it causes a blank screen, or my coordinate space not being what I expected it to be.  So I go back in and do more searching, find out a little bit more about how OpenGL handles the matrix math to convert vertices into screen coordinates, fix what was wrong, but break something else along the way.

I hope the NSA enjoys seeing my Google search records as being every possible combination of "pyglet", "OpenGL", "orthographic", "glLoadIdentity", "pixel", "GL_PROJECTION", and "gluLookAt".

pyglet

After a lot of frustrating hacking, made not less so by the fact that Pygame and Pyglet have completely different display paradigms, I've managed to get Pyglet displaying rects from the game's simulation code.  I just wanted to make a post to celebrate that quiet victory.

Progress!

What has been going on lately?  This--

Because there was do much cruft in the code from early bad assumptions and later-added misfeatures, I started it over. That turns out to have been the right choice-- progress had been rapid again, the new cellular engine is *much* faster, and my second take at a random world builder produces much more interesting output.

I've been trying to merge a previous tile engine I had gotten working in pyglet with this, which has been going slower, mostly because piglet doors things so differently from Pygame.  Anyway, more soon....

Design realignments

I've been thinking hard about the future of the project and the best way to go from here.

1. I'm more and more convinced a platformer, or at least not the style the engine currently runs, is not the way to go.  The thing is that the cellular engine can change the world, from turn to turn, in ways that it's not simple for a real-time platforming engine to overcome, at least not one developed by a single guy.

Solution: Go with something more tile-based.  Not abandoning the real-time aspects, but instead of worrying about subpixel-level movement accuracy, instead have the player move from cell to cell.  That doesn't mean it would look exactly like a roguelike, we can have intra-cell animations, but under the hood movement will be discrete, one block at a time.  And by not trying to be an action game, the framerate consistency problems, searching for a solution to which having been the secondary cause of the development delays (the primary one being having to keep myself fed and clothed -- I live well beneath the poverty line) are no longer a problem.

2. The high-degree of substance property randomization isn't as interesting as I thought it would be, and the code is a snarl in any case and difficult to maintain.

Solution: Ditch the lot of that and simulate, for the most part, a number of preset, interesting substance types, that each might have zero or one special properties.  There are too many substances that it's just plain out useful to have around for gameplay purposes: water for swimming, sand for piling up, oil for burning, and so forth.  There are reasons World Of Sand games always tend to simulate those basics.

3. The game world generator doesn't produce interesting spaces to explore.  Oddly, the nearly completely random world builder that I had in the earlier versions made much more interesting terrain than the current maze builder algorithm.  This is actually a case where more randomness is more fun than less.

4. The game world is so large that we can't simulate the whole thing easily, which makes large-scale structures (such as flowing rivers) difficult or impossible to do accurately.

Solution: Try simulating a smaller world.  Also, I've come to realize that 8 levels of fluid in each cell is almost certainly too many.  We could get away with 4 or even 2 levels, and the result would be far less calculation.

5. There is still work to be done towards optimizing the cellular engine.

Solution: There are four possible approaches to the engine, which I've tried to different degrees over time:
A. The one that I've been using the most lately uses a "spiral" scheduling system; cells are calculated going in a square-cornered spiral around the current view focus (either the player or the cursor).  Advantage: if we want to calculate fewer or more cells due to real-time requirements, we can just stop whenever we want.  Disadvanges: iteration overhead, if we're not trying to be an action game, unnecessary, and iterates over every (well, every other) in its region regardless of need.
B. The obvious alternative is a rectangular region around the focus.  Advantage: the sim
plest scheduling system, so the least overhead.  Disadvantage: Iterates over every cell in region regardless of need.  If there's nothing in a cell it moves on fairly quickly, but it's still a matter of concern.
C. Use a list of "dirty" cells, compiled as the previous frame is worked on, and only calculate those spaces.  Advantages: We can calculate much more of the world each turn within the same period of time.  Disadvantages: More complex, and thus more fragile.  This is one of the reasons I moved away from this approach before since the increase in complexity wasn't matched by the performance gains.  With a smaller world, however, we might be able to use this approach to simulate the whole world... making the large-scale structures I really want to implement possible.  I think this is the most promising avenue.  Since we're using random world building, instead of one gigantic world we could instead explore a series of smaller worlds.
D. Use the weird column-based "sort" I've been working on and wrote about in the past few posts.  I think it might be viable eventually... but it is a weird thing, it's good for moving fluids vertically quickly, but we still have to iterate through those cells anyway to handle side-to-side fluid movement.  And it's so weird that I think it'll also be even more fragile, that is, vulnerable to being broken by other engine changes, than the dirty cell list system.  So I'm not using this technique for this project, it might be an interesting experiment some other time but now isn't the time for that.

Tonight I went into the code and switched it from the spiral turn scheduler to a rectangular field one.  I also tried running the game with a smaller world, and it seems to work okay if I make sector sizes smaller.  If I make fewer sectors I get an error, but so it goes.  It'll get fixed, and so will the next one, and the one after than.  Until it's done.

It's nice to think it'll be done.  You've all been waiting all this time.  I've been laboring under this weight, this sword of Damocles, for two years now.  I don't ask for pity or for forgiveness -- I'm not sure I deserve either.  I just say, I have not forgotten it over all this time.  I have never considered abandoning the project, for even a second, and I'm not going to abandon it now.  When I make a promise to do something, I get it done, even if it takes a while.  If you thought to yourself "sure, we've heard that before," I wouldn't blame you in the least.  I'm still trying to think of this in constructive ways, though.

Well, that's where we are.  Next up: getting the smaller world size to work, derandomizing the substance properties, and designing a better world generator.

Pyglet engine work

One of the things that has blocked the project for so long has been a shift of the engine from Pygame, which is easy to use but relatively slow because it does its rendering in software, and furthermore doesn't play well with any Python speedup module other than Psyco, to something else that's hardware accelerated and will work with PyPy.  That something else being: pyglet.

But despite the fact that it's been around for a few years now there's not a huge amount of stuff on the web about pyglet.  Most of what I've found has been its own documentation and a presentation called Stretching pyglet's Wings.  A few months ago I did a promising graphics test that taught me a bit about how pyglet puts together its images.  Now I've gotten something like that going in a tile engine, and it looks promising.

The reason for making the switch, and also for trying out weird cellular automatic schemes, is that, under the old system, I wasn't getting the performance that I was looking for.  When you're simulating only a portion of the world instead of the whole thing each frame, some things start to act funny at the edge of the simulation frame.  For something like Life that would be deadly, but here in practice it doesn't seem to cause huge amounts of problems so long as we spend some time at the start of simulation calculating over the whole map, in order to let fluids settle.

But it does have one problem that has always bothered me greatly, and that is, we can't have running "rivers" in the simulation between two distant points, because if one of the points is outside of the simulation frame the whole flow stops.  This means that most bodies of fluid the player encounters are static, which greatly limits the utility of having a cellular engine game in the first place.

Well performance-wise pyglet looks like it's got what it takes.  It can display a whole screenful of tiles nearly instantaneously.  Pygame takes a lot more time to do stuff like that unless you use tricks like only drawing parts of the screen that have changed, which is somewhat problematic for a scrolling game, and even more problematic for one where arbitrary tiles may change each frame.

pyglet

The most annoying thing about using pyglet, by far, is that nearly all the good sources on the library are either 1. the "Pyglet Programming Guide," available on the pyglet site in PDF form which is unwieldy and concerned with getting you started with the barest basics and little more, or "Stretching pyglet's Wings," which is a bunch of presentation slides with accompanying source code archives.

Maybe something to talk about soon....

Having to deal with a maze of personal issues, there hasn't been much to talk about concerning In Profundis lately.  (Some of them have had to do with negative comments on the blog here -- please, don't berate me for the time it's taking, I already know and feel awful enough about it as it is.  But I have to keep myself fed.)  But I might have something to tell regarding the "sort algorithm" I mentioned before, that I've been thinking very hard about lately concerning modelling falling and mixing fluids.  I think the idea is genuinely novel, and so maybe some of you might get some use out of it?  At the very least it might represent a substantive step towards getting this thing finished.  I still have tests to run with it though.