Multitile aggravations

I've gotten multitile objects working, and not becoming inconsistent as they fall or move around or anything.  It was a bit of a debugging minefield but they seem to be working now.  There are still some problems and inefficencies, but it's not something that should hold on moving onto the next item, which is, the player and movement.

Hacking away at multitile objects

Still working on it.  I think I'm getting close.  Ultimately, in addition to boulders and moving characters who are larger than a single tile, this will make possible Mr. Driller-style, large irregular blocks, that can support each other in interesting ways.  It is not entertaining work though, it has been dragging.  I continue to muddle through it.

I've been thinking a bit about objects being able to fall more than a single tile per frame.  Currently fluids, to make up for the fact that they are no longer slices in single tiles but full tiles themselves, can fall up to four spaces per frame.  I've thought a bit about generalizing that into a generic movement system, which would result in more realistic physics to a point.  It would not be a simple change though.  I think I'd be better served reimplementing character movement, NPCs, and starting on item and inventory stuff.

Back to the sim

I'm setting aside work on the skill system for now, I'm still not sure if will rightfully become part of In Profundis after all.  Multitile objects are back on the menu.

The tyranny of a weird idea

So the past couple of days I've been continuing to hack at the weird skill system I've devised, and....

And I'm not sure if it'll actually be used in the game.  It's interesting, I can't stop thinking about it, it's wholly novel and never seen before in any way in anything I've seen, and I've seen a lot of games, and...

...and it might be practically useless.  I've considered describing here in full, even posting my implementation for people to play with, which consists of a command line shell interfacing to a pickled Python dict database, but that would require spilling a lot of words just to describe it, and as a friend I tried to explain it to pointed out to me, it solves problems that not a lot of people see as existing.

I feel like I should make it available for people to at least look at, so it would turn up in web searches and not be lost to the world, but I'm unsure as to how.  This is my dev blog, I suppose it's the right place for it, even if it might not end up being completely related to In Profundis.

Uh-oh

My laptop was making weird noises a while ago and the machine was running slow.  There are only two things (besides speakers) in a laptop that could be making noises, and there was nothing in the CD drive.

The first thing I did after power-cycling it was zip up the whole project folder and stuff it into Dropbox, so that should be okay.  But checking on hard drive health indicates nothing wrong, SMART is perfectly okay, no errors found, etc.  If something DOES go wrong I have a backup laptop (it's slower but still capable) for the short run, and replacement internal hard drives aren't really expensive.  So it shouldn't result in any lasting trouble.  I hope.

An RPG-like skill system

Been working on an RPG-like skill system that may, or may not, end up being used for the player and NPCs in the game.  The specifics are difficult to explain, and when I've tried explaining them to people have gotten decidedly mixed results, so I'm going to pass on describing it for now.  It is a weird idea the implications of which I'm still trying to get a grip on.  It might not be satisfying to hear about things like this, but at least it is interesting work.

More on multitile objects, NumPy

Cellular automation works best on entities that are the same size as a single cell.  Once you start simulating things that are smaller or larger than that, things get complicated.

The smaller case is why, in the rewrite, I abandoned the previous tactic of simulating multiple fluid layers in a single tile.  As for larger, well, I'm implementing that in the form of multitile objects.

Everything that has an extent that goes into more than one tile is a multitile object.  That includes the player and other actors in the world.  It will also include things like boulders (which are going to be two-tile objects, at least) and even oddly-shaped solid objects that fall due to gravity.

There are a number of challenges to simulating these things.  For example, how to tell if one should fall or not?  You have to check beneath every tile in the object, but more than that, if the whole object falls every time any of its cells come up for a turn it'll end up falling much faster than other things in the simulation, and yet, each tile cannot just fall itself if the object as a whole is to keep together.  How about if something breaks apart a multitile object so it splits in two?  If you have two of the same kind of object next to each other, how will the system know they aren't part of the same thing?  And so on.  And there are some weird edge cases too, like two objects each "supporting" each other, so as a collective unit they both hang in mid-air.

Today's work has been in putting in more multitile object support.  In order to hurry lookup, we're using a weird system where a Python dictionary (known as a map or hash in other languages) is used to index them by their coordinates.  This trades off a little bit of speed for much reduced memory usage -- we don't have to store zero or None for tiles without multitile objects.

By the way, one of the greatest things about Python is the universe of third-party modules available.  Of these, one of the most prominent is NumPy, which is an optimized interface to a C-style array with several powerful optimizations applied to it.  In Profundis does use NumPy, but not, in the normal case, to hold the map at this time.  This is because the various overheads of Python don't interact with C well quickly; while arrays are held in a highly optimized state storage state in memory, Python converts each value into a Python object when read, and when stored back it converts it back, and this is an expensive operation.

What we do is, simply, use NumPy as a shortcut to creating the map grid, converting the whole thing into a list of lists easily in one method call (.tolist).  It's a bit wasteful in memory, but it's really a lot faster this way, the new emphasis on smaller regions saves some memory anyway, and compared to the large amounts of memory even inexpensive systems are shipped with regularly these days it's not projected to be a problem.

The system is set up that a flag can be set to use NumPy arrays for the world state instead.  This is because Cython can potentially make much better use of C arrays, so if we take the step of going with Cython as the implementation basis we've already got one foot in that direction, so to speak.

Not much today

Last night I tried out new verts for fluids but found the results poor.  Thinking about how to resolve that is most of what I've had the chance to do today.

What the game will be like

Today I've been spending time designing verts for fluids, so they don't look so much like bricks of water, which got me to thinking some more, through some process, about the direction of the game.

One of the little disheartening things that happened early on was when Terraria was announced, which immediately seemed to steal some of the niche that I was aiming to fill with In Profundis.  But it wasn't long before I started noticing that Terraria, while good for what it is, isn't really the kind of game I want to make.  And neither is Spelunky (although I think it's aces).  Not to try to bury either of those games, but I'm trying something different here, which is part of why I had a bit of trouble with direction some time back.  Let me see if I can explain what that is through two major differences.

* Terraria has an advanced crafting system, like Minecraft.  In Profundis will not.  I've always kind of looked down my nose at overt crafting systems, where you're collecting ingredients in order to make higher-order tools that are the things you really want.  There's usually too much trial-and-error, which is a purposeful waste of resources, to find out what the recipes are, unless you're just given them (in that case why not just give the item?) or you look up a FAQ (while I am a fan of Nethack, most games which demand looking up FAQs as an aspect of play are just broken).

* Terraria is a game with action combat.  In Profundis will have a kind of combat, the exact form of which I'm still working on.  It'll probably be a RPG-style separate screen, although it won't take the form of a traditional RPG battle screen.  This is because the other actors in the game are not necessarily your enemies.

In many games, the creatures you find are opponents to fight.  In Profundis is meant to be a game of exploration, and one of the things about exploration is the finding, and interacting, with other cultures, who may have alien viewpoints.  I can't think of a better way to present that than making them actually aliens.

Real-world explorers have had a somewhat unsavory history.  Often "explorer" has been basically a synonym for "exploiter."  And In Profundis is intended to be a game in which you are a treasure-hunter, which is often just another name for thief.  It is not my intent to make a game to wallpaper over the nature of the play.  In this game, you can do some pretty mean things; you won't be able to actually kill aliens, nor will they you, but you can hurt them, and you can steal things from them, as will they be able to do to you.  How you interact with them is up to you.

I am also aiming to not have the game presenting a false choice.  You are in a tight spot, having to find a good source of income to continue your explorations and do research.  You might be forced to steal to keep the game going, and that might hurt your relations with the natives.  But on the other hand, you might find a culture with a policy of giving visitors copious valuable gifts.  The plan is to have these things semi-randomized each game.  You might find a race that thinks you're doing them a favor by cleaning their territory of all that bothersome shiny trash.  You might find a race of aliens who seem to be giving things away, but actually expect trade, or favors, later.  You might find a race who views martial aggression as a sign of honor, and so if you attack them and take their stuff they respect you for it, looking down upon mealy-mouthed peacemakers.  And you might find a race of pacifists who just want to be left alone, and as their sign of this wave around their Surrender Sticks, which just happen to look like spears aimed at you.  Circumstances can make people act in ways which seem nonsensical to us, but they have their reasons.  And there might even be individual aliens with attitudes that don't match up with their comrades, just to throw you another curveball.

But if this is too random, then players will assume it's best just to shoot everyone anyway, assuming they're screwed no matter what they do.  So the idea is to have a list of 100 possible civilizations, and the player may have access to an item by which he can look them up and try to identify which each race he encounters might be, and what their attitudes are.  Of course, there will be potential for false matches.  And there will probably be other, completely random races, but most of them will be from the list.  The intent in this is to make the player work to figure out what is going on, in a way that, while not invulnerable to FAQs, still forces him to bring more of himself to the play than your typical time waster.

Well, that is my ambition.  I think the road towards that goal is clearer now than in the past, and I am running for it.  Who knows if it will work out that way.

A screenshot

Here's a screenshot taken from the current dev version in sandbox mode.  Fluids are still blocky, but stone is looking fairly nice.  There's an outline to stone that's hidden when submerged by fluids.

Note, for some reason the background in this image is showing as a light color instead of black, this is probably something to do with pyglet/OpenGL's screenshot function and the pre-frame screen clearing function not working as I expect.  Also note, unlike the project pre-rewrite, this is running full-screen, at 1366x768, without affecting performance.  It's currently running at about 10-15 frames/second.  I have an idea to speed that up further, but it might not be necessary, depending on how complex scenes get.

I'm not sure yet how to record videos of this.  The old Pygame/SDL version was slower, but by not relying on hardware acceleration it seems like it was easier to record screencasts.

EDIT: Ah, I see now.  The background color was showing up in the screenshot as transparent!  It actually doesn't look bad on Coin Door Interlock's dark background.

Two kinds of turns

One of the things I figured out while working on Employment Project during the hiatus was an alternate way of scheduling turns for slower cellular automation.  I don't know of anyone else doing this, so I figure I should mention it aloud in case others find it useful.  I am using it for slower effects in In Profundis, such as corrosion, erosion and gas diffusion.
In Profundis uses two systems for scheduling cellular "turns."  The first is a queue of active cells, which is continually updated each frame with the output from the previous frame.  This is based on the principles that:
- Each tile is mostly static.  Solid walls of stone don't require any processing, and big airy voids don't have a lot going on.  Similarly, bodies of water don't do anything generally unless disturbed, except on the surface.
- So, instead of performing a big loop iterating over every tile in the area, we keep a queue of tiles in a Python list, and iterate through that.  While processing, each tile changed adds its coordinates, as well as the coordinates of adjacent tiles that might have been "dislodged" by this event, into a new list.  Once the first list has been exhausted, we replace it with the new list for the next frame.

(Note 1: actually, we only give half the cells turns each frame, in a checkerboard pattern that alternates, so we don't end up with as many situations where turn ordering influences the sim.  It's not entirely successful at this, due to the vagaries of the ordering of the tile queue, but it's good enough.
Note 2: The list added to is actually a Python set, as it automatically excludes dupilcates.)

This vastly cuts down on the overhead of running a cellular system, and allows us to do it in ordinary Python at a reasonable speed, at the cost of being a little less certain about the outcome (since tiles that act earlier in the queue might create a situation that prevents tiles occurring later from acting -- if this were a problem we could use the bisect module to ensure the list is ordered), and for being less efficient if very large numbers of tiles change in a frame.  The first doesn't matter too much in practice, and the second would produce too chaotic a world to be interesting anyway.

I've talked about this before, I seem to remember.  What's new, however, is a second kind of turn, used for longer, slower processes.  Not every cell gets one of these "slow turns" each frame; indeed, about 1-in-64 of them do.  Since there aren't as many of these things happening, we can afford to look at every tile, and thus we can use this to simulate a wider variety of effect.  We'll get to that in posts to come.

Upcoming targets

This is more to remind myself of the next things to get done, but I'm making it public in order to be more transparent about this.

• Continue work on multi-tile objects, and creating/editing/simulating them in Sandbox Mode.
• Pausing.
• Implement the player character, implement the vectorized artwork I have in mind for him/her (user selectable), and movement.
• Start work on the Tool Layer, a second layer of the world that will hold items and player-placed objects and structures.
• Maybe the beginnings of NPC support.  (I have a longer post I'm working on for this, I have a specific idea of how native populations should be handled.)
• Now that terrain is interesting to explore again, I should put some more effort into varying it, making a larger percentage of a map explorable.
• Maybe start on multiple map exploration, and the larger structure of the world.
• Related to that: saving and loading.
• Player inventory.

 There, that's enough for the near future.  Onward to destiny!

Past three days

Little got done Tuesday and Wednesday.  Today I've gotten some stuff done for sandbox mode, with the most work going into the display of the edit cursor.  Yeah, pyglet still isn't extremely easy to work with, but at least most of the objects in the game are going to be drawn as part of the worldsim, and that's mostly working well now.

Note: I'm trying to post here a lot more now, not just to keep you guys appraised, but it helps me to stay on target.  I'm not just having to navigate the vagaries of the design of In Profundis here, but the contours and roadblocks of my own brain, some of which is difficult traveling.

More multitile stuff, reimplementing sandbox mode

Trying to post something here every day or two, to keep up that momentum.

Done more work on multitile entites.  I've been thinking, when a large entity moves, what happens to the things where it wants to move to?

Single-tile objects just swap positions.  Large objects could do the same thing, but it seems more realistic to try to move stuff out of the way.

 One recognized flaw with the current system for implementing falling multitile objects.  It's possible for two "convex" objects to be molded around each other so that both hang in the air, each "supporting" the other.  I'll probably handle this one by forbidding such objects.  There are other solutions, we'll see if it becomes necessary to go that far.

The sandbox is being reimplemented now as a debugging aid.

Multitile entities

To trade off no longer tracking fluid levels in a tile, instead watching "blocks" of water, in the rewrite we're simulating the grid with a little finer grain: what was one tile before is now effectively two tiles, stacked vertically.

This means that the boulders, from before, will have to have special support to handle being larger objects than the tile resolution.  Also, in the updated design, actors like the player will no longer exist as entities outside the grid, but in fact will be special types of tiles, which is a more "organic" representation that significantly changes a lot of subtle things.

The result is, I'm working on a new type of entity: the Multitile entity.

Multitile entites have one tile that's considered to be the "key" tile.  This is the one that gets turns, the only one that gets added to the cellular queue; the others are just along for the ride.  A previous project (one of the things I had to pick up for money, that has delayed In Profundis for so long) also used a cellular system that used multitile entities.  I learned a lot from that concerning how to simulate these kinds of objects, and the possible pitfalls.

So you see, I don't think all that time was wasted, I think it's a subtly wiser programmer that's working on the problem now.