Thursday, June 23, 2011

In Profundis Question

Talking it over with Simon a bit, this business with chasing C++ may be taking too long.  I may go ahead and develop the Python version some more, do some releases of that, and see if it ends up being good enough.  In the meantime I'll read up on C++ and try to get the kinks out of my head concerning it.

The thing about that I'm most worried about is it might turn out to be a dead-end.  A reliance on Psyco almost certainly is; it's been dead in the water since 2009, and hasn't had an official release since 2007.  Going with its successor PyPy is perhaps possible, but I've so far been unable to get Pygame working under PyPy.  I'm pretty sure it works since PyPy has an important profiling tool that uses Pygame for graphics output, but that site is weirdly lacking on details for installing it.  The Windows installer for Pygame refuses to recognize PyPy as Python 2.7 and easy_install seems to work at first but in interactive mode refuses to import it.

Any of you have experience with PyPy?  I apologize this is all taking so long.  I never promised you an instantaneous rose garden, although I dare say you deserve something soon.


  1. According to, pygame is incompatible with pypy. I can get it to install (by moving the installer into the pypy directory and installing to python "from a different location), but it fails importing pygame.base. I'm guessing in the long run, pygame will need to provide pure python replacements for its extension modules.

    I mentioned earlier that pypy can run concurrent threads. I was wrong. It cannot. And depending on the code it may be faster or slower than CPython. Of course, other alternatives like Jython and IronPython are available, but those are even weirder than pypy and will definitely not run any extension modules.

    Anyway, I think that switching interpreters (or languages) will be a dead end until you understand where your performance problems are coming from. If it's a relatively isolated component (such as the cellular automata), I bet you can easily write that component in C and the rest in Python without needing anything like psyco (using ctypes to link them up, not extension modules; extensions are more trouble to learn than they're worth). (I would recommend C, not C++, because it's a simpler language that rarely hides anything.)

  2. I am so lucky to have such good people commenting.

    The major performance drains, I suspect, are three, and with various things I can do about them. Not necessarily in order:

    * The CA engine
    * The sort done to prioritize which cells need to be calculated
    * Rendering

    The platforming is pretty simple, at least at the moment, and is unlikely to be a performance problem. I'll see about doing some actual profiling soon but I'm pretty confident these are the things.

    Pygame doesn't run on PyPy, eh? Well nice to know that now. In my own attempts to get it work I've encountered that very error, failure to import pygame.base, so that's a pretty strong indication. I've read elsewhere that some parts of pyglet do work, but am putting off changing libraries for the time being.

    And yes, if I rewrite the CA engine it'll definitely be in C, which I have more experience in from hacking Nethack sources and minor other things.