Tuesday, April 24, 2012

Ways to speed up yer Python code

So I've been looking into ways to get Python code to run quickly, which is a continuation of the search I did last year (yikes, it's been almost a year since this began...)

I spent time some getting Cython working, and getting the code to work with it, but I kept running into difficult-to-solve problems with Python data types in Numpy arrays. Particularly, an array that wouldn't compile because the compiler claimed it was of type long when I defined it as int, or at least as far as I can tell. I eventually decided to shelve that and look in other directions.

Currently, I'm changing the Pygame bits over to Pygame in order to make use of hardware acceleration, which is iffy under Pygame. This has the additional benefit of allowing the code to work under PyPy, the successor to Psyco. A drawback, however, is that pyglet wants to run the program's main event and draw loops itself. I've gotten in some more coding experience, so I'm less standoffish about that now than I was before, but to really do things the pyglet way everything has to be a sprite, or a polygon. The way we draw liquids at the moment in Pygame uses filled polygons, so it's good that there's a way to port this over. The combination of hardware acceleration and JIT compiling is a potent one, and could potentially bring better performance than even using Psyco... if only I could figure out how to install pyglet to a PyPy installation.

If this fails for some reason, there's still weave, a system for inlining C code in Python. My problems with C and C++ tend to concern linking; C itself I have no trouble with, but getting code more complex than a handful of simply-included source files to link together into an executable has been a woeful journey for me.

Anyway, work continues. You can't see it on your end, but behind the scenes of this blog's inscrutable periods of silence are me wringing my hands over the problems I've been facing. I have considered opening the source, or at least posting bits of it, and asking for advice from you guys. I don't suppose there are any Python mavens reading this?


  1. Is performance really that huge a deal?
    Like, are common players going to have a problem?
    I think you said a while ago that your rig isn't top notch, so could this just be issue for you?

  2. When I abandoned Psyco, the framerate went down from around 30-40 fps on average to 18-22. But what I was hoping to do was switch the game to a more turn-based system, at the cost of calculating the whole game world, which would allow for liquid loops and things like that. I could put up with fluid flowing somewhat slower for that, but not that slow.

    What is more is, my laptop isn't top-of-the-line, but it's single core. Python doesn't take good advantage of multi-core systems in its threading, so I'm having to face the possibility that even people on good gaming rigs might not get all that much better performance than I do on my low-end laptop.

  3. I'm no python maven, but I know python is usually fast enough, if you cut the right corners. It's the implementation of the algorithms that make all the difference. Also, calling a c-function from python is not that hard, and there's plenty of good examples of how to do it. Also, are you totally sure where most of the time is spent? I've followed your blog from the beginning and I know that iterating over a lot of cells can be cpu intensive, but like you've described I'm sure there's a lot of ways to optimize it. Maybe it's time to put the code up on github? :-)

    Anyway, thanks for still updating the blog, it's always interesting. As is Roguelike Radio!