Fast HTML5 sprite rendering

After quite a lot of experimentation with HTML5 canvas, I’ve figured out a way to use it with the kind of big isometric game worlds used for Germination X which are built from hundreds of overlapping sprites. There are lots of good resources out there on low level optimisations, but I needed to rethink my general approach in order to get some of these working.

It was quite obvious from the start that the simple clear screen & redraw everything way was going to be far too slow. Luckily HTML5 canvas gives us quite a lot of options for building less naive strategies.

A debug view of the game with 10 frames of changes shown with two plant spirits and one butterfly moving around.

The secret is only drawing the changes for each frame (called partial re-rendering in the link above). To do this we can calculate sprites which have changed and the ones they overlap with. The complication is maintaining the draw order and using clipping to keep the depth correct without needing to redraw everything else too.

In the game update we need to tag all the sprites which change position, rotation, scale, bitmap image, transparency etc.

Then in the render loop we build a list of all sprites that need redrawing, along with a list of bounding boxes for each overlapping sprite of the changed sprites that touch them. There may be more than one bounding box as a single sprite may need to be redrawn for multiple other changed sprites.

For each changed sprite:
    Get the bounding box for the changed sprite
    For each sprite which overlaps with this bounding box:
        If overlapping sprite has already been stored:
            Add the bounding box to overlapping sprite's list
        Else:
            Store overlapping sprite and current bounding box.
    Add the changed sprite to the list.

Assuming the sprites have been sorted into depth order, we now draw them using the list we have made (we would only need to loop over the redraw list if we built it in depth sorted order).

For each sprite:
    If it's in the redraw list:
        If it's not one of the originally changed sprites:
            Set a clipping rect for each bounding box.
        Draw the sprite.
        Turn off the clipping, if it was used.

With complex scenes and multiple moving objects, this algorithm means we only need to redraw a small percentage of the total sprites visible – and we start to approach Flash in terms of performance for games (I suspect that flash is doing something similar to this under the hood). The code is here, currently written in HaXE, but will probably end up being ported to Javascript.

Germination X: More on platform independence

More detail on yesterday’s HTML5 canvas version of Germination X. The game engine it uses (known in the source code as “truffle”) is carved up into several layers in order to allow it to make use of HaXe’s cross platform abilities. This is exactly the same concept as used with console games – you set up some standard interfaces to be used from the game code, and implement them for each target at the lower level. The main thing it to make sure you continually build and test both versions.

Here’s a diagram with rounded corners:

There are a few things in the game code I needed to go directly to flash for, one was the glow effect on the plants owned by the player – using a filter. The other was doing an image pixel colour lookup to get the emotional colours for the spirits – both of these are switched off in the canvas version, to be fixed later.

One of the things I’m most impressed by is the ability to send a HaXE closure to Javascript code, store and run it from there (for example on those pesky mouse events on sprites). Of course this works as it’s all Javascript in the end, it’s just one of those cool things that takes you by surprise.

Germination X in HTML5 canvas

I’ve spent a couple of days on an experimental not-quite-working-yet port of Germination X to HTML5. Port is probably a slightly misleading word, as it’s actually running exactly the same HaXE code, it’s just compiled to Javascript rather than swf with a load of extra Javascript filling in the stuff needed to replace Flash. The game engine that Germination X is built on is now officially cross platform! It’s great to finally have a non-proprietary browser option for the game, and HTML5 is much faster now than when I first tried it, but it’s still got a long way to go. Problems I’ve noticed so far:

It’s still slow. I’m aware of the limitations and background to browser tech, but it’s depressing that it’s somehow problematic drawing a few hundreds of sprites in 2012 while my ancient Gameboy DS can manage quite fine thankyouverymuch :)

Tinting images is quite a basic thing to do in order to efficiently use images and save bandwidth and memory. Why do I have to loop through pixels in javascript to do this? This is the main cause of slowness in the HTML5 version of the game ATM, hopefully a simple bitmap cache will fix this. There are also some neat ways to do it with offscreen bitmaps and blendmode tricks, but they seem even slower.

Having to write your own mouse event detection for sprite down/up/over/out. I guess this is one of the biggest reasons for the explosion of HTML5 game engines. It’s quite non-trivial and un-fun to set up properly. Actually getting the real mouse coordinates is also quite hard to do.

On the plus side, it’s a breath of fresh air to get back the control of immediate mode – it seems historically that this tends to be the optimal approach, give us the render loop!

Now you’ve got this far – here is some accidental art which happened this morning, and which I’m particularly proud of:

Hapstar version 0.7

Hapstar has achieved some popularity in the genetics community, the paper has currently racked up 13 citations, including one for it’s use to create a figure for a Nature paper.

This version addresses some requests from users, including the ability to save graphs midway though balancing. This means it’s easier to run them for long periods of time. It can be downloaded from the official page on the FoAM site.

Germination X at Loading Bar, Falmouth

Where better for doing a game focus group test than in a games cafe? We had a fantastic turn out last night – we actually ran out of space! This was the third Lirec focus test using Germination X, but the first I’ve been physically involved in, and the biggest public event about this game I’ve done since last year’s draw your own plant spirit at Pixelache, way back at the start of the project.

Loading is based in the centre of Falmouth, and offers refreshment and the latest games to it’s dedicated clientele. We started off with around 45 minutes playing followed by a post-it note session covering the questions from SICS/mobile life, then a more freeform discussion on observations, many improvements and inevitable deep philosophy of game design as the evening wore on.

The participants included local people interested in permaculture and organic farming as well as those with a more games interest. Some interesting/unexpected things happened, and I haven’t even begun going through all the feedback, so more on that soon.

Edit: Following on from the focus group the game has it’s first ever review!

… one of the most drunken, slovenly and downright interesting game testing sessions in recorded history.

GX: Ancient oaks and tutorial info

Three hundred multicoloured oaks have been scattered around the Germination X world. These trees are all that remains of an ancient forest and require multiple players to find and sow plants around them in order to rejuvenate them into their former glory. This provides a simple additional community activity as well a first attempt at populating the wide expanses of the Germination X planet with landmarks – something that has been brought up in the focus group studies. There is also now an “oak score” to record the players who contribute bringing them back to life.

Also new today, a tutorial explaining how to do the basic picking and planting as well as some of the more advanced things (which are not so obvious and badly needing explanation) like sending seeds and fruit to other players and using butterflies to find out who planted things.

GX: Spirit personalities, asking players for help, twitter

Another Germination X server update happened yesterday, concentrating on the plant spirits.

We needed to log the emotional state of the spirits, so I thought twitter could be used for this, similar to the Naked on Pluto cleaner bot. In this case though, all three plant spirits have accounts, so they can have little conversations with each other – the wording is the same as the in game messages, and is chosen based on their current emotional state.

For the first time the plant spirit characters are programmed with different “personalities”, or settings for their FAtiMA roles, so we should see some different behaviours from them.

The last update broke the FAtiMA rules for diagnosis of ill plants and asking players for help, which is now fixed. The spirits now prioritise asking for help from players who have logged in recently, making it much more likely that currently active players see requests. The spirits are now also ‘summoned’ to ill plants throughout the entire world, meaning more activity generally – previously they were attracted to different locations by player activity only, and so would stay in the last area used (and could only “see” plants there) until new players came along.

Slub on Chordpunch CP0x08

A recording from our livecoding performance in Paris last year has been released for free on Chordpunch, a new online record label:

Chordpunch was set up to explore the many and moving forms of algorithmic music. That might mean a computer program generating every note you hear, or new electronic music inspired by algorithms, or human beings following interesting rules with musical outcomes.

Still in Paris, and still concerning slub, we are also featured as part of the Form@ts virtual exhibition at Jeu de Paume curated by Christophe Bruno. The exhibition concerns artwork, such as livecoding, that crosses borders of format and convention.

GX: soil conditions, glowing plants and empathic spirits

Along with setting up interactive installations, the last week has been a fairly intense code sprint on Germination X, based on feedback from the second focus study by SICS/mobile life in Stockholm.

In order to get a more interesting world, and shake things up a little, soil conditions now vary quite dramatically. Some areas are friendlier to plants, and some very much harder to grow in than before. We’ve talked a lot in the design process about having different types of plants for different soil types or whole regions, this is a step in that direction.

The feedback from both the focus groups in Stockholm agreed that the little star sprites used to signify plants you’d planted were confusing as they looked like flowers. I’ve removed them and added a little glow around the plants instead.

The biggest change this time is probably with the plant spirits. The FAtiMA AI system has been upgraded and as well as a bunch of fixes on my end to improve responsiveness and message wording, I’ve focused on a whole range of emotions called “fortune of others”. The OCC-model (Ortony et al., 1988) defines them like this:

I’ve attached actions to each of these four emotions which cause the spirits to react with empathy to events in the world that don’t concern them directly, depending on how much they like each other, or if the events are good or bad for the plant spirit in question. This starts to more effectively exhibit interesting properties of the FAtiMA system.