All posts by dave

AI as puppetry, and rediscovering a long forgotten game.

AI in games is a hot topic at the moment, but most examples of this are attempts to create human-like behaviour in characters – a kind of advanced puppetry. These characters are also generally designed beforehand rather than reacting and learning from player behaviour, let alone being allowed to adapt in an open ended manner.

geo-6
Rocketing around the gravitational wells.

Geo was an free software game I wrote around 10 years ago which I’ve recently rediscovered. I made it a couple of years after I was working for William Latham’s Computer Artworks – and was obviously influenced by that experience. At the time it was a little demanding for graphics hardware, but it turns out the intervening years processing power has caught up with it.

This is a game set in a large section of space inhabited by lifeforms comprised of triangles, squares and pentagons. Each lifeform exerts a gravitational pull and has the ability to reproduce. It’s structure is defined by a simple genetic code which is copied to it’s descendants with small errors, giving rise to evolution. Your role is to collect keys which orbit around gravitational wells in order to progress to the next level, which is repopulated by copies of the most successful individuals from the previous level.

A simple first generation lifeform.
A simple first generation lifeform.

Each game starts with a random population, so the first couple of levels are generally quite simple, mostly populated by dormant or self destructive species – but after 4 or 5 generations the lifeforms start to reproduce, and by level 10 a phenotype (or species) will generally have emerged to become an highly invasive conqueror of space. It becomes an against the clock matter to find all the keys before the gravitational effects are too much for your ship’s engines to escape, or your weapons to ‘prune’ the structure’s growth.

I’ve used similar evolutionary strategies in much more recent games, but they’ve required much more effort to get the evolution working (49,000 players have now contributed to egglab’s camouflage evolution for example).

A well defended 'globular' colony - a common species to evolve.
A well defended 'globular' colony – a common phenotype to evolve.

What I like about this form of more humble AI (or artificial life) is that instead of a program trying to imitate another lifeform, it really just represents itself – challenging you to exist in it’s consistent but utterly alien world. I’ve always wondered why the dominant post-human theme of sentient AI was a supercomputer deliberately designed usually by a millionaire or huge company. It seems to me far more likely that some form of life will arise – perhaps even already exists – in the wild variety of online spambots and malware mainly talking to themselves, and will be unnoticed – probably forever, by us. We had a little indication of this when the facebook bots in the naked on pluto game started having autonomous conversations with other online spambots on their blog.

A densely packed 'crystalline' colony structure.
A densely packed 'crystalline' colony structure.

Less speculatively, what I’ve enjoyed most about playing this simple game is exploring and attempting to shape the possibilities of the artificial life while observing and categorising the common solutions that emerge during separate games – cases of parallel evolution. I’ve tweaked the between-levels fitness function a little, but most of the evolution tends to occur ‘darwinistically’ while you are playing, simply the lifeforms that reproduce most effectively survive.


An efficient and highly structured 'solar array' phenotype which I’ve seen emerge twice with different genotypes.

You can get the updated source here, it only requires GLUT and ALUT (a cross platform audio API). At one time it compiled on windows, and should build on OSX quite easily – I may distribute binaries at some point if I get time.

geo-5
A ‘block grid’ phenotype which is also common.

Mongoose 2000 in the wilds of Uganda

One of our most ambitious projects: Mongoose 2000, is now up and running after 6 months of testing. This is a Raspberry Pi and Android tablet system to synchronise and store masses of data for a long running behavioral experiment recording the activities of packs of mongooses in the field site in Uganda. They broad aims of the project are to study mongoose behaviour in order to understand the evolution of society.

use

The timespan that we are working with is long, and the location – while not as remote as it could be (there is some internet access and power) required some consideration – so this was a project where we really needed to employ an appropriate technology approach which is manifested in various ways:

Open source software: means that Foam Kernow are not a bottle neck to continued development, as we do not have exclusive control over the source code (which is released into the commons). New developers can be found if required (for whatever reason) who do not need to start from nothing – this gives the research team more control and future proofing.

Use of commodity hardware: it’s likely that the hardware in use will become obsolete in this timeframe. The lifespan of android software should mean it can be installed on compatible devices for a long time, and the team can make use of advances in sensor or battery technology. The raspberry pi we are using is already an older model now, but as it’s a standard linux setup we can easily move it to other machines in the future. Currently it acts as an ‘appliance’ which just needs to be turned on, but we can add a web interface to control it from the tablet – or eventually replace it with a peer to peer syncronisation system.

The Ugandans working on the project have an healthy DIY relationship with technology, they expect to be able to repair or modify things themselves, and I’d like to figure out ways we can work with this more. The UAV toolkit project provides some indication of what can be done with programming these kinds of devices in the field. Part of the decisions on the hardware (and the design of the software, e.g. using a scheme interpreter) were to use devices that were open to a more end-user programming approach in the future.

use2

use3

Syncronising a tablet with observations previously recorded on the Raspberry Pi:

use4

New camouflage pattern engine

One of the new projects we have at foam kernow is a ambitious new extension of the egglab player driven camouflage evolution game with Laura Kelley and Anna Hughes at Cambridge Uni.

As part of this we are expanding the patterns possible with the HTML5 canvas based pattern synthesiser to include geometric designs. Anna and Laura are interested in how camouflage has evolved to disrupt perception of movement so we need a similar citizen science game system as the eggs, but with different shapes that move at different speeds.

Here are some test mutations of un-evolved random starting genomes:

3

4

2

This is an example pattern program:

5gen

Training teachers in coding, and thinking about e-waste/recycled robots

We recently had the second inset training day in programming related activities at Truro school. Following on from the previous session I didn’t want to introduce too much new stuff, so we concentrated on going back over Sonic Pi and Minecraft/Python programming in the morning, then discussed a lot more about our future workshops in the afternoon. These will include children from most of the primary schools in the Truro area and take place during the next term. I also wanted to use the day to work on some specific ideas the teachers wanted to get going. One of these involved running a Kinect camera with the Pi, which we managed to get more or less working – reading depth data in Python for potentially plugging into Minecraft at a later date.

IMG_20150617_143455

The big idea I wanted to get feedback about was the use of robotics and electronics in conjunction with code (the photo above is my desk while preparing the day before). I didn’t have any lesson plans for this, but based on some comments from the first workshop and from playing a bit with this during CodeClub teaching I felt it might be good with this age group, who already know quite a bit about screen based activities. To keep costs down (as well as building in issues like e-waste) we’re planning to make use of recycled junk to extract motors from toys and hack them to do different things. It seems this doesn’t cause too many issues from safety POV (everything will be low-voltage and we can check everything beforehand), even the use of soldering irons seems to be acceptable as they have supervision.

The advantage of using code to move real things (as shown by a long history) is that it directly connects programming with the world outside of the screen (where it most certainly now has great importance in our lives), and at the same time results in teamwork – as it’s not easy for a single person to make a robot while programming it. When we tried this at Troon Primary they self organised into a group with one person programming while another was building stuff in lego and a third provided excited communication between them and more or less managed the task. Other programming activities tend to be more individualistic – with the possible exception of networked Minecraft, which is important but a very different form of collaboration.

Cricket Tales – scaling up

Work on cricket tales the last few weeks has been concerned with scaling everything for the sheer amount of data involved. The numbers are big – we’re starting with the footage from 2013 as a test (a ‘smaller’ year), where 145 cameras recorded in total 438 days worth of video of cricket burrows. Our video processing robot is currently chopping this up into 211,889 sped up one minute clips, and encoding them into webm, ogg and h.264 mp4 for maximum browser compatibility. It looks like this would take a few months to do in total, and over the last week or so we have 8,000 videos processed.

vid1

Now we have a framework in place that will support this quantity of data, for example we can be continuously processing video while people are tagging – and swap them in and out so we don’t need to fit them all on disk. The database contains an entry for every video with a status (currently ‘not encoded’, ‘ready’ and ‘complete’). Movies could be marked complete after some number of players have watched it or some correlation metric with their tags has been met, then the files can be deleted off the disk. In terms of feasibility, we had 68K people playing the camouflage citizen science games over the last year – so if we managed to get that many people we’d need them to view 3 minutes each to get the entire dataset watched once.

map

This is a fictional map of the cricket’s burrows, with the name of the player who contributed most to them so far displayed – one idea for the ‘play’ element, which is the next big thing to consider. This is a balance between making it quick and fun and making people feel like they are contributing to something bigger and being able to see a tangible result for their efforts. Do we aim for something that takes someone five minutes of their time and getting enough people to make it work, or do we aim for a smaller number of more dedicated players who keep coming back? We have loads of options at this point, and to a large extent this depends on the precise nature of what the researchers need – so we need to do some more thinking together at this stage.

Picademy Exeter and Future Thinking for Social Living

Last week I had the chance to help out the Raspberry Pi foundation at their Picademy in Exeter. It was good to meet up with Sam Aaron again to talk livecoding on Pis, and also see how they run these events. They are designed for local teachers to get more confident with computers, programming and electronics to the point where they can start designing their own teaching materials on the second day of the two day course. This is a model I’m intending to use for the second inset teacher training day I’m doing next week at Truro school – it’s pretty exciting to see the ideas that they have for activities for their pupils, and a good challenge to help find ways to bring them into existence in a day.

IMG_20150605_141147

We also had the ending of Future Thinking for Social Living at the Miners Court summer party last week. We exhibited the map made during the workshops, made lots of tea, and had some fun with the pattern matrix in musical mode out in the garden – I adapted Alex’s music system we used with Ellen in Munich to run on Raspberry Pi so it didn’t require a laptop, or a screen at all – simply a speaker. It was interesting how quickly people got the idea, in many ways music is easier to explain than weaving as listening while coding is multi-sensory.

IMG_20150603_151747

Hungry birds 2 – the citizen science edition

One of the three citizen science game projects we currently have running at Foam Kernow is a commission for Mónica Arias at the Muséum national d’Histoire naturelle in Paris, who works with this research group. She needed to use the Evolving butterflies game we made last year for the Royal Society Summer exhibition to help her research in pattern evolution and recognition in predators, and make it into a citizen science game. This is a standalone game for the moment, but the source is here.

title

aie

We had several issues to address with this version. Firstly a lot more butterfly wing patterns were needed – still the heliconius butterfly species but different types. Mónica then needed to record all player actions determining toxic or edible patterns, so we added a database which required a server (the original is an educational game that runs only on the browser, using webgl). She also needed to run the game in an exhibition in the museum where internet access is problematic, so we worked on a system than could run on a Raspberry Pi to provide a self contained wifi network (similar to Mongoose 2000). This allows her to use whatever makes sense for the museum or a specific event – multiple tablets or PC with a touchscreen, all can be connected via wifi to display the game in a normal browser with all the data recorded on the Pi.

The other aspect of this was to provide her with an ‘admin’ page where she can control and tweak the gameplay as well as collect the data for analysis. This is important as once it’s on a Raspberry Pi I can’t change anything or support it as I could in a normal webserver – but changes can still be made on the spot in reaction to how people play. This also makes the game more useful as researchers can add their own butterfly patterns and change how the selection works, and use it for more experiments in the future.

admin

Coding with knots: Inca Quipu

This week I’m teaching at IMM Düsseldorf with Julian Rohrhuber which has given me a chance to follow up a bit on Inca Quipu coding with knots, a dangling thread from the weavecoding project. Quipu are how the Incas organised their society, as they had no written texts or money – things like exchanges (for example from their extensive store houses) were recorded via knots. Researchers have been able to decode the basic numeric system they used, but 20% of the quipu seem to follow a different set of rules, along with extra information encoded via thread material, twist direction, colour and other knot differences. I’ve written a python program for converting the Khipu Database Project excel charts into graphviz files for visualising:

quipu5

The knots are described in ascii art, with S and Z relating to the ply and knot ‘handedness’ direction they are tied in:

O : a single knot 
O/O : two single knots tied in S direction (it's rotated 90 degrees :)
(\\\\) : a long knot of value '4' tied in the Z direction
/8 : end (figure of 8) knot tied S direction

The pendant nodes also have labels describing their ply direction and the side the attach on, so “S R” is S ply & recto attached.

The hardest part of this has been a bit of more recent media archeology to figure out the colour values, I’ve had to cross reference the original Ascher-Ascher Quipu Databooks published in 1978 which contain their own colour system which more or less maps to the NBS-ISCC Munsell colour chart originally proposed in 1898. Luckily that site provides hex colour values – hopefully they are vaguely accurate, the current lookup table is here:

colour_lookup = {
    "W": "#777777",
    "SR":  "#BF2233",
    "MB" : "#673923",
    "GG" : "#575E4E",
    "KB" : "#35170C",
    "AB" : "#A86540",
    "HB" : "#5A3D30",
    "RL" : "#AA6651",
    "BG" : "#4A545C",
    "PG" : "#8D917A",
    "B" : "#7D512D",
    "0B" : "#64400F",
    "RM" : "#AB343A",
    "PR" : "#490005",
    "FR" : "#7F180D",
    "DB" : "#4D220E",
    "YB" : "#BB8B54",
    "MG" : "#817066",
    "GA" : "#503D33"
}

Weavecoding Munich

Ellen’s exhibition in Munich was always going to be a pivotal event in the weavecoding project – one of the first opportunities to expose our work to a large audience. The Museum of casts of classical sculptures was the perfect context for the mythical aspects of weaving, overlooked by Penelope and friends with her subversive woven/unwoven work, we could explore the connections between livecoding and weaving.

IMG_8477 2

Practically we focused on developing the tangible weavecoding exhibit for events later in the week, as well as discussing the many languages we have developed so far for different looms and weaving techniques. One of our discoveries is that none of the models or languages we have created seem sufficient in themselves – weaving could be far too big to be able to be described or solved from a single perspective. We’ve tried approaches describing weave structures from the actions of the weaver, setup of the loom and structure of the fabric – perhaps the most promising is to explor the story of weaving from the perspective of the thread itself.

IMG_20150510_062737

IMG_20150508_153211

One of the distinctive things about weaving in antiquity is how multiple technologies were combined to form a single piece of fabric, weaving in different directions, weft becoming warp, use of tablets vs warp weighted weaving. To explain this via the path of a single conceptual thread crossing through itself may make this possible to describe in a more flexible, declarative and abstracted manner than having to explain each method separately as if in it’s own world.

IMG_20150508_152045

IMG_20150509_100421

The pattern matrix has now been made into good shape for explaining the relationship between colour and structure in pattern formation. For the first time we also used all 4 sensors per block on the bottom row which meant we could use a special “colour” block that the system recognises from the normal warp/weft ones and use it’s rotation to choose between 8 preset colour settings. This was quite a breakthrough as it had all been theoretical before.

IMG_20150508_194220

Adding this more complex use of the magnetic patterns meant that Alex could set up the matrix as a tangible interface for his tidal livecoding software meaning Ellen could join us for a collaborative slub weavecoding performance on the Saturday evening. The prospect of performing together was something we have talked about since the very beginning of the project, so it was great to finally reach this point. The reverb in the museum was vast, meaning that we had to play the space a lot, and provide ‘music for looking at sculptures by':