Work on cricket tales the last few weeks has been concerned with scaling everything for the sheer amount of data involved. The numbers are big – we’re starting with the footage from 2013 as a test (a ‘smaller’ year), where 145 cameras recorded in total 438 days worth of video of cricket burrows. Our video processing robot is currently chopping this up into 211,889 sped up one minute clips, and encoding them into webm, ogg and h.264 mp4 for maximum browser compatibility. It looks like this would take a few months to do in total, and over the last week or so we have 8,000 videos processed.
Now we have a framework in place that will support this quantity of data, for example we can be continuously processing video while people are tagging – and swap them in and out so we don’t need to fit them all on disk. The database contains an entry for every video with a status (currently ‘not encoded’, ‘ready’ and ‘complete’). Movies could be marked complete after some number of players have watched it or some correlation metric with their tags has been met, then the files can be deleted off the disk. In terms of feasibility, we had 68K people playing the camouflage citizen science games over the last year – so if we managed to get that many people we’d need them to view 3 minutes each to get the entire dataset watched once.
This is a fictional map of the cricket’s burrows, with the name of the player who contributed most to them so far displayed – one idea for the ‘play’ element, which is the next big thing to consider. This is a balance between making it quick and fun and making people feel like they are contributing to something bigger and being able to see a tangible result for their efforts. Do we aim for something that takes someone five minutes of their time and getting enough people to make it work, or do we aim for a smaller number of more dedicated players who keep coming back? We have loads of options at this point, and to a large extent this depends on the precise nature of what the researchers need – so we need to do some more thinking together at this stage.
One of the three citizen science game projects we currently have running at Foam Kernow is a commission for Mónica Arias at the Muséum national d’Histoire naturelle in Paris, who works with this research group. She needed to use the Evolving butterflies game we made last year for the Royal Society Summer exhibition to help her research in pattern evolution and recognition in predators, and make it into a citizen science game. This is a standalone game for the moment, but the source is here.
We had several issues to address with this version. Firstly a lot more butterfly wing patterns were needed – still the heliconius butterfly species but different types. Mónica then needed to record all player actions determining toxic or edible patterns, so we added a database which required a server (the original is an educational game that runs only on the browser, using webgl). She also needed to run the game in an exhibition in the museum where internet access is problematic, so we worked on a system than could run on a Raspberry Pi to provide a self contained wifi network (similar to Mongoose 2000). This allows her to use whatever makes sense for the museum or a specific event – multiple tablets or PC with a touchscreen, all can be connected via wifi to display the game in a normal browser with all the data recorded on the Pi.
The other aspect of this was to provide her with an ‘admin’ page where she can control and tweak the gameplay as well as collect the data for analysis. This is important as once it’s on a Raspberry Pi I can’t change anything or support it as I could in a normal webserver – but changes can still be made on the spot in reaction to how people play. This also makes the game more useful as researchers can add their own butterfly patterns and change how the selection works, and use it for more experiments in the future.
On the Wild Cricket Tales citizen science game, one of the tricky problems is grading player created data in terms of quality. The idea is to get people to help the research by tagging videos to measure behaviour of the insect beasts – but we need to accept that there will be a lot of ‘noise’ in the data, how can we detect this and filter it away? Also it would be great if we can detect and acknowledge players who are successful at hunting out and spotting interesting things, or people who are searching through lots of videos. As we found making the camouflage citizen science games, you don’t need much to grab people’s attention if the subject matter is interesting (which is very much the case with this project), but a high score table seems to help. We can also have one per cricket or burrow so that players can more easily see their progress – the single egglab high score table got very difficult to feature on after a few thousand players or so.
We have two separate but related problems – acknowledging players and filtering the data, so it probably makes sense if they can be linked. A commonly used method, which we did with egglab too (also for example in Google’s reCAPTCHA which is also crowdsourcing text digitisation as a side effect) is to get compare multiple people’s results on the same video, but then we still need to bootstrap the scoring from something, and make sure we acknowledge people who are watching videos no one has seen yet, as this is also important.
Below is a simple naive scoring system for calculating a score simply by quantity of events found on a video – we want to give points for finding some events, but over some limit we don’t want to reward endless clicking. It’s probably better if the score stops at zero rather than going negative as shown here, as games should never really ‘punish’ people like this!
Once we have a bit more data we can start to cluster events to detect if people are agreeing. This can give us some indication of the confidence of the data for a whole video, or a section of it – and it can also be used to figure out a likelihood of an individual event being valid using the sum of neighbouring events weighted by distance via a simple drop-off function.
If we do this for all the player’s events over a single video we can get an indication of how consistent they are with other players. We could also recursively weight this by a player’s historical scores – so ‘trusted’ players could validate new ones – this is probably a bit too far at this point, but it might be an option if we pre-stock some videos with data from the researchers who are trained with what is important to record.
A release of Bumper Crop is now up on the play store with the source code here. As I reported earlier this has been about converting a board game designed by farmers in rural India into a software version – partly to make it more easily accessible and partly to explore the possibilities and restrictions of the two mediums. It’s pretty much beta really still, as some of the cards behave differently to the board game version, and a few are not yet implemented – we need to work on that, but it is playable now, with 4 players at the same time.
The 3D and animation is done using the fluxus engine on android, and the game is written in tinyscheme. Here’s a snippet of the code for one of the board locations, I’ve been experimenting with a very declarative style lately:
;; description of location that allows you to fertilise your crops
;; the player has a choice of wheat/onion or potatoes
(place 26 'fertilise '(wheat onion potato)
;; this function takes a player and a
;; selected choice and returns a new player
(lambda (player choice)
(if (player-has-item? player 'ox) ;; do we have an ox?
;; if so, a complete a free fertilise task if needed
(if (player-check-crop-task player choice 'fertilise 0)
(player-update-crop-task player choice 'fertilise)
;; otherwise it costs 100 Rs
(if (player-check-crop-task player choice 'fertilise 100)
(player-add-money player -100) ;; remove money
player choice 'fertilise)
(place-interface-crop)) ;; helper to make the interface
Testing the board game, which you can download on this page:
The game on tablet:
This is the game running on a phone:
Some hungry butterfly hunters making evolution happen at the Summer Science exhibition last week. Thank goodness for multitouch!
The Heliconius Butterfly Wing Pattern Evolver game is finished and ready for it’s debut as part of the Butterfly Evolution Exhibit at the Royal Society Summer Exhibition 2014. Read more about the scientific context on the researcher’s website, and click the image above to play the game.
The source code is here, it’s the first time I’ve used WebGL for a game, and it’s using the browser version of fluxus. It worked out pretty well, even to the extent that the researchers could edit the code themselves to add new explanation screens for the genetics. Like any production code it has niggles, here’s the function to render a butterfly:
(define (render-butterfly s)
;; set tex based on index
(texture (list-ref test-tex (butterfly-texture s)))
;; move to location
(translate (butterfly-pos s))
;; point towards direction
(maim (vnormalise (butterfly-dir s)) (vector 0 0 1))
(rotate (vector 0 90 90)) ;; angle correctly
(scale (vector 0.5 0.5 0.5)) ;; make smaller
(draw-obj 4) ;; draw the body
(with-state ;; draw the wings in a new state
(rotate (vector 180 0 0))
(translate (vector 0 0 -0.5)) ;; position and angle right
;; calculate the wing angle based on speed
(let ((a (- 90 (* (butterfly-flap-amount s)
(+ 1 (sin (* (butterfly-speed s)
(+ (butterfly-fuzz s)
(rotate (vector 0 0 a))
(draw-obj 3)) ;; draw left wing
(scale (vector 1 -1 1)) ;; flip
(rotate (vector 0 0 a))
(draw-obj 3)))))) ;; draw right wing
There is only immediate mode rendering at the moment, so the transforms are not optimised and little things like draw-obj takes an id of a preloaded chunk of geometry, rather than specifying it by name need to be fixed. However it works well and the thing that was most successful was welding together the Nightjar Game Engine (HTML5 canvas) with fluxus (WebGL) and using them together. This works by having two canvas elements drawn over each other – all the 2D (text, effects and graphs) are drawn using canvas, and the butterflies are drawn in 3D with WebGL. The render loops are run simultaneously with some extra commands to get the canvas pixel coordinates of objects drawn in 3D space.
Bumper crop is an android game I’ve just started working on with Dr Misha Myers as part of the Play to Grow project: “exploring and testing the use of computer games as a method of storytelling and learning to engage urban users in complexities of rural development, agricultural practices and issues facing farmers in India.”
(Warning – contains machine translated Hindi!)
I’m currently working out the details with artist Saswat Mahapatra and Misha, who have been part of the team developing this game based on fieldwork in India working with farmers from different regions. They began by developing a board game, which allowed them to flexibly prototype ideas with lots of people without needing to worry about software related matters. This resulted in a great finished product, super art direction and loads of assets ready to use. I very much like this approach to games design.
From my perspective the project relates very closely to groworld games, germination x, as well as the more recent farm crap app. I’m attempting to capture the essence of the board game and restrict the necessary simplifications to a minimum. The main challenge now that the basics are working is providing an approximation of bartering and resource management between players that board games are so good at, into a simple interface – also with the provision of AI players.
Source code & Play store (very alpha at the moment!)
I’ve been working lately with the Heliconius research group at the University of Cambridge on a game to explain the evolution of mimicry in butterfly wing patterns. It’s for use at the Summer Science Exhibition at the Royal Society in London, where it’ll be run on a large touch screen for school children and visiting academics to play.
The game models biological processes for education purposes (as opposed to the genetic programming as used on the camouflage egg game), and the process of testing this, and deciding what simplifications are required has become a fascinating part of the design process.
In biosciences, genetics are modelled as frequencies of specific alleles in a given population. An allele is a choice (a bit like a switch) encoded by a gene, so a population can be represented as a list of genes where each gene is a list of frequencies of each allele. In this case the genetics consists of choices of wing patterns. The game is designed to demonstrate the evolution of an edible species mimicking a toxic one – we’ll be publishing the game after the event. A disclaimer, my terminology is probably misaligned in the following code, still working on that.
;; an allele is just a string id and a probability value
(define (allele id probability)
(list id probability))
;; a gene is simply a list of alleles
;; return the id of an allele chosen based on probability
(define (gene-express gene)
(let ((v (rndf)))
(lambda (allele r)
(let ((segment (+ (car r) (allele-probability allele))))
(if (and (not (cadr r))
(< v segment))
(list segment allele)
(list segment (cadr r)))))
(list 0 #f)
;; a chromosome is simple list of genes
;; returns a list of allele ids from the chromosome based on probability
(define (chromosome-express chromo)
(map gene-express chromo))
When an individual is removed from the population, we need to adjust the probabilities by subtracting based on the genetics of the eaten individual, and the adding to the other alleles to keep the probabilities summing to one:
;; prevents the probability from 'fixing' at 0% or 100%
(define (calc-decrease p)
(* (min p (- 1 p)) allele-decrease))
;; remove this genome from the population
(define (gene-remove-expression gene genome)
(let ((dec (calc-decrease (allele-probability (car gene)))))
(let ((inc (allele-increase dec (length gene))))
(if (eq? (allele-id allele) genome)
allele (- (allele-probability allele) dec))
allele (+ (allele-probability allele) inc))))
9,000 players, 20,000 games played and 400,000 tested egg patterns later we have over 30 generations complete on most of our artificial egg populations. The overall average egg difficulty has risen from about 0.4 seconds at the start to 2.5 seconds.
Thank you to everyone who contributed their time to playing the game! We spawned 4 brand new populations last week, and we’ll continue running the game for a while yet.
In the meantime, I’ve started working on ways to visualise the 500Mb of pattern generating code that we’ve evolved so far – here are all the eggs for one of the 20 populations, each row is a generation of 127 eggs starting at the top and ordered in fitness score from left to right:
This tree is perhaps more useful. The ancestor egg at the top is the first generation and you can see how mutations happen and successful variants get selected.
We’ve released our latest citizen science camouflage game Egglab! I’ve been reporting on this for a while here so it’s great to have it released in time for Easter – we’ve had coverage in the Economist, which is helping us recruit egg hunters and 165,000 eggs have been tested so far over the last 3 days. At time of writing we’ve turned over 13 generations starting with random pattern programs and evolving them with small mutations, testing them 5 times with different players and picking the best 50% each time.
Here is an image of some of the first generation of eggs:
And this shows how they’ve developed 13 generations later with the help of many thousands of players:
We can also click on an individual egg and see how it’s evolved over time:
And we see how on average the time taken to find eggs is changing:
Technically this project involves distributed pattern generation on people’s browsers using HTML5 Canvas, making it scalable. Load balancing what is done on the server over three machines and a Facebook enabled subgame – which I’ll use another blog post to explain.