Category Archives: random thoughts

Project Nightjar: Camouflage data visualisation and possible internet robot predators

We’ve had tens of thousands of people spotting nightjars and donating a bit of their time to sensory ecology research. The results of this (of course it’s still on-going, along with the new nest spotting game) is a 20Mb database with hundreds of thousands of clicks recorded. One of the things we were interested in was seeing what people were mistaking for the birds – so I had a go at visualising all the clicks over the images (these are all parts of the cropped image – as it really doesn’t compress well):



Then, looking through the results – I saw a strange artefact:


Uncompressed high res version

My first thought was that someone had tried cheating with a script, but I can hardly imagine that anyone would go to the bother and it’s only in one image. Perhaps some form of bot or scraping software agent – I thought that browser click automation was done by directly interpreting the web page? Perhaps it’s a fall back for HTML5 canvas elements?

It turns out it’s a single player (playing as a monkey, age 16 to 35 who had played before) – so easy enough to filter away, but in doing that I noticed the click order was not as regular as it looked, and it goes a bit wobbly in the middle:

Someone with very precise mouse skills then? :)

Fascinate Falmouth

It’s not often that you get to go to the first edition of a festival or conference, but last week was the first ever Fascinate Conference, in Falmouth – a varied collection of artists, performers, musicians and experimenters with technology, some from far away on their first visit to Cornwall, others were local – both researchers from Falmouth University, as well as artists picking up inspiration.

For me the keynote presentations provided some powerful concepts, Atau Tanaka, opening the event presented an thought provoking timeline in terms of his extensive performance experience. Moving from laptop computers, to mobile computing, and onwards to “post-computers”, including Beagle Boards and Raspberry Pi – as more hackable, extendible and open than more restricted mobile platforms but providing largely the same needs.

Another idea running through a moving presentation from Seth Honnor regarded the 4 degree climate change ‘elephant in the room’. While it represents such a huge un-graspable problem, he points out that everything we do needs to take it into account. It doesn’t necessarily need to be centre stage, but it has to be there – as a background future reality. If we do this we can start to build up the necessary imagination that’s going to be needed in the future.


My presence at the conference was somewhat fragmentary (I had other duties to attend to) sadly missing many of the workshops, presentations and performances – it was however a chance for me to perform for the first time in Cornwall, as well as get to see first hand some of the research that’s happening in Falmouth. The event itself was just the right size, and while at times slightly chaotic and problematic in terms of gender representation – they are things that take time to get right, and it’s freshness and interdisciplinary nature was very welcome indeed. Looking forward to next year’s event!

Update: Since writing this post, the organisers have contacted me to clarify that considerable effort was put into gender representation for the conference, there was a good balance on other presentation tracks and in terms of the keynotes it was more a case of unfortunate last minute changes and other unavoidable factors.

Compiling Scheme to Javascript

Recently I’ve been continuing my experiments with compilers by writing a Scheme to Javascript compiler. This is fairly simple as the languages are very similar, both have garbage collection and first class functions so it’s largely a case of reusing these features directly. There are some caveats though – here is an example of the canonical factorial example in Scheme:

(define factorial
  (lambda (n)
    (if (= n 0) 1
        (* n (factorial (- n 1))))))

The Javascript code generated by the compiler:

var factorial = function (n)
    if ((n == 0)) {
        return 1
    } else {
        return (n * factorial((n - 1)))

Scheme has tail call optimisation, meaning that recursive calls are as fast as iterative ones – in Javascript however, a function that calls itself will gradually use up the stack. The factorial example above breaks with larger numbers so we need to do some work to make things easier for Javascript interpreters. When considering list processing, fold can be considered a core form that abstracts all other list processing. If we make that fast, we can reuse it to define higher order list processing such as filter, which takes a function and a list and returns another list. The function is called on each element, if it returns false it’s filtered out of the resulting list.

(define filter
  (lambda (fn l)
     (lambda (i r)
       (if (fn i) (append r (list i)) r))

Compiling this with an iterative version of fold – which uses a for loop in Javascript, we generate this code:

var filter = function (fn,l)
    return (function() {
        var _foldl_fn=function (i,r) {
            if (fn(i)) {
                return r.concat([i])
            } else {
                return r

        var _foldl_val=[];
        var _foldl_src=l;
        for (var _foldl_i=0; _foldl_i<_foldl_src.length; _foldl_i++) {
            _foldl_val = _foldl_fn(_foldl_src[_foldl_i], _foldl_val); 
        return _foldl_val; })()

We have to be a little bit careful that variables created by the compiler don’t clash with those of the user’s code, so we have to make really ugly variable names. You also need to be careful to wrap code in closures as much as possible to control the scope. The problem with this is that there is probably a limit to the amount of nesting possible, especially comparing different implementations of Javascript. A interesting project that has got much further than this is Spock. More context to this work and some actual source code soon…

/* vivo */ musings

So much to think about after the /* vivo */ festival, how livecoding is moving on, becoming more self critical as well as gender balanced. The first signs of this was the focus of the festival being almost entirely philosophical rather than technical. Previous meetings of this nature have involved a fair dose of tech minutiae – here these things hardly entered the conversations.

Show us your screens

One of the significant topics for discussions was put under the spotlight by Iohannes Zmölnig – who are the livecoding audience, what do they expect and how far do we need to go in order to be understood by them? Do we consider the act of code projection as a spectacle (as in VJing) or is it – as Alex McLean asserts – more about authenticity, showing people what you are doing, what you are interacting with, and an honest invitation? Julian Rohrhuber and Alberto De Campo discussed how livecoding impacts on our school education conditioning, audiences thinking they are expected to understand what is projected in a particular didactic, limited manner (code projection as blackboard). Livecoding could be used to explore creative ways of compounding these expectations to invite changes to the many anti-intellectual biases in our society.

Luis Navarro Del Angel presented another important way of thinking about the potential of livecoding – as a new kind of mass creativity and participation, providing artistic methods to wider groups than can be achieved by traditional means. This is quite close to my own experience with livecoding music, and yet I’m much more used to thinking about what programming offers those who are already artists in some form, and familiar with other material. Luis’s approach was more focused on livecoding’s potential for people who haven’t found a form of expression, and making new languages aimed at this user group.

After some introductory workshops the later ones followed this philosophical thread by considering livecoding approaches rather than tools. Alex and I provided a kind of slub workshop, with examples of the small experimental languages we’ve made like texture, scheme bricks and lazybots, encouraging participants to consider how their ideal personal creative programming language would work. This provides interesting possibilities and I think, a more promising direction than convergence on one or two monolithic systems.

This festival was also a reminder of the importance of free software, it’s role to provide opportunities in places where for whatever reasons education has not provided the tools to work with software. Access to source code, and in the particular case of livecoding, it’s celebration and use as material, breeds independence, and helps in the formation of groups such as the scene in Mexico City.

New game design

I’m working on a new game as an art/science collaboration, and thought that it might be interesting to try the technique of making a board game first. The idea is not so much to make it work first time, but use physical pieces to figure out possible rules and try them without thinking about limitations of screen or interface.

It seems to work quite well as a way of working together as it’s simple to make situations as questions, change the meaning of pieces by drawing on them – or play with the physical arrangement. It probably bears some relation to design processes like bodystorming or lego serious play.

Users > “drivers” of software

I’ve finally had a chance to sit down and comment on John Naughton’s article in the Guardian and it’s manifesto addressed to the education secretary The Rt Hon Michael Gove MP on computer education in the UK. This joins an avalanche of recognition that programming – or “coding” – is suddenly a Good Thing for People To Know.

It is wonderful to read a major media publication pouring scorn on the old idea that “computers are like cars”, “users are drivers” which has dogged perceptions of computers for years. There is a new philosophy here focusing on hacking, in the original sense – an algorithmic literacy of processes which now mediate every aspect of our lives. This is seen as good as a general life skill, and of course good for the economy for encouraging the kind of independent thinking needed for successful startup companies.

This avalanche has to be credited to some extent to game designer David Braben and his Raspberry Pi project, and an extensive PR campaign tweaking people’s memories (including an increasing number of politicians in their 30’s and 40’s) who remember school computers like the BBC micro, and their later influence on what these same politicians like to call the “creative industries”. This of course all seems instinctively good sense for those of us who have been closely watching the boom in popularity of arduino, processing and free software methodologies in hacklabs and fablabs.

However, an approach organised on this scale is unlikely to support such generalist and creative philosophies we are used to. A few days prior to this article we had an announcement of £575m for kitting out schools with computing infrastructure from familiar public sector contractors including Serco and Capita, and a bit more detail on Staffordshire county council who are spending £28m on “Apple products under the iOS lot including iMacs, Mac Books, iPods, iPads, Mac Minis and Lion Server.”

The problem here is that a rejection of “users as drivers” is a rejection of iOS (and to a lesser extent Android) and the app store philosophy. App stores are extremely successful at promoting the idea of phones as appliances (never computers in their own right) and software as small digestible “apps” encapsulated in locked down environments generally marketed as a kind of protection for users. If these models of computing are to grow as expected – they are completely at odds with an accessible understanding we need for this change in education approach, and the creative literacy of algorithms which would follow.

When I’ve interviewed graduates for creative programming jobs the thing which is really most valuable (much more so than knowing the relevant programming language) is exactly the thing that having an “edit source code” button on every app would encourage (as included on OLPC’s sugar, another older example of an education targeted effort). What is needed is a creative lack of respect for software, a cheerful abandonment of the fear that “breaking something” will be your fault rather than that of the system.

Germination X in HTML5 canvas

I’ve spent a couple of days on an experimental not-quite-working-yet port of Germination X to HTML5. Port is probably a slightly misleading word, as it’s actually running exactly the same HaXE code, it’s just compiled to Javascript rather than swf with a load of extra Javascript filling in the stuff needed to replace Flash. The game engine that Germination X is built on is now officially cross platform! It’s great to finally have a non-proprietary browser option for the game, and HTML5 is much faster now than when I first tried it, but it’s still got a long way to go. Problems I’ve noticed so far:

It’s still slow. I’m aware of the limitations and background to browser tech, but it’s depressing that it’s somehow problematic drawing a few hundreds of sprites in 2012 while my ancient Gameboy DS can manage quite fine thankyouverymuch :)

Tinting images is quite a basic thing to do in order to efficiently use images and save bandwidth and memory. Why do I have to loop through pixels in javascript to do this? This is the main cause of slowness in the HTML5 version of the game ATM, hopefully a simple bitmap cache will fix this. There are also some neat ways to do it with offscreen bitmaps and blendmode tricks, but they seem even slower.

Having to write your own mouse event detection for sprite down/up/over/out. I guess this is one of the biggest reasons for the explosion of HTML5 game engines. It’s quite non-trivial and un-fun to set up properly. Actually getting the real mouse coordinates is also quite hard to do.

On the plus side, it’s a breath of fresh air to get back the control of immediate mode – it seems historically that this tends to be the optimal approach, give us the render loop!

Now you’ve got this far – here is some accidental art which happened this morning, and which I’m particularly proud of:

Serious/educational games interview with Kara Frame

An interview with Kara Frame, who is studying Educational Technology at San Francisco State University.

Please tell us who you are, your role in designing games and why you became interested in “serious games”?

My background is in computer graphics programming for the games and film industries, but for the last 3 years I’ve been working for FoAM, an interdisciplinary research group who encourage me to work in a more generalist manner – working with people from different backgrounds and learning skills in wildly different areas.

I’ve also been heavily involved with software art and artistic projects that use games in different ways for quite a long time. For me, “art games” and “serious games” are both taking advantage of the way games allow players to take on different perspectives as they play – this makes them very powerful in terms of exploring ideas.

How does your approach to gaming make it unique from other similar games out there?

Germination X is designed to take its raw materials from the mass of online farming games, but builds around a world inspired by alternative agricultural methods (permaculture) to see how this changes the game experience.

We are also making use of a research AI system called FAtiMA, developed by the Lirec project which supports Germination X. FAtiMA models social relationships and emotions – we are exploring how using this kind of model effects player’s understanding of their relationships (with the AI characters, plants and each other) in a social game. Permaculture is mainly concerned with the relationships between plants, so it’s exciting to bring this all together.

What do you hope participants will come away learning or experiencing from your game?

I mainly want to inspire curiosity, when a game such as this represents a certain issue, I’d like players to come to their own conclusions – to explore it with them as equals rather than having some hidden “correct answer”. So the main thing is curiosity, which requires a certain depth, some mysteriousness. The best thing is when players tell me what is going on, because this means the game has allowed them to think creatively and openly.

What have been the challenges, obstacles at creating this game of your vision?

The thing I worry most about is consistency – that the world and the themes represented make sense, nothing breaks the player’s suspended disbelief. This is always the big issue, and in this game this has caused the most discussion and debate with people I’ve worked with.

What kinds of interactive assessment methods are you taking to making the goal of your games are likely to be reached?

With the help of the Mobile Life Centre (part of the Swedish Institute of Computer Science) we are setting up a series of focus group testing sessions which will give us feedback on the nature of the relationships in the game. Group discussions while playing, and responses to more leading questions at the end will be recorded.

Also, the game has been online and public since it’s first running version, and player’s actions are logged in a minimal way which allows me to immediately see the results of changes I make. This has been quite a huge discovery for me – the form of immediate feedback this makes possible. This open testing, in conjunction with it’s open source development, has meant I’ve been able to get quite a bit of feedback from brave early adopter players.

We have yet to enable this in Germination X, but in a previous game (Naked on Pluto) we have AI agents who ask players questions relating to the theme of the game (in that case privacy in social networks) which then get automatically posted to a blog external to the game. I think building in feedback this way, whether it’s about the issues or the game itself, is really important.

What do you see as the advantages/disadvantages of games in educational use?

I think games have vast educational potential – simply because from my perspective, most of what I consider learning happens via playing. I can only understand something properly if I can pick it up, shake it, take it to pieces and rebuild it in some way (whether that’s cars, Finnish language or linear algebra).

The problem is that this approach to learning doesn’t seem to fit very well with existing educational ideas. There seems a sense of the potential, but perhaps a lack of understanding of how to achieve this. I guess one problem is that people who do “get it” are perhaps put off by education, and so are not in the right place.

Where do you see the potential of digital games as a force for individual/community change in the future?

There is something about entering a game world that allows you to take on the perspectives of people you might not normally agree with, and understand the conclusions they reach. This was most clearly demonstrated by a workshop by Selena Savic at the Make Art chmod +x festival on a game prototype that examined the differences between the business strategies of super markets and local stall markets. The workshop was carried out on a bunch of mainly left leaning open source artists, all whom when playing the supermarket side took on all the monopolistic strategies they could with wild abandon! So I hope the potential of these kind of approaches might add to a shift in politics and decision making (on personal or community levels) away from restricted partisan modes of thought.

What advice would you give a novice game builder who is considering using or designing games to use in participatory agents of change?

If computers and programming are not a natural medium for you then start with drawing, models, bits of lego, make a board game, act it out with your body. It should be possible to get basically all the decisions made this way before touching a computer. Document everything, blog about it, get as much outside input as possible.

I see a lot of educational or serious games fail because they attempt to take an existing game and “bolt it on” to an issue – this rarely works. You have to take the game design seriously, and struggle with it to fit the theme or issue you are interested in. Everything in the game has to represent the theme consistently somehow.

Serious Play at FoAM

Last weekend it was the annual FoAM gathering to discuss long term ideas, some self reflection and consideration of external perceptions. We were guided on this mission by Simone Poutnik and Hendrik Tiesinga, members of FoAM and founders of Natural Innovation. They decided to trial a new method on us all, Lego Serious Play. Now, I’m a bit battle hardened when it comes to alternative business management strategies as I’ve been exposed to quite a few in various places over the years, but despite Lego’s high scoring buzzword bingo website, I can independently verify this approach as very much a success.

You start by building personal models that represent different aspects of the organisation in various states, and then work as a group to bring them together in different ways. This is done in a structured way, leading to certain questions at different stages of the process.

The core idea was to “think with your hands” and indeed the normal problem you get in these situations, a tricky question leading to an utterly blank mind, was removed as soon as you started searching for plastic bricks. Things seemed to build themselves in some way, the increasing scarcity of bricks requiring ever greater degrees of metaphor. The indirect method of having to explain your odd constructions (usually in quite an abstract way) said a lot more than I would have managed in a more traditional situation.