dave's blog of art and programming

missing acceptable use policy

Some random pictures from some excellent (and at times very loud) few days in Paris. Thanks to Sony Computer Science Labs and also the Paris contingent of the TPOLM demo crew for coming out to support us.

The gig was extensively recorded, so hopefully more about this soon.

So you have your sample loading working in a separate thread to your realtime audio code and the interface i/o communication is also tucked away in it's own thread. You have a lock free mechanism for communicating between all threads and still, sometimes - when loading data from disk jackd chucks your program off for blocking - usually during a live performance, and you have to fumble around restarting the audio server. What's going on?

Well, it turns out that when your hard drive goes to sleep, the spin up resulting from a disk access request (such as mid-performance sample loading) can cause the kernel to block, which causes jackd to panic and either stop entirely or detach your client. This is even with jackd running with the --nozombies switch, so it must be the watchdog thread at work.

The solution is simply to prevent your drive spinning down when you are performing, by adding "hdparm -S 0 /dev/yourharddisk" to your live audio startup script. Thanks to ClaudiusMaximus for helping me fix this at Piksel - it's been bothering me for ages.

Piksel 2010 (un)stable was the 8th piksel, and the second time I've been lucky enough to participate. I was there to present Naked on Pluto and general livecoding duties.

I didn't have as much time or energy to get involved with the other things going on as I'd have liked, so this is mainly a report on my activities. I did see a super talk by Audun Eriksen about the visual programming language scratch and it's use for teaching kids to make games - very inspiring stuff.

Scratch was where I started with the idea for scheme bricks, so it was great that the next day Alex and I had the functional livecoding workshop, where people could get their hands dirty with some of our live coding performance ideas.

This was followed in the evening by a slub performance, where we got a little carried away by the responsive audience and the rather nice sound system. It's always good to do performances with stuff you've done a workshop on earlier as the participants can put it in context (and tell other people what's going on!). The image is a link to a video of the performance on giss.tv.

The next day was a switch to Naked on Pluto with Aymeric and a presentation where we discussed clouds, problems with social media, farmville and the intricacies of the facebook graph api.

(pictures thanks to Régine Debatty)

It was great to get questions from people who have tried playing the game before seeing our talk. Hopefully this can be the case as we discuss the project more widely.

I have a problem when livecoding music, in that while I'm happy livecoding synth graphs to make sounds, I sometimes get a bit stuck coming up with patterns of notes for them to play. I generally start with something like (note (modulo clock 8)) and work my way fairly randomly from there.

In need of inspiration for making more complicated patterns with the minimum of code I thought I'd continue my sporadic ongoing mission to explore genetic programming (towards the much desired future of a machine programming itself). The idea is to use a fitness function to grow programs that make patterns I can specify. I could then use these programs as they are, generate them live, or steal ideas from the evolved code.

To give an example of a pattern, this is the first 13 notes of the famous techno number "mary had a little lamb" in major scale:

(2 1 0 1 2 2 2 1 1 1 2 4 4)

The first thing we need is an instruction set that can be used in machine generated programs. These need to be robust to any inputs, and rich enough to provide the sort of patterns we want to generate. I've started with a fairly minimal set:

(mod a b) : return modulo of a to b, or zero if b is zero.
(odd a) : return 1 if a is odd, 0 otherwise.
(even a) : return 1 if a is even, 0 otherwise.
(min a b) : return the minimum of a or b.
(max a b) : return the maxium of a or b.
(+ a b . . .) : addition.
(- a b . . .) : subtraction.
(* a b . . .) : multiplication.
(/ a b) : division - returns zero if b is zero

In order to drive the program we also need a (clock) function that returns the time - or more precisely, the place in the pattern we are in.

So the program: (+ (mod (time) 3) 2) repeated 5 times would result in the pattern (2 3 4 2 3) .

We also need a fitness function which will give us a measure of how close a pattern is to the one we are trying to find a program for. This could be the sum of the differences between each element of the generated pattern and that of the target pattern - where 0 is perfect and the bigger the number the worse the fit, eg:

(define (fitness pattern target)
    (foldl
        (lambda (a b r)
            (+ r (abs (- a b))))
        0 pattern target))

Lets try something simple to begin with - the pattern (50 0 50 0 50 0). We start with a population of 1000 randomly generated programs and pick the best one - hold on to your hats:

21

A complex program that surprisingly results in the pattern: (21 21 21 21 21 21). This has a fitness of 150 - and 21 is quite close to the average of 0 and 50, so although it's disappointing, it makes sense.

We now create a new population, some of the individuals are new random programs, others are mutated versions of the previous best attempt (25%/75% if you really want to know). We also include the previous best attempt without mutation - this stops generations getting worse over time. Unfortunately in this run it takes 90 generations before a fitter individual is found:

(* 35 (even (clock)))

Which generates the pattern: (35 0 35 0 35 0) with a fitness of 45. Now it is only a matter of time, in fact the next 3 generations slowly home in:

(* 55 (even (clock)))
(* 52 (even (clock)))
(* 49 (even (clock)))

and then then on generation 104 we finally get:

(* 50 (even (clock)))

resulting in (50 0 50 0 50 0), with a fitness of 0.

More soon - code here.

A new month, and the start of various new things around here. One is a slub residency with sketchpatch, where we write processing sketches and bounce them between us in a game of "sketch tennis" that lasts a month. We already have a bunch of sketches and it's only the first day.

I'm also taking part in a collaboration with Marloes de Valk and Aymeric Mansoux involving social games, text adventures and far distant planets. This is quite a new direction for me, and it's exciting to be forging new alliances. A bit more explanation coming soon!

On the groworld front, we are also doing some design and further development surrounding permaculture, in what could become quite a big game project. Again - more on that as it happens.

Scheme bricks was originally designed as a visual programming interface for functional reactive programming using frisbee - an experimental fluxus based game engine built on top of PLT's FrTime language. I've ended up spending the last 18 months beta testing it in livecoding performances with slub using fluxa, culminating in the workshop last week.

This example is one of the "hello world" type scripts in fluxus, a recursive cube structure. It's the first time I've tried this with scheme bricks. Having the interface in the same world as the rest of the objects opens up lots of possibilities, and enters the realm of IOhannes m zmölnig's "do sinusoids dream of electric sweeps" performances in pure data - code could be written to modify the representation of itself.

Higher priority though, is to spend some time on making the interface itself easier to use :)

The most gloriously aptly named festival for livecoding I've ever seen, the Lambda Festival in Antwerp this weekend will see quite a lot of action from Slub. Firstly we are VJing on the Saturday, then the first ever Scheme Bricks workshop on Sunday followed by a performance in the evening.

Alex Mclean, Nick Collins and I have written a paper for EVA 2010 on Visualisation of Live Code (link to preprint version).

In this paper we outline the issues surrounding live coding which is projected for an audience, and in this context, approaches to code visualisation. This includes natural language parsing techniques, using geometrical properties of space in language semantics, representation of execution flow in live coding environments, code as visual data and computer games as live coding environments. We will also touch on the unifying perceptual basis behind symbols, graphics, movement and sound.

The paper includes betablocker, daisy, al jazari and scheme bricks.

Good pics of slub taken by Evan Raskob at the anatomy theatre performance a couple of days ago:

Live coding dissections by Michele Pasin, Wrongheaded, Thor Magnusson, Slub and Evan + Jag at Kings College Anatomy Museum on the 14th of January, more info here.