dave’s blog of art and programming

user generated content

Betablocker-DS can now be programmed, and I’ve spent the last week testing it on public transport (with headphones…) Lots of niggles to fix, and not enough samples, but the basic interface works.

The DS has such a nice feel – it also seems like a more appropriate interface than the gamepad version. It’s still not really suitable for public consumption, but I’ve updated the binary here. Also I’ve started some minor documentation effort here. This version has a slightly altered instruction set to the original version and is still subject to extensive fiddling!

A holiday project run amok, an idea to make version of betablocker for musical livecoding on minimal hardware – eschewing the world of yuppie smartphones for the gameboy ds.

The betablocker virtual machine is up and running, sound is working (although only one sample at the moment) and the tiles are rendered using nintendo’s 2D hardware, with really nice smooth scrolling – as you’d expect I guess. The icons need a bit of work yet, and I’m only using the top screen for debugging at the moment.

The instructions are represented by images, which are designed to only be a little more cryptic than assembler instructions tend to be. No actual livecoding is possible yet, that’s next. Code here, executable here (you’ll need some additional kit to run it on hardware). I’ll provide some level of documentation, when it is actually working.

I have a problem when livecoding music, in that while I’m happy livecoding synth graphs to make sounds, I sometimes get a bit stuck coming up with patterns of notes for them to play. I generally start with something like (note (modulo clock 8)) and work my way fairly randomly from there.

In need of inspiration for making more complicated patterns with the minimum of code I thought I’d continue my sporadic ongoing mission to explore genetic programming (towards the much desired future of a machine programming itself). The idea is to use a fitness function to grow programs that make patterns I can specify. I could then use these programs as they are, generate them live, or steal ideas from the evolved code.

To give an example of a pattern, this is the first 13 notes of the famous techno number “mary had a little lamb” in major scale:

(2 1 0 1 2 2 2 1 1 1 2 4 4)

The first thing we need is an instruction set that can be used in machine generated programs. These need to be robust to any inputs, and rich enough to provide the sort of patterns we want to generate. I’ve started with a fairly minimal set:

(mod a b) : return modulo of a to b, or zero if b is zero.
(odd a) : return 1 if a is odd, 0 otherwise.
(even a) : return 1 if a is even, 0 otherwise.
(min a b) : return the minimum of a or b.
(max a b) : return the maxium of a or b.
(+ a b . . .) : addition.
(- a b . . .) : subtraction.
(* a b . . .) : multiplication.
(/ a b) : division – returns zero if b is zero

In order to drive the program we also need a (clock) function that returns the time – or more precisely, the place in the pattern we are in.

So the program: (+ (mod (time) 3) 2) repeated 5 times would result in the pattern (2 3 4 2 3) .

We also need a fitness function which will give us a measure of how close a pattern is to the one we are trying to find a program for. This could be the sum of the differences between each element of the generated pattern and that of the target pattern – where 0 is perfect and the bigger the number the worse the fit, eg:

(define (fitness pattern target)
    (foldl
        (lambda (a b r)
            (+ r (abs (- a b))))
        0 pattern target))

Lets try something simple to begin with – the pattern (50 0 50 0 50 0). We start with a population of 1000 randomly generated programs and pick the best one – hold on to your hats:

21

A complex program that surprisingly results in the pattern: (21 21 21 21 21 21). This has a fitness of 150 – and 21 is quite close to the average of 0 and 50, so although it’s disappointing, it makes sense.

We now create a new population, some of the individuals are new random programs, others are mutated versions of the previous best attempt (25%/75% if you really want to know). We also include the previous best attempt without mutation – this stops generations getting worse over time. Unfortunately in this run it takes 90 generations before a fitter individual is found:

(* 35 (even (clock)))

Which generates the pattern: (35 0 35 0 35 0) with a fitness of 45. Now it is only a matter of time, in fact the next 3 generations slowly home in:

(* 55 (even (clock)))
(* 52 (even (clock)))
(* 49 (even (clock)))

and then then on generation 104 we finally get:

(* 50 (even (clock)))

resulting in (50 0 50 0 50 0), with a fitness of 0.

More soon – code here.

Scheme bricks was originally designed as a visual programming interface for functional reactive programming using frisbee – an experimental fluxus based game engine built on top of PLT’s FrTime language. I’ve ended up spending the last 18 months beta testing it in livecoding performances with slub using fluxa, culminating in the workshop last week.

This example is one of the “hello world” type scripts in fluxus, a recursive cube structure. It’s the first time I’ve tried this with scheme bricks. Having the interface in the same world as the rest of the objects opens up lots of possibilities, and enters the realm of IOhannes m zmölnig’s “do sinusoids dream of electric sweeps” performances in pure data – code could be written to modify the representation of itself.

Higher priority though, is to spend some time on making the interface itself easier to use :)

A self building text adventure game which starts off with an empty world:

You build new places with “build my house n” (which means build my house to the north) then you can go there “n” and give it a description “describe a blue house with red doors”. You can also make objects with eg “make cheese” and take it (when it will be added to your inventory), move around and drop it elsewhere.

If you type “dot” you get a text description of the world which can be read by graphviz. Here is part of my mental model of Helsinki:

As it says in the embedded page, it’s very much inspired by Craig Latta’s quoth musical livecoding system, which uses an interface like this to create musical entities you can interact with.

A picture from the livecoding workshop in Antwerp, the first time scheme bricks has been exposed to so many people. We covered various synthesis techniques, as well as introducing livecoding more generally. This has given me a lot of motivation to move this stuff further – I’m thinking of putting it in the fluxus distribution as an alternative livecoding option.

The most gloriously aptly named festival for livecoding I’ve ever seen, the Lambda Festival in Antwerp this weekend will see quite a lot of action from Slub. Firstly we are VJing on the Saturday, then the first ever Scheme Bricks workshop on Sunday followed by a performance in the evening.

I’m still trying to work out exactly how to livecode this, but I’ve written a little language for building robots in fluxus:

(Audio: “Dang Spot” by Plaid)

These fellows were premiered at Lazy Sunday yesterday, raving to DJ Conehead’s early 90s hardcore techno lushness.

The idea (inspired by Gabor Papp’s marching maggots) was to make a high level language in fluxus which takes care of all the physics operations – automatically creating active objects and joints in ODE. The code can be found here.

This weekend it’s Lazy Sunday (organised by the The Planet of Leather Moomins) – a distributed streamed festival from Helsinki, London, New York, Paris, Stockholm, Barcelona, Montreal – well you get the idea… I’m going to be doing some visuals livecoding, possibly something very new, which usually means it won’t work properly!

Finally an actual release of scheme bricks! (make sure you read the README, it’s not user friendly yet).

I’ve unified the maths operations, previously you had to remember if you were working on nodes or numbers (the only two types in fluxa) now you can just use the standard + – * / (before you had to use add sub mul div for graph nodes). This makes it much nicer to program. I’ve also replaced the asterisk in the font, the old one looked like a smudge on the screen.

This is all in preparation for a scheme bricks livecoding workshop I’m giving in a few weeks at the LAMBDA ELEKTRONISCH MUZIEKFESTIVAL.