Flotsam: A prototype screenless livecoding language

Two languages are working with Flotsam, the new name for the prototype screenless tangible programming language I’ve been building (which comes from the fact it’s largely made from driftwood). It’s somehow already been featured on the Adafruit blog!

The circuit seems to be fully debugged now, with short circuits fixed – which took a little while and more than a little frustration :) The Raspberry Pi python code is currently on the weavingcodes repository (more on this project on the kairotic site), and the first language is a declarative style L system for describing weave structure and pattern with yarn width and colour. The LEDs indicate that the evaluation happens simultaneously, as this is a functional language. The blocks represent blue and pink yarn in two widths, with rules to produce the warp/weft sequence based on the rows the blocks are positioned on:

The weaving simulation is written in pygame (which I’ve been using lately for teaching), and is deliberately designed to make alternative weave structures than those possible with Jacquard looms by including yarn properties. The version in the video is plain weave, but more complex structures can be defined as below – in the same way as Alex’s gibber software:

star

This is a completely different language for building shapes in Minecraft, and is an imperative, stack based language for driving a turtle in 3D space. Eventually (when I’ve manufactured a few more programming blocks) it will be possible to change Minecraft block materials and react to player actions. The LEDs indicate here the more sequential evaluation of this Forth like language:

All that’s needed to switch languages is to redraw the symbols with chalk and run a different script. It won’t be truly screenless until I write a musical language for it, which is obviously coming soon…

IMG_20141016_142021

Learning to read, notate and compute textiles in Aarhus

Setting off from Copenhagen, the weaving codes tour continued as Emma Cocker, Alex McLean, Ellen Harlizius-Klück and I sped across the Danish countryside on the train. We were heading for Aarhus to spend some time working with people at the other end of the technology spectrum – The Center for Participatory IT in Aarhus University.

We were invited by Geoff Cox to run a workshop over two days, and given the nature of the faculty and time working together being at a premium on this project, we decided to to run the workshop more as invitation to join in our work (and therefore learn a bit about how artistic research is done), rather than having specific material to cover.

Workshop participants working in Aarhus Lots of pieces of woven fabric

Alex and I needed to learn from Ellen how to ‘read’ a textile sample and notate it’s structure, a kind of reverse engineering process. Day one consisted of sharing out different types of weave and carefully figuring out the crossing points of warp and weft. The first challenge is to attempt to find the smallest repeat of the pattern, then record on graph paper a cross where the warp threads show over the weft threads. The big surprise is that the weave does not obviously relate to the visible pattern. It can also be hard to determine from a small sample which direction is warp and which is weft, so you can choose this arbitrarily if need be.

Alex recording warp crossings on graph paper

We could then compare different weaves, and also different notation styles that people used – little shorthand ways of describing large areas of plain weave for example. Much of the fabric came from Ellen’s Pepita Virus exhibition, and contained many different sizes of the “dog tooth” pattern. By comparing the notation we had all recorded for our different samples, we could tell that to scale up a pattern it’s not the case that you can also scale up the weave – the structure needs to change completely between 4 or 5 different types.

Example fabric and notation Example fabric and notation

Day two consisted of testing Alex’s Javascript loom/computer, by entering the weave structure and colour sequences and checking if the resulting patterns matched. By livecoding, we could also play with the patterns without the need to physically weave them, in order to gain a better insight into how changes affected the resulting pattern – and a better overall understanding of the weaving process.

Ellen pointing at code Two workshop participants programming patterns

To finish up, I talked about the patterns inherent in computation, based on my previous Z80 explorations to show how the pattern creating origins of computation are still present, and the role of programming languages as ways for people to come to shared understanding via notation rather than simply telling a machine what to do.

Unravelling technology in Copenhagen

Last week the weavingcodes/codingweaves project started with a trip to Denmark, our first stop was the Centre for Textiles Research in Copenhagen where we presented the project and gathered as much feedback as possible right at the beginning. The CTR was introduced to us by Eva Andersson Strand, and is an interdisciplinary centre which focuses on the relationships between textiles, environment and society.

IMG_20141006_150755

This long-view perspective of technology is critical for us, as we are dealing with a combination of thinking in the moment via livecoding and a history of technology dating back to the neolithic. This is a warp weighted loom, the focus of much of Ellen Harlizius-Klück’s research and the technology we are going to be using for the project.

IMG_20141007_114259959

Weights like this are widespread in the archaeological record for many cultures around the world, with the earliest ones around 5000 BC. Similarly – a post-it note including a handy cuneiform translation:

IMG_20141007_165613

Alex talked about livecoding as a backwards step, removing the interface – thinking about it as an unravelling of technology. His introduction to Algorave led to many connections later when Giovanni Fanfani described the abstract rhythmic patterns of Homeric rhapsodic poetry. These were performed by citizens, in a collaborative and somewhat improvised manner – the structures they form musically and in language are potentially of interest as they seem to echo the logic of weaving pattern.

IMG_20141006_143209

Ellen described her research into tacit knowledge of ancient Greek society – how weaving provided thinking styles and ordering concepts for the earliest forms of mathematics and science which is the basis for much of the weavingcodes project. One additional theme that has come up fairly consistently is cryptography – Flavia Carraro’s description of ‘The Grid in the decipherment of the Linear B writing system: a “paper-­‐loom”?’ was another addition to this area.

Emma Cocker talked about Peneolopeian time – constant weaving and unravelling as a subversive act, and the concept of the kairos, as a timely action – the name given to the point at which the weft is made when the warp ‘shed’ is provided, as well as a part of the warp weighted loom. Her input was to provide a broader view to our explorations (as coders, weavers and archaeologists all tend to get caught up in technical minutiae from time to time). From our discussions it was apparent that one of the strongest connections between livecoding and ‘weaving as thought’ is a subversion of a form of work that is considered by the dominant culture as entirely utilitarian.

A screenless programming language: putting it all together

More background on this project here. This is the finished Raspberry Pi interface board, most of the circuitry here is an LED driver to light up one of the 32 programming blocks as they are being scanned, or in any pattern required depending on the needs of the particular programming language in use. The small IC is an old 74LS02 NOR gate I’m misusing as a NOT gate (which I couldn’t find in my supplies). I hard wired one of the gate’s inputs to ground so the other one gets inverted. I need this in order to switch between the two 16 way multiplexers to get 32 outputs, by alternating their enable pins.

IMG_20141002_084650

This time I remembered to include decoupling capacitors (thanks to Julian). Here is the completed wiring for the block input plugs, with everything connected up. I’m new to this scale of building, so really pretty happy it seems to be working:

IMG_20140927_155501

The same thing, the right way up with a single programming block plugged in, and the Raspberry Pi in the foreground. The first time it’s starting to look like something:

IMG_20140927_162218

I’m also starting to document the project, as it will be released as open source hardware, here’s an unfinished schematic. This is attempting to be complete, but it’s not really as complex as it looks – I’ll work on a block diagram that makes it easier to describe:

unfinished-circuit

Screenless programming language: assembling the plug panel

Never seen so many VGA ports, ironic perhaps. It’s great that it’s transparent, but I’m regretting the material choice for the panel – acrylic is horrible to work with and leaves behind lots of microplastics pollution I have to carefully collect up. Lesson learnt for next time, lots more soldering to do now – then more testing with the Raspberry Pi. Following on from the previous post, the instruction blocks will plug in to these 9 pin plugs.

IMG_20140924_150824

IMG_20140924_165531

Migrations in django 1.7

For the wild cricket tales project I’m using django again, the all new 1.7 version which has amongst lots of other shiny things has an all new migrations system. This is so you can change the underlying database ‘model’ for your site and it automatically tracks your changes and applies them to the data (via python scripts it generates and you can tweak if needed).

However, if you trash them (my 0001_initial.py was deleted – not sure what by, or why I didn’t have them in version control but that’s another question) it’s helpful to know how to start again from scratch – the official documentation doesn’t have much to say on this topic yet. This is what I cobbled together, and it requires that your current database matches the code state exactly:

Delete all existing migrations:

rm my-app/migrations/*

Remove references to them from the database – presumably you need to be a little more selective here if you are running more than one app:

sqlite3 db.sqlite3 "delete from django_migrations"

Create new starting state:

python manage.py makemigrations --empty my-app

Fake the initial migration – this will leave the migrations system assuming that the current database state match the code but won’t actually apply the changes (i.e. making the tables again):

python manage.py migrate --fake

I’m still a little unclear how migrations in source control work across development/production servers etc, but time will tell…

Symbai field site testing

Some photos from Shakti Lamba who is currently testing Symbai in the Chhattisgarh state in north eastern India.

ByC5r0BIMAAgr6N.jpg:large

The whole system is solar powered, and provides it’s own networking via the Raspberry Pi synchronisation node shown here. The android tablets also recharge from the same power source. The Raspberry Pi networking is a direct descendant of the experiments we carried out in London during the Sonic Bike workshop.

ByDBImzIUAAw_Tw.jpg:large

Here are three of the tablets syncing their data – photographs for people to identify each other (names are used differently to western culture), audio recordings of verbal agreements (a requirement in preliterate societies), and information of who knows who.

ByC7REBIMAAToql.jpg:large

More on this free software project, links to source etc over at foam kernow.

Screenshot_2014-09-15-10-47-48

Wild Cricket Tales

We have a brand new citizen science project starting with the wild crickets research group at Exeter University! These researchers are examining how evolution works with insects in their natural environment, rather than in lab conditions. In order to do this they have hundreds of CCTV cameras set up recording the burrows of field crickets, resulting in many hundreds of hours of footage. This footage needs to be watched in order to determine the various events that make up the life story of the insects. Each individual it turns out has quite distinct characteristics, and we thought it would be fun to open up this process and make it into a citizen science project – partly to get some help and speed up the job, but also the vast quantity of material (hundreds of thousands of hours in total) has it’s own appeal – and it would be great to be able to use it for a creative project like this.

Here is an example of the footage, rather sped up – check out the frog and the sudden switch to daylight mode:

This is the first interface sketch – my plan is to focus on the individual insects and visualise the information coming together by displaying their characteristics, along with which players are their ‘biggest fans’, i.e the people who’ve put the most time into tagging them:

sketch2

The video tagging interface itself is the focus of the first prototype I’m working on at the moment. I’ve got a database set up for storing relationships between crickets, their movies and events that people create based on a combination of django and popcorn.js. Below is the first attempt, the buttons add new events as the movie plays that get recorded on the database, and displayed on the timeline bar at the bottom. Currently all players can see all the events globally, so that’s one of the first things to figure out how to handle.

ui-1

Building a screenless programming language for the Raspberry Pi #6

Following on from the last in this mini series, the giant 16 byte multiplexer is finished and tested – lots of head scratching due to shorts and wonky solder joints (it’s amazing how sensitive CMOS logic is, interference on accidental free pins causes strange effects) and I’ve replaced the breadboard with a new interface board between the Pi and the multiplexer.

IMG_20140914_163350

Now it’s time to think more about the ‘front end’ – here are some prototype programming blocks made from the ‘holes’ from a hole saw (I’m chopping up bits of driftwood). These include a central drill hole, which we can use for an indicator LED.

IMG_20140914_155709

This is the minimal circuitry needed for each programming block – it’s just a 4 bit identification code (reconfigurable via jumpers) and the LED, mounted around the 7 pin VGA style plug.

IMG_20140914_155742

Here it’s mounted in the block after a bit of hollowing out. To be fully kid-proof I’ll probably eventually set the internals in epoxy glue.

IMG_20140914_162353

Testing the plug/socket wiring on the breadboard. The plan is to paint the top surface of the programming block in blackboard paint so different instructions and languages can be tested and built with the same blocks. It also intriguingly makes abstraction possible, which is usually a problem with non-text based approaches, but this needs more thinking about.

IMG_20140916_080606

Here it is with another unenclosed block attached to the main board, and via the new interface to the Raspberry Pi. The interface will need a bit more circuitry to drive the LEDs – which will be able to be lit in different ways depending on the needs of the language, perhaps sequential in some cases, more dependant on logic in others. Also, different shaped blocks or different coloured LEDs may have different meanings in a language too.