Tag Archives: tangible programming

Picademy Exeter and Future Thinking for Social Living

Last week I had the chance to help out the Raspberry Pi foundation at their Picademy in Exeter. It was good to meet up with Sam Aaron again to talk livecoding on Pis, and also see how they run these events. They are designed for local teachers to get more confident with computers, programming and electronics to the point where they can start designing their own teaching materials on the second day of the two day course. This is a model I’m intending to use for the second inset teacher training day I’m doing next week at Truro school – it’s pretty exciting to see the ideas that they have for activities for their pupils, and a good challenge to help find ways to bring them into existence in a day.

IMG_20150605_141147

We also had the ending of Future Thinking for Social Living at the Miners Court summer party last week. We exhibited the map made during the workshops, made lots of tea, and had some fun with the pattern matrix in musical mode out in the garden – I adapted Alex’s music system we used with Ellen in Munich to run on Raspberry Pi so it didn’t require a laptop, or a screen at all – simply a speaker. It was interesting how quickly people got the idea, in many ways music is easier to explain than weaving as listening while coding is multi-sensory.

IMG_20150603_151747

Weavecoding Munich

Ellen’s exhibition in Munich was always going to be a pivotal event in the weavecoding project – one of the first opportunities to expose our work to a large audience. The Museum of casts of classical sculptures was the perfect context for the mythical aspects of weaving, overlooked by Penelope and friends with her subversive woven/unwoven work, we could explore the connections between livecoding and weaving.

IMG_8477 2

Practically we focused on developing the tangible weavecoding exhibit for events later in the week, as well as discussing the many languages we have developed so far for different looms and weaving techniques. One of our discoveries is that none of the models or languages we have created seem sufficient in themselves – weaving could be far too big to be able to be described or solved from a single perspective. We’ve tried approaches describing weave structures from the actions of the weaver, setup of the loom and structure of the fabric – perhaps the most promising is to explor the story of weaving from the perspective of the thread itself.

IMG_20150510_062737

IMG_20150508_153211

One of the distinctive things about weaving in antiquity is how multiple technologies were combined to form a single piece of fabric, weaving in different directions, weft becoming warp, use of tablets vs warp weighted weaving. To explain this via the path of a single conceptual thread crossing through itself may make this possible to describe in a more flexible, declarative and abstracted manner than having to explain each method separately as if in it’s own world.

IMG_20150508_152045

IMG_20150509_100421

The pattern matrix has now been made into good shape for explaining the relationship between colour and structure in pattern formation. For the first time we also used all 4 sensors per block on the bottom row which meant we could use a special “colour” block that the system recognises from the normal warp/weft ones and use it’s rotation to choose between 8 preset colour settings. This was quite a breakthrough as it had all been theoretical before.

IMG_20150508_194220

Adding this more complex use of the magnetic patterns meant that Alex could set up the matrix as a tangible interface for his tidal livecoding software meaning Ellen could join us for a collaborative slub weavecoding performance on the Saturday evening. The prospect of performing together was something we have talked about since the very beginning of the project, so it was great to finally reach this point. The reverb in the museum was vast, meaning that we had to play the space a lot, and provide ‘music for looking at sculptures by’:

Tangible programming: detecting flip, rotation and id with magnets

When we started designing the pattern matrix we wanted to include the possibility of encoding more than binary (which side is up) using the magnets. In order to test this, we made the bottom row of sensors with 4 in a square – the rest only have one sensor currently (to avoid blowing the budget on hall effect sensors).

Here are some test blocks with four magnets glued on. The one at the back is easy to make as they naturally snap together edge to edge in this pattern, the closer one required superglue and lots of patience – I’m still expecting it to fire a magnet off unexpectedly at some point:

IMG_20150429_153706

The orientation seems to work well in our tests so far, as you rotate the blocks the sensors latch from one state to the other – and it seems like they stick to their previous reading until the block is very nearly aligned straight. I’ve added some sound on the Pi to give some haptic feedback which is turning out to be very useful.

The next job was to head back to makernow make some better blocks with the magnets inside. Oliver Hatfield milled out new holes in some of our spares:

IMG_20150501_114719

Luckily the fit is really tight so with some force the magnets can be placed inside without the need for any gluing – and they don’t rattle around at all:

IMG_20150501_115947

The next thing was to make some visual indication of the polarity and meaning of the patterns, and show how the binary encoding changes with flipping and rotating. Andy Smith designed and laser engraved these new caps and locating rings:

IMG_20150501_180731

The 4 bit binary codes read in clockwise order from the top left (same as the notation for tablet weaving) so rotation causes the same effect as bitwise rotate in programming – multiply/divide by 2 with overflow. There are 4 possible different configurations of magnets (which can provide block identification). Two of the configurations are mirrored on both sides but you can read rotation still, with the other two you also can tell which side is up, and one – bottom left in the photo below, can represent 8 states all by itself (flip as well as rotate).

In future we’ll make more of these with specific meanings dependant on the language we use them for and what they actually do – at this point they are for debugging/experimenting further.

IMG_20150502_124856

Pattern matrix – putting it together

Here is a member of staff at Miners Court trying some tangible weave coding in the midst of our crafts area – at the moment it’s simply displaying the weave structure on the simulated warp weighed loom with a single colour each for warp and weft threads, the next thing is to get ‘colour & weave’ patterns working.

DSC_1064

The pattern matrix is the second generation of tangible programming device from the weavecoding project. It’s been built as an open hardware project in collaboration with Falmouth University’s Makernow fablab, who have designed and built the chassis using many 3D printed parts and assembled the electronics using surface mount components (far beyond my stripboard skills).

Here you can see the aluminium framework supporting the AVR based row controller boards with the Raspberry Pi in the corner. The hall effect sensors detect magnetic fields – this picture was taken before any of the wiring was started.

IMG_20150408_105446

The row controllers are designed to read the sensor data and dispatch it to the Raspberry Pi using i2c serial communication running on their atmega328 processors. This design was arrived at after the experience of building flotsam which centralised all of the logic in the Raspberry Pi, resulting in lots of wiring required to collect the 128 bits of information and pass it to the GPIO port on the Pi. Using i2c has the advantage that you only need two wires to communicate everything, processing can be distributed and it can be far more modular and extendible in future. In fact we plan to try different sensors and configurations – so this is a great platform for experimenting with tangible programming.

This video shows the current operation of the sensors and row controllers, I’ve programmed the board with test code that displays the state of the magnetic field with the status LED, making sure that it can tell the orientation of the programming block:

The row controllers have a set of multiplexers that allow you to choose between 20 sensor inputs all routed to an analogue pin on the AVR. We’re just using digital here, but it means we can try totally different combinations of sensors without changing the rest of the hardware.

After getting the first couple of rows working and testing it with elderly people at our Miners Court residency there were a couple of issues. Firstly the magnets were really strong, and I worried about leaving it unattended with the programming blocks snapping together so violently (as we plan to use it in museum settings as well as at Miners Court). The other problem was that even with strong magnets, the placement of the blocks needed to be very precise. This is probably to do with the shape of the magnets, and the fact that the fields bend around them and reverse quite short distances from their edges.

To fix these bugs it was a fairly simple matter to take the blocks apart, remove 2 of the 3 magnets and add some rings to guide placement over the sensors properly:

IMG_20150418_114347

Future Thinking for Social Living: Weavecoding in assisted housing

Our work on weavecoding is now reaching out to other uses and projects. One is Future Thinking for Social Living, run by Magda Tyżlik-Carver and Fiona Hackney.

This research project aims to look at the relationship between wellbeing, home, making and technology and is centred on Miners Court, who provide assisted housing in Redruth in Cornwall. As well as a range of flats and accommodation, the residents have shared communal areas with a variety of activities throughout the week. Along with Christiane Berghoff, Robin Hawes and Lucie Hernandez we set up camp with a lot of materials for knitting, crochet and weaving as well as some Raspberry Pis and the all new pattern matrix tangible weavecoding device.

miners

The Future Thinking for Social Living project is set up to research how we can think more critically about home and community, and with particular focus on the future. From discussions with the staff at Miners Court – specific issues they are interested in are how to make better use of communal spaces, and how can they get more men involved with crafts and shared activities.

I’m also interested in how we can use these settings for artists residencies – how does working with people like this affect a design process, does working in such a place – and using it as way to start conversations (rather than being too much in ‘teacher mode’) affect the people living there positively? Also the weavecoding project provides some ideas in bridging gaps, both between technology and people – but also across gender gaps, mixing textiles with electronics for example.

miners2
Here is the new magnetic pattern matrix, running the 3D Raspberry Pi warp weighted loom simulation (more on this soon!) with a nice 4 shaft loom in the background.

On Monday and Tuesday we spent a long time talking, weaving, knitting and making cups of tea of course (and a bit of time debugging magnets on my part). I’ve found helping people weave with tablets on the inkle loom is a good way to get talking, as this seems new to even people who are experienced with crafts. It also appeals to people with mathematics or design background who normally are uninterested in knitting and other crafts, and seems gender neutral perhaps for the same reasons. It also helps to talk about the history of what we are weaving with, the fact that this is an ancient technique and yet there are so many surprises – I can’t really predict to them what will happen e.g. to the pattern when we change rotation direction, and this seems to be important.

What we have yet to do (but a few weeks to experiment yet) is bridge the technology gap. Many of them have an immediate reaction of distaste to computers, as most of them have them but report that they have become unusable or feel that they are not designed well with their needs in mind. Partly the situation of having some circuit boards getting tangled up in the more familiar materials and using the Raspberry Pi simulation to show what is happening on the loom next to it is a start. One interesting thing is that neither the Pi nor the AVR boards look enough like ‘a computer’ for it to stand out too much (which also part of the Pi’s role in the classroom) – this was more so after plugging it into their large TV and getting rid of the monitor. As it gradually gets into a working state, I’d like to first try using it to demonstrate well known weaves – e.g. plain, twill and satin.

Working in this environment on the pattern matrix between weaving with different people has already had an effect on it’s design process. One initial observation resulted in reducing the magnet strength – I hadn’t even considered before that having them snap together too forcefully would be a problem for some people. Such things are obvious in these kinds of settings.

Screenless music livecoding

Programming music with flotsam – for the first time, it’s truly screen-less livecoding. All the synthesis is done on the Raspberry Pi too (raspbian release in the works). One of the surprising things I find with tangible programming is the enforced scarcity of tokens, having to move them around provides a situation that is good to play around with, in contrast to being able to simply type more stuff into a text document.

The programming language is pretty simple and similar to the yarn sequence language from the weavecoding project. The board layout consist of 4 rows of 8 possible tokens. Each row represents a single l-system rule:

Rule A: o o o o o o o o
Rule B: o o o o o o o o
Rule C: o o o o o o o o
Rule D: o o o o o o o o

The tokens themselves consist of 13 possible values:

a,b,c,d : The 'note on' triggers for 4 synth patches
. : Rest one beat
+,- : Change current pitch
<,> : Change current synth patch set
A,B,C,D : 'Jump to' rule (can be self-referential)
No-token: Ends the current rule

The setup currently runs to a maximum depth of 8 generations – so a rule referring to itself expands 8 times. A single rule ‘A’ such as ‘+ b A - c A ‘ expands to this sequence (the first 100 symbols of it anyway):

+b+b+b+b+b+b+b+b+b+b-c-c+b-c-c+b+b-c-c+b-c-c+b+b+b-c-c+b-c-c+b+b-c-c+b-c-c+b+b+b+b-c-c+b-c-c+b+b-c-c

I’m still working on how best to encode musical structures this way, as it needs a bit more expression – something to try is running them in parallel so you can have different sequences at the same time. With a bit more tweaking (and with upcoming hardware improvements) the eventual plan is to use this on some kid’s programming teaching projects.

New tangible weavecoding device – pattern matrix

We’re starting construction of version 2 of the flotsam tangible programming device, specialised to weaving – and henceforth known as the ‘pattern matrix’. This will be tested during May at our upcoming performance/workshop/residency at Munich’s Museum für Abgüsse Klassischer Bildwerke (Museum of Casts of Classical Sculpture) with the Coding weaves project, and then for later use in Cornwall (more on that part soon).

1

The first thing we are exploring is removing the need for physical plugs – although I like them a lot, they are problematic for people as it takes time to learn how to align the blocks in the current prototype. In order to get around this, and maintain the cheapness of the programming blocks themselves we’re looking at using magnetism to represent information. We can use blocks with no connections, painted white and black on different sides and detect their orientation and position via a magnet in the centre.

Initially this idea came from thinking about reed switches with Francesca, and playing with mobile phone magnetometers on the UAV project led to us investigating Hall effect sensors (the building blocks of magnetometers). We had a bit of a testing workshop with Andy from the Falmouth University makernow fablab who are helping with construction of this project.

halleffect

Hall effect sensors allow us to detect the polarity of nearby magnetic fields – and seem to be restricted enough in range that they can be very precise. Even with fairly weak magnets we found we could put the sensors right next to each other (see above) and still determine the difference between two opposed or aligned fields.

For the warp/weft weave pattern structure we only need 1 bit of information to be detected, but for future extensibility for the yarn colour programming setup it’s important to be able to read more (4 bits are encoded in the flotsam blocks).

Our plan is to try putting 4 sensors in a square which adds an intriguing possibility of rotating the blocks to change their meaning, as well as flipping them. The great thing is that this gets very close to tablet weaving in terms of the notation and the actions required. We can also represent all 16 states with only 4 blocks – if negative is 0 and positive is 1, and we read the code as binary clockwise from top left:

Starting state [0,1,5,6]
- -   + -   + -   - +
- -   - -   - +   - +

Rotate clockwise [0,2,10,12]
- -   - +   - +   - - 
- -   - -   + -   + +

Horizontal flip [15,11,10,12]
+ +   + +   - +   - - 
+ +   + -   + -   + +

Rotate counter-clockwise [15,13,5,6]
+ +   + -   + -   - + 
+ +   + +   - +   - +

Vertical flip [0,4,5,6]
- -   - -   + -   - + 
- -   - +   - +   - +

Here is Andy’s design for the PCB we’ll use under each of the 25 board locations:

hallboard

Tangible livecoding tests in the wild, and material as type in programming

Last week I took the flotsam tangible livecoding system to my programming tutoring lesson for some first tests with the real experts. To provide some background, we started a while back with Raspberry Pi, messing around with the Minecraft API and python and we’ve recently moved on to laptops and pygame. I arrived with the system set up for the Minecraft building language, and we gave it 10 minutes or so before resuming normal activities – although it did get a bit more use during natural breaks in the lesson. Here’s a recent pic of the weaving l-system setup:

tangibleweaves

First impressions were that it was immediately playful, most of the blocks were eagerly removed and examined before we’d turned the thing on. One problem with this was that the chalk symbols rubbed off very quickly! A similarity with the Scratch programming language was also picked up on fairly soon.

A big issue was getting the connectors the right way round – this is not easy as the blocks are circular with little indication of which way is ‘up’. This could be fixed by altering the shape or cutting a little notch – learning how to manipulate them is part of the point, but right now it’s too difficult. This will also be a big problem in livecoding performance situations.

Using Minecraft, the screen was still a distraction. The connection between what’s happening in the 3D world and with the physical blocks is not obvious enough – even with the LED indicators. Also the mouse and keyboard need to be plugged in so we can move the camera around and see stuff, leading to a few too many things going on. I’m not sure how this can be solved regarding Minecraft, but indicates that a musical approach (with no screen or any other peripherals needed other than speakers) is the way to go.

Overall there is huge potential to think much more about the touch/feel of the blocks. Lots of these problems can be solved by using different shapes for different symbols (removing the need for chalk) with a clear orientation. Using different materials to provide textures and ‘feel’ in order to represent different sounds, or in terms of weaving, using the actual yarn material to represent itself is a huge area to explore.

materials

In the photo above, I’m using thick and thin blue yarn wrapped around the blocks that represent them, and tinfoil for l-system rule symbols (X, Y and Z) – this is a kind of material based type indication. Interestingly the yarn feels very different even though it looks the same in the photo. In use this results in much less checking of the block when you pick them up, as your fingers tell you what they are.

dremelling