With my mind on our upcoming AHRC weave/code project (and seeing as Alex has already started writing code) I thought I’d have a go at visualising how computers work in relation to pattern manipulation. These screenshots are from a ZX Spectrum where I’ve modified some library assembler code for higher level arithmetic to display the contents of 7 CPU z80 registers graphically between each instruction – time runs from top to bottom.
Most processors don’t actually have circuits for mathematics, they simply implement ‘add’ along with bitwise instructions for ‘and’, ‘or’, ‘not’, ‘xor’ and a handful of instructions for shifting the bits left and right. This is true even with modern CPU’s where the arithmetic instructions for multiply, divide etc are built with hidden ‘microcode’ routines. For this reason the underlying operation of a computer has more to do with patterns than it does with concepts such as language or even numbers as we normally think of them.
The simplest (and shortest) are multiply in 8 bits. In this function, the ‘a’ register contains one number and the ‘h’ register contains the other – at the end the ‘a’ register contains the result. In the first screenshot the numbers are fairly simple so it’s possible to see what’s going on (ie. in 1*1 the ‘a’ and ‘h’ registers both contain 00000001)
170 in 8 bits looks like ‘10101010’ so easy to see – here are some different ways of reaching the same answer:
16bit multiply operates over 2 registers – the first value is stored in ‘h’ and ‘l’ and the other is on the stack, but is loaded into ‘d’ and ‘e’ after a few instructions:
43690 is ‘1010101010101010’ so in the first run here we multiply it by one as a test:
Some 16 bit divides – these take a longer time to calculate, so a whole page for all the instructions involved, and I have no idea how this works:
65535 is the largest value we can store, divide by itself to end up with 1:
I’ve recently been building the Mongoose 2000 “group composition” tool that the researchers will use for recording information about a whole pack of mongooses (and synchronise data via a Raspberry Pi providing a local wifi node) in their field site in Uganda. As I wrote a bit about before, one of the interesting things about this project is that the interface design has to focus on long term speed and flexibility over immediate ease of use. In this way it seems appropriate that it’s moving in the direction of a musical interface rather than a normal touch screen interface. The different colours in the mongoose selectors show which individuals are present and which have data recorded from them already, the screenshot below is the section where they record relationships between the adult females (at the top) and adult males that may be guarding – or pestering them (below). At the same time, they need to be able to record events that may be occurring with the pack as a whole – in this case an interaction with another pack of mongeese.
The source code is here, it’s the first time I’ve used WebGL for a game, and it’s using the browser version of fluxus. It worked out pretty well, even to the extent that the researchers could edit the code themselves to add new explanation screens for the genetics. Like any production code it has niggles, here’s the function to render a butterfly:
(define(render-butterfly s)(with-state;; set tex based on index(texture(list-ref test-tex (butterfly-texture s)));; move to location(translate(butterfly-pos s));; point towards direction(maim(vnormalise(butterfly-dir s))(vector001))(rotate(vector09090));; angle correctly(scale(vector0.50.50.5));; make smaller(draw-obj4);; draw the body(with-state;; draw the wings in a new state(rotate(vector18000))(translate(vector00 -0.5));; position and angle right;; calculate the wing angle based on speed(let((a(-90(*(butterfly-flap-amount s)(+1(sin(*(butterfly-speed s)(+(butterfly-fuzz s)(time)))))))))(with-state(rotate(vector00 a))(draw-obj3));; draw left wing(with-state(scale(vector1 -11));; flip(rotate(vector00 a))(draw-obj3))))));; draw right wing
There is only immediate mode rendering at the moment, so the transforms are not optimised and little things like draw-obj takes an id of a preloaded chunk of geometry, rather than specifying it by name need to be fixed. However it works well and the thing that was most successful was welding together the Nightjar Game Engine (HTML5 canvas) with fluxus (WebGL) and using them together. This works by having two canvas elements drawn over each other – all the 2D (text, effects and graphs) are drawn using canvas, and the butterflies are drawn in 3D with WebGL. The render loops are run simultaneously with some extra commands to get the canvas pixel coordinates of objects drawn in 3D space.
On Saturday I teamed up with Falmouth University’sMakernow team to do a kids coding event at the Royal Cornwall Show with a new Raspberry Pi cube (based on the one used at the DeerShed Festival last year). We had a constant stream of families and kids wanting to try Scratch coding, and we had a 50/50 gender balance in terms of helpers which I think with these sorts of events is critical.
The part I like best about public events like this are talking with the parents and teachers. The best questions are the fundamental ones: “why should we be teaching them how to program?” which was a great opportunity to get my thoughts straight – the official economic reason is not one I’m so motivated by: “to encourage more talent in the tech sector”. The actual reason I do all this (mostly on a voluntary basis) is the feeling that as we find ourselves living in a computational society, where everything we do is algorithmically processed, the future for people who only know how to consume technology is very different from those who are not afraid to question it, who know it’s possible to take it to bits and rebuild it in new ways.
This is also the reason that I can’t get very excited when teachers tell me they are buying iPads for use in their school – there are some interesting programming environments on them, but the kind of creativity they support cannot, due to Apple’s core business model, encompass this kind of questioning – they can’t escape the sandbox. For example, when I last checked, you can program iPads, but not share the code or work collaboratively as it would bypass the AppStore distribution model to do this.
This relates to the answer I gave to “why should we get a Raspberry Pi” – as it provides a platform that encourages a fearless relationship with technology, it doesn’t have the family email account on it, it’s cheap and nobody cares if you manage to delete the entire operating system, just copy a new sdcard. This stuff has to be possible, and encouraged – if we are to eventually have a society that can have any meaningful debate on increasingly thorny computational/network/society issues such as those related to GCHQ mass spying.
Back to the kids – it was interesting that the majority of the older ones had already used Scratch, either as part of their normal school lessons, or an after school activity (anecdotally, this is sharp improvement over the last year or so). Some of them were keen to show off their skills, which was a great way to demonstrate to the younger ones what was possible.
For the older ones I’m continuing work on the Minecraft API coding project – making simple 3D primitives to demonstrate functional programming in Python. You can explore the results of your programs by walking around and digging into structures generated in a familiar world. More on new versions of that soon.
Bumper crop is an android game I’ve just started working on with Dr Misha Myers as part of the Play to Grow project: “exploring and testing the use of computer games as a method of storytelling and learning to engage urban users in complexities of rural development, agricultural practices and issues facing farmers in India.”
(Warning – contains machine translated Hindi!)
I’m currently working out the details with artist Saswat Mahapatra and Misha, who have been part of the team developing this game based on fieldwork in India working with farmers from different regions. They began by developing a board game, which allowed them to flexibly prototype ideas with lots of people without needing to worry about software related matters. This resulted in a great finished product, super art direction and loads of assets ready to use. I very much like this approach to games design.
From my perspective the project relates very closely to groworld games, germination x, as well as the more recent farm crap app. I’m attempting to capture the essence of the board game and restrict the necessary simplifications to a minimum. The main challenge now that the basics are working is providing an approximation of bartering and resource management between players that board games are so good at, into a simple interface – also with the provision of AI players.