Category Archives: android

Hindi translations in Symbai

A couple of screenshots of the hindi version of Symbai – our solar powered Raspberry Pi/Android anthropological research tool. As is usual we’re still having a few issues with the unicode but it’s nearly there. We’ve been working on this software for the last few months, making sure the data (including photos and audio recordings of verbal agreements) synchronise properly across multiple devices.

Screenshot_2014-09-15-10-45-14

Screenshot_2014-09-15-10-45-45

Mongoose 2000: Group composition

I’ve recently been building the Mongoose 2000 “group composition” tool that the researchers will use for recording information about a whole pack of mongooses (and synchronise data via a Raspberry Pi providing a local wifi node) in their field site in Uganda. As I wrote a bit about before, one of the interesting things about this project is that the interface design has to focus on long term speed and flexibility over immediate ease of use. In this way it seems appropriate that it’s moving in the direction of a musical interface rather than a normal touch screen interface. The different colours in the mongoose selectors show which individuals are present and which have data recorded from them already, the screenshot below is the section where they record relationships between the adult females (at the top) and adult males that may be guarding – or pestering them (below). At the same time, they need to be able to record events that may be occurring with the pack as a whole – in this case an interaction with another pack of mongeese.

Screenshot_2014-06-26-11-34-40

Bumper Crop

Bumper crop is an android game I’ve just started working on with Dr Misha Myers as part of the Play to Grow project: “exploring and testing the use of computer games as a method of storytelling and learning to engage urban users in complexities of rural development, agricultural practices and issues facing farmers in India.”

Screenshot_2014-05-31-12-13-04

(Warning – contains machine translated Hindi!)

I’m currently working out the details with artist Saswat Mahapatra and Misha, who have been part of the team developing this game based on fieldwork in India working with farmers from different regions. They began by developing a board game, which allowed them to flexibly prototype ideas with lots of people without needing to worry about software related matters. This resulted in a great finished product, super art direction and loads of assets ready to use. I very much like this approach to games design.

From my perspective the project relates very closely to groworld games, germination x, as well as the more recent farm crap app. I’m attempting to capture the essence of the board game and restrict the necessary simplifications to a minimum. The main challenge now that the basics are working is providing an approximation of bartering and resource management between players that board games are so good at, into a simple interface – also with the provision of AI players.

Source code & Play store (very alpha at the moment!)

Procedural landscape demo on OUYA/Android

A glitchy procedural, infinite-ish landscape demo running on Android and OUYA. Use the left joystick to move around on OUYA, or swiping on Android devices with touchscreens. Here’s the apk, and the source is here.

Screenshot_2014-01-06-07-18-45

It’s great to be able to have a single binary that works across all these devices – from OUYA’s TV screen sizes to phones, and using the standard gesture interface at the same time as the OUYA controller.

The graphics are programmed in Jellyfish Lisp, using Perlin noise to create the landscape. The language is probably still a bit too close to the underlying bytecode in places, but the function calling is working and it’s getting easier to write and experiment with the code.

(define terrain
  '(let ((vertex positions-start)
         (flingdamp (vector 0 0 0))
         (world (vector 0 0 0)))

     ;; recycle a triangle which is off the screen
     (define recycle 
       (lambda (dir)         
         ;; shift along x and y coordinates:
         ;; set z to zero for each vertex
         (write! vertex       
                 (+ (*v (read vertex) 
                        (vector 1 1 0)) dir))
         (write! (+ vertex 1) 
                 (+ (*v (read (+ vertex 1)) 
                        (vector 1 1 0)) dir))
         (write! (+ vertex 2) 
                 (+ (*v (read (+ vertex 2)) 
                        (vector 1 1 0)) dir))

         ;; get the perlin noise values for each vertex
         (let ((a (noise (* (- (read vertex) world) 0.2)))
               (b (noise (* (- (read (+ vertex 1)) 
                               world) 0.2)))
               (c (noise (* (- (read (+ vertex 2))
                               world) 0.2))))

           ;; set the z coordinate for height
           (write! vertex 
                   (+ (read vertex) 
                      (+ (*v a (vector 0 0 8)) 
                         (vector 0 0 -4))))
           (write! (+ vertex 1) 
                   (+ (read (+ vertex 1)) 
                      (+ (*v b (vector 0 0 8)) 
                         (vector 0 0 -4))))
           (write! (+ vertex 2) 
                   (+ (read (+ vertex 2)) 
                      (+ (*v c (vector 0 0 8)) 
                         (vector 0 0 -4))))

           ;; recalculate normals
           (define n (normalise 
                      (cross (- (read vertex)
                                (read (+ vertex 2)))
                             (- (read vertex)
                                (read (+ vertex 1))))))

           ;; write to normal data
           (write! (+ vertex 512) n)
           (write! (+ vertex 513) n)
           (write! (+ vertex 514) n)

           ;; write the z height as texture coordinates
           (write! (+ vertex 1536) 
                   (*v (swizzle zzz a) (vector 0 5 0)))          
           (write! (+ vertex 1537) 
                   (*v (swizzle zzz b) (vector 0 5 0)))          
           (write! (+ vertex 1538) 
                   (*v (swizzle zzz c) (vector 0 5 0))))))

     ;; forever
     (loop 1
       ;; add inertia to the fling/gamepad joystick input
       (set! flingdamp (+ (* flingdamp 0.99)
                          (*v
                           (read reg-fling)
                           (vector 0.01 -0.01 0))))

       (define vel (* flingdamp 0.002))
       ;; update the world coordinates
       (set! world (+ world vel))

       ;; for each vertex
       (loop (< vertex positions-end)         

         ;; update the vertex position
         (write! vertex (+ (read vertex) vel))
         (write! (+ vertex 1) (+ (read (+ vertex 1)) vel))
         (write! (+ vertex 2) (+ (read (+ vertex 2)) vel))

         ;; check for out of area polygons to recycle 
         (cond
          ((> (read vertex) 5.0)
           (recycle (vector -10 0 0)))         
          ((< (read vertex) -5.0)
           (recycle (vector 10 0 0))))
         
         (cond
          ((> (swizzle yzz (read vertex)) 4.0)
           (recycle (vector 0 -8 0)))
          ((< (swizzle yzz (read vertex)) -4.0)
           (recycle (vector 0 8 0))))

         (set! vertex (+ vertex 3)))
       (set! vertex positions-start))))

This lisp program compiles to 362 vectors of bytecode at startup, and runs well even on my cheap Android tablet. The speed seems close enough to native C++ to be worth the effort, and it’s much more flexible (i.e. future livecoding/JIT compilation possibilities). The memory layout is shown below, it’s packing executable instructions and model data into the same address space and doesn’t use any memory allocation while it’s running (no garbage collection and not even any C mallocs). The memory size is configurable but the nature of the system is such that it would be possible to put executable data into unused graphics sections (eg. normals or vertex colours), if appropriate.

jvm

Open Sauces: A structured recipe notebook

It’s been busy in the android department this week. Here’s a prototype recipe notebook for open sauces. At this point it’s primarily a test of a drag/drop interface to make it possible for anyone to make the recipe graphs we explored visualising in October. I’m testing it with the recipes designed for FoAM’s recent Smoke and Vapour event.

Currently it’s structured a bit like Scheme Bricks where you ‘read it’ inside -> outside, and right to left. We need to look at ways to reverse this – it might be simply making everything float to the right, or more like a bottom up tree, where the finished dish is at the bottom rather than the top.

Screenshot_2013-12-14-07-39-46

10747546824_664a0d3318_b

Mongoose 2000

A screen shot from the Mongoose 2000 project, we now have most of the ‘pup focal’ interfaces working and syncing their data via the Raspberry Pi. This is the interface for recording a pup aggression event – including the identity of the aggressive mongoose and some information on the cause and severity. Each mongoose has a code, and we’re using sliding toggle button interfaces for quickly picking them – these can be filtered to restrict them to adults, pups, males or females where required.

pupaggr

The interface was written using “starwisp” – my system for building android applications in Scheme. The Mongoose 2000 app has lots of reusable interfaces, so it’s mostly constructed from fragments. There are no specialised database tables, so I can simply add or modify the widgets here and the data automagically appears in the Raspberry Pi export, which makes it very fast to build. I’ve abstracted the mongoose button grid selectors and tristate buttons (yes/no/maybe) as they are used in a lot of places. Here is the entire definition of the fragment for the interface above, the code includes everything for creating and recording the database entity for this event and all the android callbacks it needs to respond to external events.

(fragment
   "ev-pupaggr"
   
   ;; define the interface layout first
   (linear-layout
    (make-id "") 'vertical fillwrap pf-col
    (list
     (mtitle "title" "Event: Pup aggression")
     (build-grid-selector "pf-pupaggr-partner" 
                          "single" "Aggressive mongoose")
     (linear-layout
      (make-id "") 'horizontal 
      (layout 'fill-parent 100 '1 'left 0) trans-col
      (list
       (vert
        (mtext "" "Fighting over")
        (spinner (make-id "pf-pupaggr-over") 
                 (list "Food" "Escort" "Nothing" "Other") fillwrap
                 (lambda (v)
                   (entity-add-value! "over" "varchar" v) '())))
       (vert
        (mtext "" "Level")
        (spinner (make-id "pf-pupaggr-level") 
                 (list "Block" "Snap" "Chase" "Push" "Fight") fillwrap
                 (lambda (v)
                   (entity-add-value! "level" "varchar" v) '())))
       (tri-state "pf-pupaggr-in" "Initiate?" "initiate")
       (tri-state "pf-pupaggr-win" "Win?" "win")))
     (spacer 20)
     (horiz
      (mbutton "pf-pupaggr-done" "Done"
        (lambda ()
          (entity-add-value! "parent" "varchar" 
            (get-current 'pup-focal-id ""))
            (entity-record-values db "stream" "pup-focal-pupaggr")
            (list (replace-fragment (get-id "event-holder") 
                                    "events"))))
      (mbutton "pf-pupaggr-cancel" "Cancel"
        (lambda ()
          (list (replace-fragment (get-id "event-holder") 
                                  "events")))))))

   ;; define the fragment's event callbacks 
   (lambda (fragment arg) ;; on create, return layout for building
     (activity-layout fragment))

   ;; on start - update contents from the db
   (lambda (fragment arg)  
     (entity-reset!)
     (list
      (populate-grid-selector
       "pf-pupaggr-partner" "single" ;; select single mongoose
       (db-mongooses-by-pack) #t     ;; from the whole pack 
       (lambda (individual)          ;; <- called when selected
         (entity-add-value! "id-with" "varchar" 
           (ktv-get individual "unique_id")) 
         (list)))
      ))

   (lambda (fragment) '()) ;; on stop
   (lambda (fragment) '()) ;; on resume
   (lambda (fragment) '()) ;; on pause
   (lambda (fragment) '())) ;; on destroy

Ouya development experiments

The Ouya is a tiny game console which is designed for promoting indy games rather than traditional high budget productions. It’s cheap compared to standard games hardware, and all the games are free to play at least in demo form. It’s very easy to start making games with as it’s based on Android – you can just plug in a USB cable and treat it just the same as any other Android device. You also don’t need to sign anything to start building stuff – it’s just a case of adding one line to your AndroidManifest.xml to tell the Ouya that the program is a game:

 <category android:name="tv.ouya.intent.category.GAME"/>

and adding a 732×412 icon in “res/drawable-xhdpi/ouya_icon.png” so it shows up on the menu.

130327_ouya_0021

There is a lot to like about the Ouya’s philosophy, so I tried out some graphics experiments with it to get better acquainted:

This program was made using Jellyfish, part of my increasingly odd rendering stack which started out as a port of Fluxus to PS2. It’s a type of microcode written inside TinyScheme and running in a separate virtual machine that makes it possible to process a lot of geometry without garbage collection overheads. At some point I might write a compiler for this, but writing the code longhand at the moment means I can tweak the instruction set and get a better understanding of how to use it. Here is the microcode for the ribbons above, they run at 20,000 cycles per frame each (so about 1.2MHz):

;; register section
   8 20000 0 ;; control (pc, cycles, stack)
   mdl-size prim-tristrip 1 ;; graphics (size, primtype, renderhints)
   0 0 0 ;; pos
   0 0 0 ;; sensor addr

   ;; program data section
   mdl-start 0 0     ;; 4 address of current vertex
   mdl-start 0 0     ;; 5 address of accum vertex (loop)
   0 0 0             ;; 6 influence
   0 0 0             ;; temp

   ;; code section 
   ;; add up differences with every other vertex
   ldi  4  0         ;; load current vertex
   ldi  5  0         ;; load accum vertex
   sub  0  0         ;; get the difference
   lda  6  0         ;; load the accumulation
   add  0  0         ;; accumulate
   nrm  0  0         ;; normalise
   sta  6  0         ;; store accumulation
   add.x 5 1         ;; inc accum address

   ;; accumulation iteration 
   lda  5  0         ;; load accum address
   ldl  mdl-end 0    ;; load end address
   jlt  2  0         ;; exit if greater than model end (relative address)
   jmp  8  0         ;; return to accum-loop

   ;; end accum loop
   ;; push away from other verts
   lda  6  0         ;; load accum
   ldl -0.1 0        ;; reverse & make smaller
   mul 0 0

   ;; attract to next vertex
   ldi 4 0           ;; load current
   ldi 4 1           ;; load next
   sub 0 0           ;; get the difference
   ldl 0.4 0
   mul 0 0           ;; make smaller
   add 0 0           ;; add to accum result

   ;; do the animation
   ldi 4 0           ;; load current vertex
   add 0 0           ;; add the accumulation
   sti 4 0           ;; write to model data

   ;; reset the outer loop
   ldl 0 0           ;; load zero
   sta 6 0           ;; reset accum
   ldl mdl-start 0   
   sta 5 0           ;; reset accum address

   add.x 4 1         ;; inc vertex address
   lda 4  0          ;; load vertex address
   ldl mdl-end 0     ;; load end address
   jlt 2  0          ;; if greater than model (relative address)
   jmp 8  0          ;; do next vertex

   ;; end: reset current vertex back to start
   ldl mdl-start 0
   sta 4 0

   ;; waggle around the last point a bit
   lda mdl-end 0     ;; load vertex pos
   rnd 0 0           ;; load random vert
   ldl 0.5 0         
   mul 0 0           ;; make it smaller
   add 0 0           ;; add to vertex pos
   sta mdl-end 0     ;; write to model

   jmp 8 0

Mongoose 2000

Mongoose 2000 is a system I’m developing for the Banded Mongoose Research Project. It’s a behavioural recording system for use in remote areas with sporadic internet or power. The project field site is located in Uganda in the countryside and it needs to run for long time frames, so there are big challenges when it comes to setting up the system and debugging it remotely.

In order to make this work we’re using a Raspberry Pi as a low power central wifi node, allowing Android tablets to communicate with each other and synchronise data. There are a couple of types of observations we need to record:

  1. Pack composition: including presence in the pack, individual weights and pregnancy state.
  2. Pup focal: studies of individual pups, who’s feeding them, when they feed themselves or playing.
  3. Group events: warning calls, moving locations, fights with other packs.

We also need to store and manage the pack information, so names, collar and chip ids of individual animals. The data is passed around a bit like this:

web

The interface design on the tablets is very important – things may happen quickly, often at the same time (for instance group events happening while a pup focal observation is being carried out), so we need multiple simultaneous things on screen, and the priority has to be on responsiveness and speed rather than initial ease of use. For these reasons it has similarities to live music performance interfaces. We can also take advantage of the storage on the tablets to duplicate data on the Raspberry Pi to add redundancy. Data is transferred from the field site by downloading the entire database onto the Android tablets, which can then be emailed using the normal internet, either when it’s working locally or by taking the tablets into the nearby town where bandwidth is better.

IMG_20131105_185918

The project is a mix of cheap, replaceable hardware and mature well used software – Raspberry Pi’s mean we can afford a backup or two on site, along with plenty of replacement sdcards with the OS cloned. The observation software can also be updated over the Android play store (for bug fixes, or changing the data gathered) without any changes required on the Raspberry Pi. The platform is based on the one I built for the ‘Crap App’ along with experimental stuff I was doing with bike mounted wifi nodes with Kaffe Matthews, and includes SQLite for the underlying database on both platforms (providing atomic writes and journalling) and TinyScheme for Android and Racket for the Raspberry Pi allowing me to share a lot of the code between the hardware platforms.

Building software for farmers

During the summer I’ve been working with the Swarm Knowledge Hub at Cornwall’s Duchy College. We’ve been building an android application that forms part of a scheme to highlight the value of organic fertilisers compared to costly and unsustainable synthetic fertilisers.

The core of the software is a calculator based on tables provided by DEFRA (the UK government’s Department for Environment Food & Rural Affairs) which provides the quantities of nitrogen, phosphorus and potassium nutrients provided by different types of manure spread on different types of soils.

1 2 3

Farmers are required to supply records of the fertiliser they use to DEFRA, so the program also allows you to input your fields and record the levels of nutrients spread on each one separately. You can then export the data as a csv file over email.

We are planning a workshop with local farmers in the coming weeks, which I’m really interested to be part of. To me this is an extension of the groworld project providing a connection to an additional, important group – the people who actually grow the food we eat.

The majority of the code was written in Scheme which meant a lot of it could be rapidly prototyped (I’ll be blogging more about this soon) and the source can be found on github here.