General – $20.00
Industry – $15.00
Student/Educator – $10.00
Donna Hewitt – Macrophonics
In late 2012 the artist collective Macrophonics visited internationally-recognised physical theatre company, Legs On The Wall, to experiment with sensor-based technology and how it can be adapted specifically for live performances. What emerged was a tool for creating live art, one that uses both the body of a performer and the space they move through. Now Macrophonics and Legs On The Wall offer audiences a glimpse into what comes next – the creation of a new work that transforms the dancer into the conductor, the musician into the actor, and the stage into the interface.
This presentation will feature Professor Julian Knowles of Macquarie University’s Department of Media, Music, Communication and Cultural Studies and Patrick Nolan, Artistic Director of Legs On The Wall, and a demonstration of the Macrophonic’s technology with performer Tim Ohl.
THE MAKING OF…
In a series of fast-paced, one-hour tag-team sessions, creative teams from the worlds of television, gaming, music, and animation take us behind-the-scenes to introduce us to the people and processes that make huge creative projects a reality.
These sessions explore strategies for making international real-time collaborations work, find out what tools are essential for co-ordinating multi-part projects, and meet some of the world’s most interesting companies and creatives.
The accelerometer and wearable work progressed smoothly, without significant hitches. Donna’s wearable was completed in the final week. A few issues are worth mentioning here. She found the use of conductive thread and ribbon to be somewhat problematic. Great care needed to be exercised to prevent thread touching and shorting out (which reset the boards) and the high resistance also posed some issues. This caused Donna to make a decision to replace most of the conductive thread with conventional insulated hookup wire. Whilst initially concerned about appearance, the hookup wire actually looked great. Take a look.
Donna – showing off finished wearable with insulated wire
A second issue was the total power draw of the sensors and Xbee wireless system. Donna found that the single AA battery could not supply enough power to run the system properly. With insufficient time to source and fit a higher capacity battery system before the public showing of our work, she decided to power the system from a USB cable and make the necessary modifications to the battery system following the residency.
To everyone’s surprise, when we connected the wearable to our test audio and video patches for the first time, the results were beyond our expectations. The wearable was highly responsive, ‘playable’ and Donna reported that she felt a fine sense of control over the media. She felt immersed in the media and the interface was highly intuitive, providing a rich set of possibilities for gestural control. Here is a video showing the initial hookup of the wearable interface to a test audio patch with the triaxial accelerometer and flex sensor mapped to audio filter parameters.
In the final week we experimented with a range of sensing techniques that picked up the position of performers on a stage. We experimented with the cv.jit and with Cyclops systems within MaxMSP. Both proved to be excellent. However given that they are video tracking systems they are inherently light dependent and so their output and behaviours are fundamentally affected by changing light states and conditions.
Cyclops patch detailCyclops grid and hotspot ‘zones’
All of this is manageable with precise light control and programming (or the use of infra red cameras), however in the absence of more sophisticated lighting and camera resources, we decided to use ultrasonic range finders on the stage to locate the position of performers.
Ultrasonic rangefinder
Two of these devices were placed on either side of the stage, outputting a stream of continuous controller data on the basis of the performer(s) proximity to the devices. These sensors were connected to Tim’s computer via an Arduino Uno. This allowed us to have a simple proximity sensing system on stage. The system was robust in respect of light, but suffered from occasional random noise/jitter that would last for a few seconds without obvious cause. This meant we had to apply heavy signal conditioning to the source data in Max/MSP to smooth out these ‘errors’ which in turn resulted in a significant amount of latency. Given these constraints, we used these sensors to drive events which did not have critical timing dependencies and could ramp in and out more gradually.
I spent a number of days in our final week programming the relationships between the sensing systems and the video elements, working with ‘retro’ 70s scanline/raster style video synth processing and time domain manipulations of quicktime movies. My computer also operated as a kind of ‘data central’, with all incoming sensor data coming in to my computer and from there being mapped/directed out to the other computers from a central patch in MaxMSP+Jitter.
Detail of Jitter patch for Open Source residency
I’ll post some documentation of the final results once we have an edit.
Julian’s computer. Wiimote to Osculator to MaxMSP to Ableton Live
While Donna Hewitt works on building the wearable top with sensors, I am working on making ‘musical scenes’. These scenes are modular, with a range of musical elements which can be triggered or manipulated in an improvisational manner. They are designed for non-musician physical theatre performers/acrobats. The wearable top will have a Triaxial Accelerometer (reading X, Y, Z axes) mounted on the right arm, a flex sensor on the left elbow and some buttons. I have been using a Wii remote to simulate the accelerometer and buttons so I can work on parameter mapping and prototype some a/v scenes while Donna is working on building the wearable interface. The Wii remote can act as a hand held device which contains much of the functionality of the wearable (accelerometer and buttons)
Nintendo Wii Remote (Wiimote) puts out pitch, roll, yaw and z (g force) data
I’m using the fantastic software Osculator to take the bluetooth Wiimote data and convert it to midi. I then stream the midi into a patch in MaxMSP which allows me to condition, scale and route the data streams before they get sent to Ableton Live to control audio.
The system is very robust and today I had it working to a distance of over 15 metres. Let’s hope the wifi from the Lilypad is as solid. Here is an example of the wii remote being used to control audio. The accelerometer XYZ outputs control various audio filter processing parameters and volume changes, while the buttons are used to trigger audio events.
On the video side, I’ve been building a realtime video processing system in the Jitter environment.
Julian Knowles: Working on video processing system in Jitter. Photo: Catherine McElhone
The following example shows the wii remote accelerometer XYZ parameters mapped to video processing. No audio processing is taking place in this example. I’m just listening to some music while i program and test the video processing system. In this example the wii remote is driving a patch I have written in Jitter. A quicktime movie is used as input and the wiimote is driving real time processing. I have also extended the jitter patch to allow me to take live camera input.
Donna has been working away on the wearable interface that will contain the functionality of the above (plus more). She has been designing the layout of the sensors and working out how to connect everything within the given constraints of the LilyPad system. The conductive thread that can be used to sew the sensors in and connect to the main board has quite a high resistance, and so runs need to be kept short. Likewise the run between the battery board and the Lilypad/Xbee board needs to be short, so as to keep maximum current available. Runs of conductive thread cannot be crossed over or they will short out.
Donna – roughing out the sensors on a black sports top
We’re hoping to get the wearable interface completed in the next day or two so we can start to test it out with the modular musical materials. For the purposes of the showing, we’ll demonstrate three a/v ‘scenes’ in sequence, demonstrating different approaches and relationships between gesture and media.
The first scene will be drone/video synthesis based (with the performers stage positions driving processing). This state will have a very strong correlation between the audio and video processing gestures and will allow for multiple performers moving in relation to ultrasonic range finder sensors.
The second scene will involve the wearable interface, with direct/detailed gestural control of audio and video elements from a solo performer.
The third involve a complex interplay of sensors and parameters. The wearable interface will perform time domain manipulation and transport control on quicktime materials whilst driving filters and processors in the audio domain. The scene will also make use of physical objects sounding on stage, driven by the wearable interface. The interface data will be used to control signals flowing through physical objects (in this case cymbals and a snare drum) and audio/spatial relationships will unfold between the performer’s gestures, proximity to objects and sound behaviours. Tim Bruniges is taking care of the actuator setup.
Wearable interface. Pressure sensors on the shoulder blades.
Ultimately we are aiming for a fully responsive media environment. The aim is for the stage performer(s) to drive both audio and video elements and feel immersed in a highly responsive mediascape.
Wearable interface. Pressure sensors on shoulder blades
We’re now well into our Macrophonics Open Source creative development project at Legs on the Wall in Sydney. Lots of experimentation going on at this stage and we’ll start to resolve things down during the week before our stint in the theatre next week. We’ve opened out the exploration to sounding objects – driving audio signals into snare drums, cymbals and other resonant objects to set them off acoustically. The idea is that certain parts of the stage area contain assemblages of resonant objects and, through video and hardware sensing, performers will be able to activate the array of objects within an auditory ‘scene’ that we create. These auditory scenes will contain flexible sonic ‘modules’ – collections of sounds and musical motifs that can be recombined freely.
15″ loudspeaker driver coupled to a 15″ snare
At the moment, we’re experimenting to find the resonant frequency of each of the trial objects. We also want to get a sense of how the objects behave with different input signals. At the moment, we’ve set up the input to the ‘actuators’ to run from discrete sends so that they can be blended/balanced with the signals from the main loudspeakers. From a control perspective, we will compare wearable sensors (using the Lilypad Arduino platform) located on the performer, with video tracking from above the stage. The wearable sensors have the advantage of being robust in respect of different lighting states, but need to be protected from damage by the physical theatre performer. This could prove challenging. The video tracking approach is robust from a physical point of view, but highly light dependant – so changing light conditions can affect the threshold settings for the tracking patches so that tracking becomes less reliable through changing light states.
Donna working with Lilypad Arduino. Seen here with small accelerometer sensor
The plan is to use the accelerometers on the wrists of the performers so that arm movements and rotations will output X,Y,Z co-ordinates. We’ll also be trying out light sensors on the performers, flex sensors and some heat sensors. The Lilypad is a great platform for wearable computing – the main issue is the small number of analog inputs, so we will need to investigate the potential to have multiple Lilypads sending data over wifi (via the Xbee platform) to a single computer. If this poses an issue for us, we could mount a second Lilypad/Xbee set up and have it transmitting to a second receiver at a second computer. As we have not yet established an approach to data mapping and distribution to the media performers, this second model may actually be more ergonomic. We shall see.
Lilypad Arduino with Lilypad Xbee (top left) at Donna’s workstation
In the shot above you can see the Lilypad Arduino connected to a Lilypad Xbee board which takes care of the wireless communication of sensor data to an Xbee receiver at the computer. The Lilypad Xbee is currently getting power from a USB cable, but will soon be powered by an AAA battery. Once all the sensors have been connected up and prototyped in this way, the whole thing will be sewn into a garment and the sensors will be connected to the board via conductive thread.
Julian Knowles and Tim Bruniges (positioning sonic actuators). Photo: Lucy Parakhina
The residency sees the group examine a range of interface technologies and approaches to explore the nexus between theatrical performers and a four member live media ensemble. The objective is to build a responsive performance environment that will allow the theatre performers to have direct gestural input into the media control system, defining new relationships between the performance ensemble, the media design elements and the media ensemble.
We’ve broadly structured our investigation across two approaches. The first is video tracking/computer vision techniques (using the cv.jit suite of objects for Jitter) where the stage area can be analysed and moving objects tracked.
Jitter patch – cv.jit suite of ‘computer vision’ (video analysis) objectsMoving masses in the video frame are identified and then tracked
The second area of investigation is wearable garment interfaces that sense movement directly at each performer (accelerometers, pressure and flex sensors, light sensors etc) and send data wirelessly to the media ensemble. We are using the LilyPad Arduino platform for this work. The LilyPad is a small, flat implementation of the Arduino platform that can be sewn into garments. Conductive ‘thread’ can then be sewn in ‘tracks’ into the garment to form the links between the device and the attached sensors.
LilyPad Arduino
The LilyPad can be run from a battery source and can, with a bit of extra stuff, transmit data wirelessly over an ad hoc wireless network.
Lilypad arduino sewn into garment with conductive thread. Image credit: Rain Rabbit (flickr)Donna’s LilyPad – ready for sewing
Even As We Speak, John Peel’s House (‘Peel Mansions’), Stowmarket, UK 1993
Here is a recent interview and live to air performance from the studio of FBi Radio in Sydney on November 3, 2012 – immediately before our 20 year reunion gig at The Red Rattler. We speak about the experience of basing the band in the UK in the early 1990s, the independent scene at that time and the experience of working with John Peel. The show previews two previously unreleased tracks ‘Falling Down the Stairs (ft. Younger Youth)’ and ‘Best Kept Secret’ recorded for the BBC in 1993, and soon to be released on the forthcoming Even As We Speak Peel/Goodier sessions release ‘Yellow Food’. We also play a stripped back in-studio live version of a further unreleased Peel Session track ‘Everywhere I Go’.
Even As We Speak. BBC Maida Vale UK. Peel Session 1993
Even As We Speak at BBC Maida Vale UK. Peel Session 1993
Even As We Speak at BBC Maida Vale. Peel Session 1993
Even As We Speak at BBC Maida Vale. Peel Session 1993. Tympani overdubs ‘Best Kept Secret’
BBC Maida Vale Studio 4. Home of Peel Sessions. Main live room 2011
To celebrate the imminent release of our John Peel and Mark Goodier BBC Sessions, Even As We Speak played a special one-off show at The Red Rattler Theatre in Sydney Australia on November 3.
Even As We Speak – Live at The Red Rattler, Sydney Australia. Nov 3, 2012
This marked the first time the band had played together since 1993 when we returned to the Australia from the UK, following the release of our final ‘Blue Eyes Deceiving Me‘ EP on the legendary Bristol/UK label Sarah Records.
http://www.youtube.com/watch?v=of-KfBFsUiI
Joined by supports Alannah Russack (ex The Hummingbirds), and Living with Robert, the night brought out a bunch of die hard fans, with people even taking international flights to get to the show. Now that’s serious dedication…
Mary Wyer, Matt Love and I did an interview for FBi Radio the afternoon before the show. We talked about the experience of recording sessions for John Peel and previewed two of the tracks from the impending release. We also did a seat-of-the-pants live to air from the radio studio of the otherwise unreleased Peel Session song ‘Everywhere I Go’. Stay tuned for updates on this brand new release, which should be announced in the next fortnight or so, as soon as contracts are finalised with the BBC.
Even as We Speak. Live at the Red Rattler. Sydney Australia. Nov 3, 2012.
Join us at The Waiting Room in Brisbane’s West End for this diverse show featuring artists from Sydney, Brisbane, and the Gold Coast. Don’t miss how four men use four different approaches to create a million unusual sounds – all in one Thursday night.
HINTERLANDT is a Sydney-based solo act juggling numerous instruments, sounds and influences. Hinterlandt has toured internationally and released truckloads of material. In his first-ever Brisbane show, Hinterlandt mastermind Jochen Gutsch presents his brand new album “Cartography” on Laughing Outlaw Records.
ANONYMEYE is the nom de musique of Andrew Tuttle. Anonymeye reconfigures various organic and mechanic musics within a sonic framework akin to an abstract musical Esperanto. Utilising electronic and acoustic instrumentation including acoustic guitar, computer, banjo, and synthesiser, Anonymeye straddles and blurs boundaries between improvisation and composition, experimentation and song-form, and melody and dissonance.
JULIAN KNOWLES has been creating pretty, ambient electronic noise for 20 years as a solo artist and with the iconic field recording group, Social Interiors. His work crosses every musical boundary possible – experimental, post-rock, ambient, field – and is fuelled by destroyed analog media, guitars and laptops.
PUZAHKI is the alias of Gold Coast based electronic music producer Craig Parry. Puzahki’s preceding reputation lies in breakcore & jungle however his current musical focus is instrumental hiphop and abstract turntablism. He has just released a 7 track EP titled Emergency Exit, has a full length mix album coming out on Ender Records and is about to self release a 3 track split single with Chupi.
Just presented the first performance of The Flashcard Sequences Part 1: Amsterdam 14.12.10 last night. I’ve uploaded a copy to Vimeo. Switch to High Definition and view in full screen for the full experience.
Since mobile phone video has been improving at such an alarming rate, I’ve renewed my interest in field/environmental a/v work, which was a strong focus for me through the 1990s, both in my solo work and with the group Social Interiors. I’ve found myself capturing moments or locations from the every day or my travels and have started making a series of short works based on these moments in time and place. The focus is on capturing the moment or a perspective with very modest means – nothing but a tiny hand held iphone. There are challenges, such as image stability due to lack of tripod and, as will be seen in the first work, clumsy/cold hands in the snow. I have tried to work with these instabilities as an aesthetic element and make them feel part of the work.
The Flashcard Sequences has thus made its way into the world.
The Flashcard Sequences comprises a series of short semi-improvisatory works produced with handheld mobile phone video and portable laptop production technologies. The production process is intentionally swift and the works are produced (often in hotel rooms) as quickly as possible following the image capture. These pieces function as snapshots which attempt to capture an underlying essence of a moment or place.
Amsterdam 14.12.10 is the first piece in the series, and was created during a residency at STEIM, Holland in December 2010. The harsh European winter of 2010/11 gave rise to heavy snowfalls, closing roads and bringing many essential transport services to a halt. The visual material consists of a series of static video shots containing patterns of snow falling from the sky. In this piece, the video source is digitally analysed and used to affect processing parameters in the audio domain. Such an approach attempts to find, and compositionally engage with, the direct structural relationships between the visual and sound materials whilst stopping short of direct and literal sonification. Such an approach aims to take transductive visual/sound mapping processes and bring them into creative dialogue with freely interpretive compositional processes.
Just spent the afternoon with Dom Stephens (ex Oh Ye Denver Birds and Outerwaves) mastering his new Nimble Animal album. It will be released digitally and on cassette through the Fallopian Tunes cassette label http://fallopiantunes.bandcamp.com/ – one of a number of new artist run cassette labels which seem to be popping up everywhere. Although mastering is kind of like surgery, where you often don’t think too much about the patient on the table, this really is a great album of lo-fi experimental pop. The sound textures are very abstract at times, but there are always strong pop hooks present in each of the songs. It’s great to hear a level of adventurousness and musical intelligence emerging from a sea of undifferentiated indie safety. Earlier in the year, I showed Dom a little of the secret ‘destroyed tape loop‘ stuff I have been working with and some of the output of those studio sessions is audible in the final result. Stay tuned for immanent release. Highly recommended.
