Mobile Interfaces

Sound Initiated Drawing and Short-Term Memory Impairment by Brenda Irene Hutchinson/ Network Landscape by Teri Rueb/ Control: Enabling Participatory Art With Mobile Devices by Charles Roberts
Monday, 19 September, 2011 - 14:45 - 15:45
Chair Person: 
Chris Hales
Brenda Hutchinson
Teri Rueb
Charles Roberts

Sound Initiated Drawing and Short-Term Memory Impairment

by Brenda Irene Hutchinson

The presentation includes a participatory workshop of an interactive interface and program for creating, storing and replaying animated drawings with sound, a description of its development and the case study for which it was created. This interface was initially developed to reestablish and maintain coherent connection with a longtime friend and artist who suffered severe memory impairment due to metastasized breast cancer. Through this effort we found a way for her to continue to work as an artist by increasing and sustaining her ability to focus and concentrate for extended periods of time. The interface includes a computer, drawing tablet and a series of programs I have written to record and assign personally significant sounds (i.e. sounds from familiar environments, daily activities, speech, music, etc.) to regions on a two-dimensional area corresponding to the surface of the drawing tablet. Once sounds are assigned, the movement of the hand is tracked as it initiates and scrubs through the sampled sounds, and the data of the movement is recorded and stored. When this data is fed back through the interface, the drawing recreates itself, unfolding both a map and legacy of the attention of the person who made the drawing.

Network Landscape

by Teri Rueb

What is at stake when bodies, technologies and landscapes become merged at the site of mobile interface?  The proliferation of wireless networks and pervasive computing over the last decade has intensified the expansion of mediated human presence and agency in both the built and natural environment, while equally transforming the influence of environments on bodies, subjects and consciousnesses.  The increasing degree to which our lives unfold in and through mobile media interfaces has led to significant transformations in the social and spatial relationships of bodies and environments at the level of the individual subject, as well as the collective.   In this post-human “post-environment” condition, social and spatial distinctions become increasingly blurred across self and other, organism and environment, space and time, and technology and the body.

Mobile media interfaces, as mediations of networks and bodies that operate across intimate and large scales, both spatially and symbolically constitute a form of landscape.  As a landscape, what do representations of where we are, as objectified in mobile media interfaces reveal about who we are and who we might strive to be as citizens of a globalized network society?  In this paper, which draws upon examples from my practice since 2007, I propose the model of “network landscape” as a non-anthropocentric and non-techno-centric approach to the design mobile media interface as social, spatial and symbolic form.

Control: Enabling Participatory Art With Mobile Devices

by Charles Roberts

The proliferation of touchscreen smartphones and tablets has created a surge of new interest in incorporating personal mobile devices into visual and sonic art. These devices provide millions of users access to an array of sophisticated sensors that artists can take advantage of in interactive installations and performances. Unfortunately, the development of software enabling the creation of custom interfaces for such works lags far behind the hardware advances of the last three years. Apple's prohibition against the majority of user scripting has stopped developers from allowing artists to define interfaces with complexities going beyond simple banks of virtual sliders and knobs; such scripting is a necessity when creating dynamic interfaces that can change their behavior based on hardware sensor readings, user input and network requests. An additional problem with current artistic interface applications is that they only permit artists to take advantage of two of the many sensors in mobile devices (the touchscreen and triple-axis accelerometer) while ignoring other valuable sensors for interactivity such as gyroscopes, microphones and video cameras.

In response to these problems the author presents his own open-source software for creating interfaces on iOS devices (it will soon be ported to Android), named Control ( Control allows artists to define interfaces in JavaScript that can easily be pushed to mobile devices for use in interactive installations and audience participation pieces. Every element of Control is scriptable (Apple permits JavaScript to run in their own web rendering engine, which Control uses) and the software provides access to the full array of sensors possessed by mobile devices.

This paper will contextualize Control by looking at the history of previous touchscreen devices like the JazzMutant Lemur, by examining other iOS and Android solutions for artistic interface development and by looking at the influence of two movements in the field of Human Computer Interaction on the project: End-User Programming and Meta-Design. The paper will end by discussing the author's personal use of Control in his artistic and musical practice and by offering propositions about the future role of mobile devices in audience interaction pieces and participatory artistic installations.