T/Act - social empowerment through interaction with media artworks

This paper presents results from research made through a collaborative design process with selected individuals with severe physical disabilities. The work encourages and enables creative expression by the participants beyond everyday norms. Can a disruption of institutionalized conditioning according to class, education, gender and physical abilities be orchestrated by careful design and presentation of interactive artworks?


Our current lifestyle is focused and reliant upon media technologies. Our lives are organised through and by technology, such that we can easily forget the importance of physical social interaction rather than which are mediated by online social networks. Instead of being empowered by technology, humans are enslaved to its seductive powers. Is it possible to move away from this focus on the technological and rather discuss the act of using the interface and the product of that action and the content? Does access to media technology in itself empower the participant, particularly if that person is herself on the margins of society? The Eye Writer project is a superb example of open source media technologies being used to empower a specific individual (Tempt One) and others with a similar disabilitating disease (ALS). [1] As Tempt One himself states:

“Art is a tool of empowerment and social change, and I consider myself blessed to be able to create and use my work to promote health reform, bring awareness about ALS and help others.”

It is clear that the act of empowerment for Tempt One comes through a combination of access to the technology, the ability to once again create graffiti art, and his possibility to have a presence in the public city environment through the large scale urban projections of his tags. Each element is very specific to the individual in question. In the research described in this paper the author attempts a broader area of investigation. Can the use of media technologies enhance the possibilities for people with disabilities to express themselves creatively on equal terms with able bodied people?
This paper presents ongoing research into the effects of physical interaction with audiovisual systems through a discussion of the results and observations from collaborative design workshops organised for a group of people with disabilities. The author, as a media artist, had not considered working with people with disabilities until a visit by a group of students from Beaumont special school to the Lanternhouse International arts centre in the north of England where he was undertaking a residency. As these students with severe cerebral palsy were encouraged to touch and interact with the installation which was on display for them, it became apparent that the colour, form, sound and overall interactive environment they were confronted with provided a powerful and provocative stimulus, causing emotional reactions which surprised their carers. A follow-up visit to the college showed that although well equipped with musical instruments, media and audio software, most solutions were generalized rather than individually tailored to each student’s needs. This approach may work for the able-bodied person where we all have approximately the same physical abilities, but for a person with disabilities this can be totally inappropriate and very frustrating for all involved. Together with musician Alan Fitzgerald the author proposed to develop bespoke electronic interfaces for a small group of students. In particular it was hoped to examine the following question: If a unique interface is created specifically for a particular individual, can an examination of the use of this interface lead us to answer questions regarding interface design in general? Unfortunately at the time it was not possible to carry out this project in England, but since the beginning of 2011 the author has been investigating similar themes through participatory design workshops with people with disabilities belonging to the Taika Dance group in Turku, Finland. The majority of the participants are electric wheelchair users and have severely limited use and control of their physical bodies, while some have more mobility. They have their own social networks, yet as a whole they can be regarded as on the margins of society with little voice or visibility. Does access to media technology and the ability to create visual and audio performance lead to a wider social empowerment in society for people like these with disabilities? Does the same effect happen for the wider public at large when they are able to interact deeply with a media art work?

Through a participatory design process, the aim of the workshop sessions has been to develop personal interfaces which might be thought of as bespoke electronic musical instruments made for each individual. Due to the practical difficulties involved with all aspects of the collaboration – logistics, communication, and basic bodily needs – progress has been slow, but fruitful. As this group of people have had no prior possibility to make sound or music, the process started with getting to know each other via “off the shelf” solutions. A midi keyboard and controller were used to provide an immediate experience of actually creating different sounds. Using Max/MSP and Reason software, samples and sound parameters could easily be modified. Sounds were also recorded from the participants own voices and mobile phones to use as samples. Even at this simple level, the experience of hearing one’s own voice played back and modified to create interesting or weird sounds was stimulating for the group. Participants soon felt confident to contribute their own ideas and suggestions for the sounds.

The next level of interaction involved gradually introducing different types of electronic sensors and interfaces, allowing the participants to experiment and play with sound in ways that were totally new for them. The author is familiar with using analogue sensors for data collection, interfacing through the Arduino microcontroller to PCs. Now it was necessary to develop methods of using the electronics so that they would not restrict the users’ limited physical movements. Fortunately there are many small footprint solutions readily available on the market. The selected solution was to use short range radios to send the data to remote PCs. The X-Bee radio together with an Arduino Fio has so far proven to be the best solution, as radios can be networked to send data simultaneously to one PC. The type of sensors used range from simple flex and pressure sensors, accelerometers, and compass modules, to perhaps the most useful, the 9 DOF Razor IMU which provides angle of orientation data in all directions. [2] The emphasis on hardware development had been on the novel use of existing electronic components and not the actual development of new technology per se, although this does include the creation of custom sensors and switches using soft circuitry for example. The exploitation of small wireless devices means that the usual restrictions caused by signal wires are removed, and any impediments to the physical body are minimized. The approach used is to concentrate on the movements that the participants are able to make, rather than design an interface that they would have to adapt to.

The focus is on ABILITY rather than DIS-ability. They play according to their own abilities, and can focus on developing that skill. The aim is to discover appropriate forms of interface and sound according to each person’s physical abilities and musical interest. The dynamics of social interaction between the members of the group is also mediated by the technology. It can be observed that there is an eagerness to be the one performing. At the current stage of the project only one or two people have been able to use the interfaces simultaneously. Now that the physical abilities of each of the members have been understood, appropriate personalised interfaces are under development.

As much as possible the motivation for the design of these interfaces comes from the participants themselves as they experiment with the prototypes. One example is a control interface made as a cushion for a wheelchair user – she can control media and play sounds by shifting her weight on the chair. Made with Arduino and Open Frameworks, the interface is very sensitive, intuitive and fun to use. It can be thought of as a dance mat for wheelchair users, yet it is equally useable by the able-bodied. This is at the core of the research: through the development of new media interfaces for a small group of very particular people, gain insight into empowerment through human interaction with audio visual systems in general. Even though the participants have sensory systems different to the regular population, the goal is to make this difference invisible through the medium of the art performance. With the Taika Dance group the aim is to perform publically at the end of 2011.

The use of computer mediated technologies opens up further possibilities for social interaction. Networked technologies, such as video, audio and telematic control of devices allow these physically challenged participants to interact with others over large distances (such as Finland-UK). There is the potential to enable people with disabilities to collaborate remotely and perform highly advanced works to a geographically dispersed public audience. The use of telematic and virtual spaces allows flexibility in developing personal navigable space for each participant – finding the comfort zone for each individual is extremely important when they may not feel comfortable exposing their physical self to a live audience, but a tele-mediated performance maybe an exciting and liberating alternative. The author can foresee other groups of users/participants such as older people making use of these same systems to create their own networked performative works, mixing the security of their personal space with the empowerment of performing to a virtual audience online.
Collaborative performance shifts interaction and participatory behaviour onto a social level. The research aims to develop a methodology for observing the changing role of creator-interactor-viewer and the effects on the social interaction of the participants. How does narrative structure and a shared sense of social space lead towards development of temporary community? In the case of the Taika Dance group, the participants are already known to each other, but through the performative act they are able to transform their own self-image and their perceived role in society. They become activators of their own destiny for that moment in time – they are no-longer abject objects on the margins of society but proud performers in their own right. These works enable investigation of enactive engagement in collaborative activity with playful, participatory artworks, environments and performances. These include accessible and easy use – easy control interfaces that give inexperienced users control over creative acts and allow them to explore artistic experience through their natural body movements and perceptually guided actions.
The dialectical method facilitates the benchmarking of the generalist approach with that of the highly defined individually focused approach. By focusing on people with special needs (brain damage, physical handicap) in this case, the research adds to the discussion of reactions to interaction stimuli and control in the average adult human. Just as the blind person’s sense of hearing is amplified, so it may be that someone with severely limited movement can actually have an acute sense of control over a range far too limited for the normal person to perceive. Work by Saranjit Birdi with special needs patients in the UK supports this proposition. [3] The bespoke device or environment designed for the individual also acts as a window into their world, as we are able to experience the physical or virtual world through their interface, their experience. In particular Merleau-Ponty’s discussion of the body schema illustrates how examination of a unique individual helps us to understand the wider landscape. [4]

As is alluded to in the title of this paper, the motivation for the research is to understand if and how social empowerment can be orchestrated through interaction with media artworks. Can a disruption or disturbance of institutionalized conditioning according to class, education, gender and physical abilities be affected by careful design and presentation of the interactive artwork? It is vital that the interactive experience invites and encourages social interaction between the participants themselves, as it is only through social activity that the self-image can be positively developed. Can the artwork create a community of presence, an opportunity for living in the moment leading to unpredictable (inter)activity within the social group? The artistic TAZ (Temporary Autonomous Zone) acts as a revealing agent within society using the tools of poetic terrorism to disrupt the status quo. [5] Hakim Bey’s concept of the Temporary Autonomous Zone has been proposed by Geert Lovink as a model for network based communities of interest. [6] Having worked extensively with 3D virtual communities in the past, the author can say that the behaviours observed in physically interactive environments can be identical to those seen in the TAZ of virtual communities. The physical artwork (environment, installation) becomes a point of focus for social interaction AND empowerment, as the normal rules of engagement within the public (museum) space are temporarily ignored in favour of those created by the participants themselves.  We are forced to reappraise the traditional models for spectator vs. artist, as new tools and technologies allow the barriers to interaction to become transparent. The role of the artist or designer changes to become that of a facilitator or producer for a larger group of participants. In fact, the artist creates the situation, and the possibilities for others to bring to life, and accordingly the role of the artist as the author becomes less significant. Curator and theorist Nicolas Bourriaud regards that we have passed into a new “altermodern” era where artistic production is concerned with the weaving of “relationships” between people and things, where the artist “viatorises” objects to build narratives through “post production” techniques – the re-use of artefacts, sampling, a mixing of cultures and signs. [7] The discourse, the social activity, becomes the work itself.

By contrasting the generic with the specific, this research has set out to uncover new information about the benefits, desire and motivation to interact with complex technologically driven systems, as well as proposals for rules and methods for the creation of artistic communities of presence. The work together with Taika Dance encourages and enables creative expression by the participants beyond their everyday norms. The eventual goal is to have an understanding of how to enable deep audience participation in live performative events and interactive environments through their interaction and control of audiovisual and robotic systems.

References and Notes: 

1. The EyeWriter Project website. Free Art and Technology (FAT), OpenFrameworks and the Graffiti Resarch Lab: Tempt1, Evan Roth, Chris Sugrue, Zach Lieberman,Theo Watson and James Powderly. http://www.eyewriter.org (accessed June 28, 2011).
2. An inertial measurement unit, or IMU, is an electronic device that measures and reports on a craft's velocity, orientation, and gravitational forces, using a combination of accelerometers and gyroscopes. http://en.wikipedia.org/wiki/Inertial_measurement_unit (accessed June 29, 2011).
3. Saranjit Birdi, Thisability (2010) Online documentation and artist statement, http://dig.axisweb.org/Artwork.aspx?WORKID=74441&VISUALID=0 (accessed July 4, 2011).
4. Maurice Merly-Ponty, “The spatiality of one’s own body and motility” in Phenomenology of Perception (Abingdon and New York: Routledge Classics 2008, 1945, eng 1962), 112-177.
5.  Hakim Bey, "The Temporary Autonomous Zone, Ontological Anarchy, Poetic Terrorism" (Autonomedia 1985, 1991) http://www.hermetic.com/bey/taz_cont.html (accessed July 4, 2011).
6. Geert Lovink, “The Data Dandy and Sovereign Media, An Introduction to the Media Theory of ADILKNO,” Lecture for the Fifth International Symposium on Electronic Art, Helsinki, 24 August 1994.
7. Nicolas Bourriaud, Relational Aesthetics (Paris: Les Presses du Reel, 1988, eng 2002); Altermodern (London: Catalogue Tate Triennial, 2009).