Sonifying Data, Visualizing Sound

Ludic Listening: Sound Art in Video Game Design by Aaron Oldenburg/ On breathing and geography – sonifying the Severn as shared generative art practice by Michaela Palmer/ Hybrid Art Forms: The Way of Seeing Music by Evrim Bilge Erkin/ Crossing foliage: Activation profile for audio-graphic navigation in foliage clusters by Roland Cahen and Marie-Julie Bourgeois
Wednesday, 14 September, 2011 - 14:45 - 16:05
Chair Person: 
Steve Gibson
Aaron Oldenburg
Michaela Palmer
Bilge Evrim Erkin
Roland Cahen
Marie-Julie Bourgeois

Ludic Listening: Sound Art in Video Game Design

by Aaron Oldenburg

Digital games have historically been a highly visual medium.  Interactive virtual game worlds regularly immerse players in foreign, uncharted territories.  However, they have so far provided little opportunity for players to explore the world of hearing.

This paper will discuss the history of sonic experimentation in digital game design, including games with blindness as a mechanic, such as Increpare’s Forest and Eddo Stern’s Darkgame, as well as a series of games the author is in the process of creating that directly address ideas in sound art theory.  In his book, Sinister Resonances, David Toop describes hearing as “allow[ing] us constant access to a less stable world, omnidirectional, always in a state of becoming and receding, known and unknown.  This is the world that surrounds us and flows through us, in all its uncertainty.”   These projects focus on the slippery world of audio with the intention of distancing video games from the rational and concrete.  Video games in turn facilitate the exploration of relatively new paths in sound art, in areas such as spatial music composition.

Prior gameplay research has focused on eavesdropping and vicinal noises as a game mechanic.  In the game Escape the Cage, gameplay actively involves the listening player as accidental co-composer with the performing player.  Invisible Landscaping requires players to distinguish sound in noise.  The paper will discuss game designs in progress that base their mechanics on sonic strategies inspired by, for example, Dziga Vertov’s sense of implied sound.  Synesthesia will be explored through audio that attempts to represent other senses.  Game environments will become audio recorders, exploring William S. Burroughs’ engramatic inscribing of sounds in bodies.

Artistic video games are in a sense indebted to sound art, as works like John Cage’s 4’33” helped us see the audience as collaborators and composers.  Therefore, it may be beneficial to return to sound art as one method of expanding the potential of video games.  The goal is to look beyond visual-centric game design, which will give game designers conceptual tools to bypass its expressive limitations.

On breathing and geography – sonifying the Severn as shared generative art practice

by Michaela Palmer

Dr Michaela Palmer and Dr Owain Jones are Tidal Severn, an interdisciplinary research team that provides information, teaching materials and presentations in order to raise public awareness about the Severn Estuary, a large and important intertidal landscape in Southwest England.

The Severn Estuary deserves great attention: with a huge tidal range and three million people living around its shores, tidal processes and people are intrinsically and uniquely interlinked. While the Severn Estuary only concerns the livelihoods of some families directly, the passage over or under the estuary, tourism and recreational activities indirectly affect a much wider, and not necessarily local, population group.

This is the target audience for Sonic Severn, a small but growing online collection of soundscapes, sonifications and compositions about the Severn Estuary (, curated by Tidal Severn.

Sonic Severn is an open invitation to local composers, sound designers and sound artists to sonify some of the phenomena of the Severn Estuary. Many of the Sonic Severn artefacts have combined information technologies with audio technologies in order to communicate some of the tidal phenomena in the form of sound, and by this means to share what is in living memory and to open listeners’ minds to the fragile relationships between human experiences and local landscape.

Sonification of data is a key technique hereby. Although strongly linked to scientific data acquisition, it was here often applied as an artistic method: data ‘farmed’ from the Severn Estuary was used to further generative sound practice. In order to make the generative forces that affect an intertidal landscape clearly perceptible to listeners, the question of how data is mapped onto sound is often crucial.

This paper explores the necessary compromises between quantifying and qualifying data that can aid listeners’ interpretation of sounds. Based on this it discusses different kinds of ‘accuracies’ that may be applied in the creation of sonic representations. These accuracies are often connected with listeners’ perceptions of where they are located within this landscape. It is argued that skillful use of sonification and mapping techniques can deepen and intensify listeners’ experiences of a local landscape.

Hybrid Art Forms: The Way of Seeing Music

Evrim Bilge Erkin

What fascinates people to use visual elements in a musical thinking? Throughout the centuries, musicians and image-makers had suggested new kinds of representation combining music and visual elements. For instance, in Romantic era, color had its leading role to carry out a desired aesthetic. Even as it was in painting, the idea of using color in a musical piece was realized by the inventions of some specific instruments, like color organs. However, it can be stated that the collaboration of seeing and hearing is not confidential among musical sounds and colors. It flourishes other senses with a cross-modal relationship. Beginning of 20th century is the time when this interaction between music and visual art became more intensive. It drives new perspectives for existing arts and made the development of hybrid art forms such as visual music, abstract animation and installation art, possible. By offering new perceptual experiences, these art forms inevitably engaged a relationship between scientific developments and the culture in change.

This paper will focus on relationship between music and visual arts through the idea of hybrid art forms. Within this interdisciplinary approach, it aims to consider scientific and cognitive developments and their effects on art and perception. In this context, examples with regard to some of hybrid art forms will be analyzed and finally compared with recent multi-media works.

Crossing foliage: Activation profile for audio-graphic navigation in foliage clusters

Roland Cahen, Marie-Julie Bourgeois

An analytical and experimental approach of activations profiles for audio-graphic navigation in foliage clusters.

The paper is part of the Topophonie research project, the aim to which is navigation within audio-graphic clusters. Clusters are wide ranges of objects of the same class. By Audio-graphic we mean synchronized audio and graphic object behaviour, both modalities been implemented in a single action.

Among the various examples of these kinds of objects, such as rain, flocks, grains etc, this paper focuses on foliage. We have selected two main audio and visual behaviour in order to find a good and costless way to simulate: the wind and a character crossing foliage.

This paper presents the work of designers and sound designers. It is an experimental approach, where we have tried to analyse the audio-graphic characteristics of foliage through video and simple simulation models. Within the project we are working on the concept of activation profile. An activation profile is a simple way to represent active and shaped event triggers. We needed to be sure that this concept was perceptible. Therefore, we have compared the user experience relative to two different symbolic activation profiles: a point for a character and a line for the wind.