Alternative Approaches to Representing Knowledge in the Human Environment

How is knowledge represented in the environments that surround us? What messages are best promoted, most compelling, or most sophisticated? The impact of our knowledge environments is becoming more apparent as economies become increasingly information-driven and facing our global challenges relies on reliable knowledge. On the scale of the individual, knowledge environments influence the thoughts and feelings that we act upon.


/* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-parent:""; mso-padding-alt:0cm 5.4pt 0cm 5.4pt; mso-para-margin:0cm; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:12.0pt; font-family:Cambria; mso-ascii-font-family:Cambria; mso-ascii-theme-font:minor-latin; mso-hansi-font-family:Cambria; mso-hansi-theme-font:minor-latin;} /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-parent:""; mso-padding-alt:0cm 5.4pt 0cm 5.4pt; mso-para-margin:0cm; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:12.0pt; font-family:Cambria; mso-ascii-font-family:Cambria; mso-ascii-theme-font:minor-latin; mso-hansi-font-family:Cambria; mso-hansi-theme-font:minor-latin;}

So much of what a civilization does, internally and externally, is defined by how it handles information: the degree to which it is controlled, the emphasis placed on discovering new knowledge, and ultimately how knowledge becomes embedded cultural wisdom.

Historically, civilizations have produced special artifacts to contribute to building culture from key knowledge they (usually, their leaders) believed was essential. The ideas of the Catholic Church were famously promulgated by the art and architecture in Florence, and the sand paintings of the Navajo people of North America express spiritual ideas that form core beliefs of their culture. In the present-day United States, however, the two best funded, most visually arresting and ubiquitous informational artifacts are not intended to increase cultural wisdom. Instead, commercial advertising and entertainment share the purpose of producing revenue. Social psychologist Albert Bandura's social learning theory advanced our understanding of how signals embedded in people's environments impact individuals' beliefs, attitudes and, ultimately, behaviors. [1] With their ability to dominate the public messaging environment, commercially motivated interests purchase determinant influence on social norms. Advertising in the US is a $400 billion a year industry. For over one hundred years it has employed not only highly skilled message and image-makers, but psychologists, anthropologists and more recently, neuroscientists, to produce highly sophisticated persuasion schemes. [2] These have had a cummulative, synergistic effect on socially normal beliefs and attitudes that many believe is out of line with wisdom. [3] In contrast, media that could contribute to valuable cultural knowledge is severely under-resourced. Hollywood films routinely cost 1,000 times as much to produce as educational programming ($100 million, versus $100 thousand per product). Broadcast venues have been commercially controlled, leaving negligible room for social concerns. Even when educational or prosocial programming proves to be commercially successful (as was the case with the well-known 1977 television series Roots, or producer Norman Lear's series All in the Family) the industry eschews socially motivated endeavors. [4] Such commercial control of the airwaves sets the US apart from other developed countries from Britain to South Africa to Nepal, where governments reserve significant parts of the broadcast spectrum for prosocial and educational programming.

The proliferation of inexpensive, high quality production tools and the open venues of the Internet and mobile networks now allow alternative and valuable information artifacts to be created and to enter the cultural discourse. Evading both state and commercial attempts to control or bury them, such artifacts can ignite political change and also serve the quieter mechanisms of cultural evolution such as the slow growth of common wisdom.

For over ten years, transdisciplinary collaboration between the Imaging Research Center at the University of Maryland, Baltimore County, the media research and development firm, InfoCulture, LLC and other researchers from the US and Canada has led to experimentation with new forms of knowledge to test how contemporary media tools and venues might best be used for social goals. Each of the three projects described below is aimed at learning how to engage a population in knowledge that might help them improve their health, education and wellbeing.


Like nearly everyone, young people want to feel as good as they can for as long as they can. They want to know how to survive and thrive emotionally. In the US, despite that the pursuit of happiness is a founding ideal, young people are typically provided little knowledge of what helps the brain sustainably produce chemical rewards. Neuroscience and psychology literature as well as the experience of psychiatric clinicians support that when a person commits to the pursuit of meaning and engagement as a way to achieve the most positive emotional states, the euphoric effect meets or surpasses that produced by mood altering drugs or adrenalin-producing high-risk behaviors, but also is sustainable. However, for lack of that wisdom prevailing in socially normal thinking people engage in more destructive pursuits - a problem that has increased human suffering and plagued societies around the world for decades or longer.

To find a way to engage young people in considering the pursuit of meaning and engagement, the US's National Institutes of Health funded The Euphoria Project. Artists and filmmakers worked with neuroscientists to develop content. It became clear that profound discoveries that had been made about the brain had not entered mainstream thought from which young people were taking cues. Specifically, in addition to the neurochemical rewards of pursuing meaning and engagement, such a pursuit and the rewards it supplies feed off each another in a feedback loop - suggesting to young people that the effort they might expend will be multiplied when returned. Further, the sheer power, complexity and beauty of even a single neuron firing, combined with the astronomical number of connections in the brain is impressive knowledge that could build self-efficacy (one's belief in one's own ability to accomplish and objective). Especially when combined with information about neural plasticity - the concept that a brain grows and changes to increase its ability to face new challenges.

The story seemed strong, but to learn more about how to tell it in a way that might cause young people to engaged with it, researchers chose to represent the knowledge in a feature film, but to abandon the structural conventions of educational and informational media designed to be clearly understood in real time on the first screening. Instead, the new experimental design would be informed by structural concepts found in celebrated works in all the arts - works that have historically engaged audiences and become culturally significant. It would use aesthetics, analogy, ambiguity and authenticity - what the team came to call the 4 As, to compel audiences to reflect and discuss the work to understand it - thus facilitating more personal connection to the ideas in the film. The experimental structure would juxtapose a stream of visual metaphors - three-dimensional sculptures and tableaus built on urban and rural landscapes, against a narrative that wove together some of the neurobiology, social psychology, anthropology and history related to the pursuit of happiness. In one scene, we see a man standing on one leg, constrained by a big box he is wearing and surrounded by the outline of a head drawn in metal pipe. At the same time we hear dialog about the neural basis of depression. In another scene, multicolored clay feet are dropped, one at a time, into a pool of clear blue water where each releases dye which all combine into polychrome clouds that form abstract designs. All the while, the narration discusses the negative impact cross-cultural traumatic conflict can have on an individual's ability to pursue happiness. The narrator himself, rather than appropriating conventional host's garb, wears copper colored, bejeweled shoes and often appears in only his underwear. The incongruity of these juxtapositions plays out for viewers, mostly unexplained. Researchers gambled that it would be better for the film to confuse the audience than be didactic, unimaginative or condescending. The objective was to give the audience something they could not dismiss with easy categorization or predictability, and thus good engage in open-mindedly.

A randomized, controlled study of the film's effect on 500 high school students found that students who saw Euphoria were able to make the connections necessary to understand the content, and reported liking the film more than those who saw the sham film, Storm Chasers reported liking that film. Perhaps most interesting was data from a follow-up survey that showed that the beliefs and attitudes of students who only saw the sham film had shifted toward the ideas in the presented in the Euphoria film, suggesting as the only plausible explanation that the film initiated a social discourse. Though an experiment, Euphoria was accepted into several national film festivals, winning a gold award at the Houston International Film Festival and garnering a substantial amount of praise in the press. Most important, the film showed that the conventions of informational and educational media could be replaced by a more artistic approach and be more effective as a result.


Online and mobile technologies don't just change the ways we do things; they change what things we can do. Fieldtrip is a research project that explores how to best leverage today's portals and venues of communication to provide a specialized social network where teenagers can engage one another in discussions about their thoughts about and feelings toward education. Developing such a discourse on the contemporary technologies that are woven into students' lives outside school, in environments where attitudes about education often form, is something that was previously unaffordable for educators and school systems. In the past, motivational issues had to be addressed by parents, or in school. Of course, young people use these technologies to connect with one another, not with adults. Accordingly, Fieldtrip is based on literature supporting the promise of peer mentoring and peer support to deal with a range of issues.

Researchers used $20 iTunes gift certificates as incentives to recruit a population of 14-19 year-olds to join an online community. Members supplied assent and parental consent forms and filled out an online survey about their attitudes toward, and achievement in, school in order to establish baseline data from which to measure potential changes. One another's real identities were unknown to recruited members. Instead, new online identities were begun as members created screen names by combining three words from a large list (resulting in names such as FreeSushiCasserole and TheWildRose). To prompt dialog on the site, 2-3 short films were posted each day. They were personal video journals made by high school-aged filmmakers. They documented the impact that family, peer and internal struggles were having on the filmmakers' orientations toward school. Through members' written comments, a dialog emerged among community members that would be analyzed to assess the project's potential for shifting members' educational motivations. (Members were not required to watch the films or comment in order to get their iTunes voucher).

The project posed two key challenges: First, to integrate the expertise of adults in order to make the films compelling enough for teenagers to elect to watch, without losing the fact that these were authentic teenage voices. Second was the problem of moderating and facilitating the discussion without distorting it. These questions lie far beneath the veneer of the technologies that made the project possible and reflect the larger, historic question: What is the most constructive relationship between young people and adults in situations where adults are trying to encourage growth?

Professionals mentored the young filmmakers and edited their footage to increase production value, but this expertise was invisible to most people because the faces and voices viewers saw remained those of teenagers. [5] Near-peer-aged, college students of psychology moderated the discussion, chosen with the hope that they could be sensitive to the need to preserve the adolescent-owned character of the discussion but move them forward in constructive directions. [6] Thus, the perception that teenagers controlled the site was maintained.

During the month-long pilot, hundreds of comments by community members accumulated. Analysis showed that the content of comments mapped well onto motivational literature: These were the kinds of discussions that could affect teens' ability to succeed at school. Modifications to the interface of the site and moderation practices were made for a second pilot, and helped further orient community members to the messages in the films, increasing the adolescent engagement in discussions, suggesting self-reflection and the development of beneficial self-perception had occurred. The next step in the research is to scale the online community to reach a wider teenage public and keep it open indefinitely.


The US spends more on healthcare per capita than any other nation and is home to some of the greatest advances in medicine and medical technologies, yet the health of US citizens is ranked 37th in the world. [7] The primary cause of this problem is destructive behaviors such as eating habits and a sedentary lifestyle, rather than a lack of available care. It is clear that people are acting in ways they know will hurt them. Further, commercial messaging aimed at selling potentially harmful consumption is unlikely to be significantly countered by more helpful messages. Could an online public discourse infuse common attitudes with new ways to think about health and tilt the balance back toward more constructive social norms?

That was the research question driving the Speakhealth project. Like the Fieldtrip project, the effort would build discourse with media. This time, however, experts would be a very visible be a part of the mix. To develop content, a transdisciplinary team of medical practitioners, artists and social media producers created three extensive graphic information maps: the first of constructive health ideas, the second of US cultural traits that might facilitate or undermine the adoption of those ideas, and the third of potential co-mission groups. Content emerged when lines were drawn across the three maps, linking ideas with cultural traits and potentially supportive groups. It was decided that the most supportive initial group was likely to be health professionals themselves. The hope was that they would then spread the ideas through and beyond their own networks. Given the modest budget of the project, media that was edgy and improbable would be used to draw attention. A similar strategy had worked in the Euphoria film, and in the Truth anti-smoking campaign sponsored by the Legacy Foundation ( That effort also used unusual, extremely sarcastic and imaginative online films and is credited with 22 percent of the decline in young adult smoking from 25.3 to 18.0 percent between 1999 and 2002. [8]

The website was launched with a talk at a major integrative health conference in front of doctors and other healthcare practitioners. Reaction was extreme and mixed. While some in the audience were enticed, many found the films disturbing. One film presented an enormous computer-generated, though very real-looking human heart orbiting the earth, which then entered the atmosphere and slammed into a suburban cul-de-sac, bouncing nearby residents out of bed and leaving their traffic circle in flames. The short film was intended to introduce the site's visitors to research findings indicating that a sense of community, something US suburbs often struggle to establish, is significantly correlated with improved cardiovascular health. [9] The audience did not expect to see such departures from the norm. Over the following six months, the Speakhealth project built an active online community. It was clear that the most controversial and/or imaginative media created the biggest draw. However, the project's research sponsor, an independent integrative medicine organization, became fearful that this media would harm their funding support and alienate some colleagues. Rather than allow the sponsor's organizational needs to redirect the project in ways not supported by literature and experience, the research team chose to end the project.

As the projects described above indicate, the opportunities for social progress offered by unprecedented access to mass audiences provided by new media and communication technologies are only beginning to be understood. A great deal seems possible, but research is necessary to test new approaches.


References and Notes: 
  1. Albert Bandura, Social Foundations of Thought and Action (Englewood Cliffs: Prentice Hall, 1986).
  2. Bryant Paul, Michael Salwen and Michel Dupagne, "The Third-Person Effect: A Meta-Analysis of the Perceptual Hypothesis," Mass Communication and Society 3, no.1 (2002): 57-85.
  3. Tamara Piety, "Merchants of Discontent," Seattle University Law Review 25, no. 377 (2001).
  4. Arvind Singhal and Everett M. Rogers, "A Theoretical Agenda for Entertainment-Education," Communication Theory 12, no. 2 (2002): 117-135.
  5. Lee Boot et al, "The Fieldtrip Project: An Online Community Featuring Teen's Cellcam Films Sparked Substantive Peer Discussion," International Journal of Ubiquitous Learning 1, no. 4 (2009): 79-88.
  6. David Gurzick and K. White, "Developers and Moderators: Lessons Learned in the Co-development of an Online Social Space," paper presented at HCI International (HCII), at San Diego, CA, USA, 2009.
  7. The Commonwealth Fund Commission on a High Performance Health System, "Why Not the Best?" New York: The Commonwealth Fund, (2008).
  8. Matthew Farrelly, et al, "Evidence of a Dose--Response Relationship Between 'Truth' Antismoking Ads and Youth Smoking Prevalence," Am J Public Health 95, no. 3 (2005): 425-431. 
  9. Penelope Hawe and Alan Shiell, "Social Capital and Health Promotion: A Review," Social Science & Medicine 51, no. 6 (2000): 871-885.




Unnecessary Research, what's the point?

This panel outlines “The Institute of Unnecessary Research”(IUR) and presents a new paradigm in the way artists are engaging with the world through transdisciplinary practices. It brings together art, science and philosophy by creating participatory audience experiences. Sometimes humorous and sometimes grotesque, our work pushes boundaries and critically questions the means of knowledge production in the 21st Century.


Artists are innovators, if a new piece of technology or a new medium, becomes available; artists want to try it, to experiment with it- from microbiology to robotics; from tissue culture to neuroscience. Some artists take on the role of a scientist in almost a performative way and some scientists become artists themselves. Philosophy and ethics is always at its core and the work unpacks the instrumentalization of science and art for commercial and political ends.

Forms of “connective aesthetics” (Gablik) are used to engage audiences in participatory experiences that extend and generate new outcomes through exhibitions and events going beyond simple interactivity, throwing authorship into question, as members of the audience are inspired to become Unnecessary Researchers in their own rights.

The IUR was founded in 2005 by Artist Anna Dumitriu following discussions at the “Rules of Engagement” Conference on the nature of Art and Science collaboration, held at York University, UK and organized by Arts Council England. The original ‘blue sky’ vision for the IUR was a major research facility where scientists were employed to work with artists, thereby avoiding the common situation of scientists’ lack of availability/time when engaged in art/science collaborative projects. Scientists tend to view a collaborative art/science project as extra-curricular to their ‘day jobs’, whereas to an artist the collaboration is often key to their ‘day jobs’ in terms of being either a grant funded project, commissioned piece or artists’ residency. This inconsistency is one of the biggest hurdles for art/science collaboration to overcome, often far greater than issues, such as funding, audience engagement and linguistic incommensurability. Key to the notion of art/science collaboration are these reoccurring questions, “What is the purpose of it?” “What can an artist offer to science”, “In terms of art, why engage with science at all?” “What levels of cross-fertilization should happen” and most importantly “what has art got to do with knowledge anyway?” The IUR attempts to work with these questions.

There are obvious financial issues with building a major research facility for artists to work with scientists (the IUR favours a underground facility carved out of a rocky island that can only be reached by boat or helicopter (for purely aesthetic reasons)) so it was decided that The IUR should initially be started as a hub for artists or scientists working a high levels of trans-disciplinary practice, strongly concerned with the philosophical implications of their methodolologies, interested in public engagement and practicing in ways that could be described as ‘performative’ in nature. A web site was set up in 2005 and a very low-key performance event took place in Dumitriu’s studio above The Phoenix Gallery in Brighton, England. Since then the has project attracted wide interest and has grown form there, including further performances and interventions at Sussex University, The Whitechapel Gallery in London, ETH in Zurich and as part of many festivals.

The Institute of Unnecessary Research is now an international hub for researchers and artists working experimentally and deeply engaged with their specific research areas. We present our research through performative and experiential methods, engaging the public and new audiences in our ideas.

The IUR uses performance as a means of conveying research; often events have an interactive component, the audience taking part in experiments and research activities thus changing the direction, development and final outcomes of the artwork. Critical theorist Suzi Gablik discusses in her essay on “Connective Aesthetics” that the traditional relationship of the artist to the artwork has come to be superseded, and that this social role of art has become increasingly important, since there is:

“.. a rejection of modernism’s bogus ideology of neutrality. Many artists now refuse the notion of a completely narcissistic exhibition practice as the desirable goal for art”. (Gablik)

Artists have now come to see the process as equal to, or even more important than the outcome, or the performance is more important than the documentation of it. So the means of production of the artwork as a dialogical and collaborative process is also the outcome of the artwork in this model, which is what makes it so relevant to Art/Science practice, it is an analogue of the typical, natural relationship of the artist to the scientist (and vice versa), the journey rather than the destination. Although not inherent in all Art/Science practice it would seem logical to include the audience in the collaboration, with their own vast tracts of knowledge and experience. Gablik states:

“..there is distinct shift in the locus of creativity from the autonomous, self contained individual to a new dialogical structure that frequently is not the product of a single individual but the result of a collaborative and interdependent process”. (Gablik)

This influence of performative, dialogical aesthetics, which comes from the collaborative structure of Art/Science practice makes it a useful technique for reaching out to new audiences in a non-hierarchical way. But these forms of collaboration are not easy and require huge conviction, and effort from all partners involved.

The IUR mimics and subverts the Institutional model it is based on. There are various ‘departments’ each ‘headed’ by an unnecessary researcher. The ‘department’ names are created by the individual artists, scientists and philosophers and based on their personal research areas. When a researcher joins they come up with a ‘department’ name, if they leave (and the IUR is a dynamic group in this sense) it is likely that the ‘department’ is discontinued (at least for a while)

For instance the Head of Crockery resigned his role (from within the online cyber world Second Life in 2006), as part of a multimedia performance at Sussex University, the position of Head of Crockery currently remains unfilled.

There is no official selection procedure for department heads, unsolicited enquiries are responded to with a warning that selection procedures  “are entirely nepotistic”, in fact the IUR grows organically through increasing networks of international contacts.  Current departments include: ‘Projective Geometry’ (Alex May), ‘The Digital Simulacra’ (Luke Robert Mason), ‘Neuroplastic Arts’ (Gordana Novakovic), ‘Textile Abuse’ (Bettina Shuelke) and ‘Viral Contagion’ (Tagny Duff). There are currently 25 departments across distributed locations and the project is directed and co-ordinated by Anna Dumitriu (whilst working on her own research interests which cross microbiology, artificial life, robotics and ethics).

The name “The Institute of Unnecessary Research” is, in many ways, confrontational. It raises the question what is necessary research? Unnecessary does not imply pointless, it often means going beyond the normal (in the Kuhnian sense of ‘normal science’) and crossing boundaries, asking where do we draw the line with what we study or with what can be studied? Unnecessary Research encourages eccentric, obsessive, creative working practices and is an antidote to the stranglehold placed on research by central government and the gatekeepers of academia.

References and Notes: 

Suzi Gablik,, Connective Aesthetics: Art after Individualism in Mapping the Terrain: New Genre Public Art. Ed. Suzanne Lacy (Bay Press, 1995), p85

The Institute of Unnecessary Research official website  (accessed 23 June, 2011) 



Art and Play in Interactive Projections: Three Cases

 We examine how three art-related projection projects approached issues of viewer participation, interactivity, user input and artistic expression differently. Each project presented video projections to a non-specialist audience with software controlled interactivity. One objective was to create an ambient play experience in a public space – something without a beginning or an end that participants could join and leave casually.


Terms of Engagement

The three interactive video projects presented here were led by the authors and emerged from an academic research context. The projects started with different assumptions about the user, different communication goals and different production and collaboration strategies. Although they share some features with simulations, they all avoid standard gaming conventions; there are no levels, no overt objectives, no winners or losers. Additionally, the core requirements of each project were linked to the contexts they emerged from: experience design, art practice and healthcare delivery.

In each of these three productions, Tentacles, Trio and The Art of Waiting, the viewer is encouraged to participate in unstructured play. As with every interactive experience (and in fact, most other things in life) there is the initial satisfaction resulting from simply figuring out how one’s decisions, gestures and actions cause reactions and create effects in the surrounding environment.

Tentacles is a large public projection with game-like user controls accessible through an iPhone. [1] In Tentacles, launching the app on your phone instantiates your creature on the screen. The main control interface on the phone allows you to move your creature around in the projected space. Some things you encounter make you bigger and some make you smaller. You might find yourself the sole inhabitant of this watery world, or you might share it with other players standing nearby. But what you do beyond that is up to you. We observed that some players try to grow their creature by directing it towards nourishing bits floating around, only to find that the bigger they become the more sluggishly they react to the controls. Others entangle with their peers, either affectionately or aggressively, thrusting themselves about like virtual egos. Finally, there are players who seem to delight in simply moving around on the screen, perhaps dancing, flirting or simply pleasing onlookers with their grace and style. This spontaneous performance could be the form of action which most connects the player with their creature. Proudly aware that they are watching their own avatars, we have often observed players with their free hand outstretched towards the screen, pointing out their movements to their human friends, but also appearing to want to touch their creature, in a way reminiscent of Michelangelo’s image of God reaching to Adam in the Sistine Chapel.

Trio is an interactive video art installation displaying three folk musicians playing a song together. Viewers using mobile phones can switch between different musicians to create alternate arrangements of the song. In Trio, like Tentacles, there is much happening on the screen before the viewer even chooses to interact. Three large projected musicians sway gently as they strum and pluck their country music song. A prominently displayed phone number encourages people to engage by dialling in, but with little explanation of what they might expect to happen next. Unlike Tentacles however, Trio presents a dense layer of detailed instructions once the viewer has logged in. Prompted by a long rambling poem on the other end of the phone line, the viewer learns they can control the images by substituting one musician for another:

“Press 1 to reach out to Iriz. Press 2 to connect with Steven. Press 3 to tickle Diego. 1 for bass; 2 for uke; 3 for squeezebox, 4 for zither, 5 for slower, 6 for faster, 7 for heaven, 8 for eleven, 9 if you want to call Golan Levin, 0 for naught, 0 for naught…” [2]

At first the parameters seem straightforward: substitute one musician for another by pushing the buttons on your phone. But beyond that, the rules are invented by the players. Some might try to press the right button sequence to put together an all-girl band. Or a bearded band. Or create a grunge version of the ensemble. At the same time other players in the crowd are thwarting your plans because they have their own agendas. The loose structure of the play interaction encourages this sort of spontaneous improvisation and ad hoc gameplay. Whatever value the content had for the viewer is supplanted by the thrill of empowerment over the interaction.

In the third project, The Art of Waiting fused the productive impulses of the designers with those of the players. [3] A group of university art and design students worked with researchers at a children’s rehabilitation hospital to produce interactive experiences for a large-screen projection in a clinic waiting area. The requirements were unusual. A nine square-meter area with an array of 100 densely packed pressure sensors in the floor created an input sensor that would be equally accessible to children with motor impairments including those using wheelchairs and assistive devices. Even parents and attendants could engage from their seats at the periphery of the area by reaching a toe in and touching the floor.

The designers of this interactive environment had therapeutic goals in mind. [4] In order to calm children before potentially stressful medical appointments and to empower mobility restricted children, the interactions needed to reward slow or static behaviours as much or more than energetic behaviours. Collaborative or social actions were also considered desirable. And since many visits involved more than one period in the waiting area, persistence within the interactive experience would create a sense of familiarity and comfort for children returning 30–40 minutes later.

The third year art and design students created two fully functional interactions which addressed these demands. One depicted a sloped grid of 100 squares, each corresponding to one of the in-floor sensors. A player pausing on a sensor would cause a virtual plant to begin growing on that spot on the screen. Moving to another sensor would cause the original plant to shrink and start another one growing. But if a player stayed long enough in one spot, their plant would become more permanent, persisting over a period of time proportional to the time spent creating it. Slowly moving across all of the sensors would create a virtual forest and several players working together could come closer to achieving maximum density.

As with the previous examples, the richness of the imagery suggests that the ‘game’ is fully fashioned and that one simply has to discern the rules. But once again, the activity ends up encouraging unstructured play, and social rules of engagement are negotiated in real time among the participants.

User Control – Input Paradigms

While all three of these installations share a common presentation form – a single projected video image controlled by software – they each use different techniques and strategies for collecting input from the users. Tentacles and Trio use handheld devices and The Art of Waiting uses environmental sensors.

Tentacles was originally designed to be controlled with an iPhone or iPod Touch and was later extended to include the iPad and Android devices. The application presents a graphic interface that reflects the images projected on the large screen. Users drag their fingers across their touch screens to steer their creatures around in the projection. The further they move from the centre of their handheld screen the faster their creature moves. There is visual feedback on the small screen to indicate direction and speed.

In addition to the visuals, however, there are sound components which go further to link the large screen to the small screen of the device. A background soundtrack plays in conjunction with the large image, augmented by smaller musical elements which play asynchronously on each user’s device. When you engage, your handheld unit springs to life, emitting sounds which intersect with the musical soundscape, calling to and enveloping passersby and proliferating as more people join in. At this point players and non-players become acutely aware that the creatures on the large screen represent participants who are in the crowd all around them. The multiple sound sources, like the multiple participants holding their small devices, combine to form one single social entity, which is only partially revealed on the main visual and aural display in front of them.

The custom device application for Tentacles is meant to extend the experience from the large screen to the small screen and to afford control features specifically designed for this interaction. In contrast, Trio uses the familiar paradigm of a phone call directed to an automated response system. Pre-recorded messages prompt uses to input their control choices by pressing the number keys on their phones. While hardly intuitive, this form of interaction is so familiar to anyone who has ever tried phoning a company or institution that users quickly move on and begin exploring how their choices affect the projected images. Separate from the longwinded recorded message, patterns correlated to the user’s input begin to emerge. The leftmost musician on the screen can be swapped with two others by using the leftmost buttons on the keypad: 1, 4 and 7. The centre musician is controlled with the centre keys: 2, 5 and 8. And the right with the right: 3, 6 and 9. This discovery frees the user from having to try to understand the complex verbal instructions. Additionally, the one anomalous key: 0 turns out to create a short, stuttering effect as though pressing it came close to crashing the system.

The decision to use the simplest phone interface was based on several considerations. First was the techno-social fact that not everyone has chosen to invest in a touch screen phone and their data subscriber plans (about $80/month in Canada). Second, the imagery in the work features a cultural celebration that is somewhat out of sync with contemporary media culture – i.e. folksy, amateur musicians – and seemed incongruous with a slick, technical presentation. Finally, the separation of the recorded message from its functional value allowed it to take on its own poetic role within the overall experience. Once users figured things out and became engaged with controlling and altering the large public images, and once they had shared the experience with friends and strangers, they would often return to the spoken text on the phone, as though to a little private performance in their ear. As with Tentacles, Trio uses the handheld device to create a small scale, private experience within the larger shared public interaction.

So many interactive installations rely on sensor input (rather than device control) that it is not uncommon to have young people walking up to a screen and start waving their arms around expecting it to respond. The Art of Waiting chose to use in-floor sensors for user input for reasons related to the specifics of the user group. The installation is to be installed in the clinic waiting area of a children’s rehab hospital. Users of the waiting area range in age from infants to 18-year-olds. Many have physical or cognitive disabilities and most are waiting with their parents or an attendant for a medical examination, treatment or diagnosis. The goal of the project was to create an interaction that would be accessible to almost all users of the space, and that would be a calming activity in a potentially stressful situation. [5] Relatively few of the users were likely to have cell phones or other devices. Switches and touch screens were considered too inaccessible to users with limited motor abilities and also created risks associated with infection or contamination. Motion sensors typically use some kind of camera which was considered inappropriate for a sensitive medical facility. The floor sensors were chosen as the most accessible means of input, regardless of ability. A grid of one hundred 30-centimetre square tiles is installed under a carpet in an area bordered by chairs. With this configuration, every visitor to the space is automatically providing input to the system. Even a passive engagement – merely being present – causes actions and transformations to take place on the projection screen.

Scale = Public = Shared = Social

The interplay of scale in the first two installationsthe small screen in the palm of one’s hand contrasted with the large public screen on the facade of a buildingparallels other central human experiences. The intimacy of touch, for example, is contrasted by the dominance of projected, broadcast visual stimuli, while the screenthe signforms a kind of text waiting to be read. Your personal space simultaneously shrinks and expands as the tiny gestures you make with your fingers are magnified for all to see.  Public and private stand in stark contrast, highlighting dichotomies like wireless and wired, perception and cognition, knowing and being.

Operating from within the crowd, viewers or players had the opportunity to step onto the stage of the projected environmentto display themselves in action, engaged with other virtual beings. Movements, gestures and displays become part of this spontaneous public performance, suggestive of the activity on a dance floor, where typical rules about decorum, reservation, engagement with strangers and physical contact are suspended. Each private, gestural experience is amplified publicly as a by-product of being within a crowd.

Taking action in public in this way constitutes one layer in the creation of community. Our behaviours and others’ meld to generate simultaneous effects, creating a joint awareness that forms the cornerstone of our collectivity.

In all three installations play is presented as a free-form, creative activitya childlike enthrallment with exploration, skill-learning and sharing. The scale and location of the displays encourages parallel play and the growing awareness of the activities of other players nearby. The public nature of the experience creates the opportunity for ambient performance, where other players’ awareness of you subtly influences and rewards your behaviour. Finally, these factors combine with the ambiguous structures and activities built into each project to encourage social play and collaboration in an emerging, shared activity.

Gamesor rules-based playemerge later in life and become the standard in the adult world. But the works presented here offer a simpler experience to their usersone that is direct and immediate.

References and Notes: 

  1. Tentacles' official Web Site. (accessed September, 2011).
  2. Geoffrey Shea, Trio, 2010, (accessed September, 2011).
  3. E. Biddiss, P. McKeever and G. Shea, “The Art of Waiting – Interactive Displays in Healthcare Settings” (paper presented at CHI 2011, Vancouver, May 7-12, 2011). (accessed September, 2011).
  4. P. Jessee, H. Wilson and D. Morgan, “Medical Play for Young Children,” in Childhood Education 76, no. 4 (2000): 215-218.
  5. C. Spielberger , R. Gorsuch and R. Lushene, STAI Manual for the State-Trait Anxiety Inventory (Palo Alto, CA: Consulting Psychologist Press, 1977).

European Media Art Network / European Media Artists in Residence Exchange

The Program includes a showcase of selected EMARE Productions of the past 15 Years presented by Peter Zorn, EMAN / EMARE Manager and Prof. Mike Stubbs, CEO of FACT, Liverpool.
Friday, 16 September, 2011 - 13:00 - 14:30
Peter Zorn
Mike Stubbs
Peter Zorn
Mike Stubbs

Chair: Peter Zorn
2nd Chair: Prof. Mike Stubbs

European Media Art Network
European Media Artists in Residence Exchange

Syndicate content