embodiment

Circles and Props - Making Unknown Technology

The OWL project is an evolving interrogation of how we might imagine technologies that do not yet exist. The paper describes the theoretical background and structure of a series of workshops aimed at allowing participants to create their own personal technological fantasies. We explain the background for each conceptual shift in the process and attempt to outline how and why they may work.

Author(s)

How will you go about finding that thing the nature of which is totally unknown to you?

Meno, from Plato's dialogue (in Solnit, 2005) [1]

Any sufficiently advanced technology is indistinguishable from magic.

Arthur C. Clarke, Profiles from the Future [2]

The Unknown

It is almost impossible to imagine what lies ahead. What will the future bring? How could life be different? The OWL project is an evolving interrogation of how we might imagine technologies that do not yet exist. How can we support the emergence of radical future technologies that reflect and respond to our personal desires? Asking someone to imagine yet-to-be-imagined technologies puts a large strain on our ability to bring ideas into being. What do you really want, if you could have anything? It is an awful question to ask and when you do, you will mostly get simple, modest answers. In the quote above Meno asks how we will go about finding that thing “the nature of which is unknown to us.” [1] The OWL Circles were created as an attempt to find a way to blot out the most immediate answers, so that we might access more instinctual, and perhaps less plausible responses.

The Circles are purposely designed as a way to sneak up on ourselves, to be caught unaware and unselfconscious for a moment so that we dare to begin. Our aim is to elicit nuanced, imaginative and implausible responses that challenge and stretch what we consider to be possible. We begin with the body and use ideas of enchantment, ambiguity and play, as vehicles through which to contemplate Meno’s question, and thereby support the conception of “sufficiently advanced technology.” [2] The Circle workshop experience takes the participant through a rapid series of formalised conceptual shifts, that each draw on large areas of work in theatre and performance theory, game-play and motivational psychology. This paper is an attempt to account for these shifts and the body of work that lies behind them.

The OWL Circle Workshops

The purpose of the OWL Circles is to allow participants to create their personal technological fantasy. They are hosted in a neutral, utilitarian space, containing a large shared worktable with various tools and lights, and another table, off to the side, containing various neatly organized recycled materials. Neutral colors predominate. The materials are chosen to afford a large range of structural possibilities and aesthetics. A small area is also set up for video interviews, with a video camera on a tripod in front of a black wall. Ideally, the circles are conducted with twelve participants and two workshop facilitators. The format has evolved until it was reduced to the following, strict sequence of conceptual shifts:

  • Introduction: Welcome and brief introduction, including the reading out loud of the quotes from Arthur C. Clarke and Meno. [1] [2]
  • The Desires: A list of common desires are read aloud and placed on the table in the form of index cards. [3] Participants are asked to choose one.
  • Transfer to Body: Participants are asked to identify in which body part their chosen desire resides.
  • The Material Switch: Participants choose materials they find appealing.
  • Thinking with Your Hands: Without knowing what to do in advance, participants begin making.
  • Being 'Done': When they recognize that they are 'done,' each participant is led to the video interview corner.
  • Description: While being fitted with a microphone participants are instructed to tell us: their name, their desire, what their object is called and what it does. The answers are filmed in one take.
  • Debrief: A short debrief is performed to complete the process

What is Happening?

In the following we explain the background for each conceptual shift in the workshop process and attempt to outline how and why they may work. The main component is a series of estrangement switches that shift the mindset of the group away from the predictable and towards a temporary moment of otherness.

The Introduction:

The introduction functions as the drawing of a circle or the beginning of a game and as such it serves a number of functions. In a theatrical sense,  it declares that a game is beginning. Caillois specifies a number of characteristics for games: they are engaged in by choice; they are separate from the routine of life, and occupy their own time and space; games are uncertain: the results cannot be predetermined, the players’ initiative is therefore required; games are unproductive: they create no wealth and end as they begin; games are governed by strict rules that suspend ordinary laws and behaviors; and, finally, they involve make-believe that confirms in players the existence of imagined realities that may be set against 'real life.' [4] By framing the circle as a game, Caillois’s characteristics automatically come into play. This liberates qualities of attention and engagement that are useful when trying to find “that thing the nature of which is unknown,” [1] while Clarke’s assertion that “any sufficiently advanced technology is indistinguishable from magic” [2] further emphasizes the game-like quality of what we are trying to do; at the same time as it focuses our quest into the realm of technology.

The Desires:

The list of desires that we use is borrowed from Steven Reiss's research on motivational psychology. [3] Reiss’s desires are usefully provocative: they reduce a complex emotional field down to someone else’s shorthand definition of the world. They also introduce language before we know what we might be describing, and thereby provide an uncommon point of departure for an embodied discovery process. Choosing to approach a difficult subject in a complex or convoluted manner is a common strategy of fine art. The underlying assumption is that to ‘free up’ the creative and expressive body to respond to the unanswerable, we must first ‘busy’ the reasoning part of the brain so that it will not interfere. [5] The sparse, yet strict instructions that we provide act as a structure that engages the reasoning part of the brain; freeing participants to be spontaneous, to follow their intuition, aesthetic scents and creative whims. [6] It allows them to trust and follow their instincts. The list of desires acts as the first estrangement switch, and is followed very closely by the next conceptual shift: the transfer to the body. Importantly, the facilitators remain neutral throughout, accepting all choices as equally valid.

The list of desires is:

  • Acceptance, the need for approval
  • Curiosity, the need to learn
  • Eating, the need for food
  • Family, the need to raise children
  • Honor, the need to be loyal to the traditional values of one's clan/ethnic group
  • Idealism, the need for social justice
  • Independence, the need for individuality
  • Order, the need for organized, stable, predictable environments
  • Physical activity, the need for exercise
  • Power, the need for influence of will
  • Romance, the need for sex
  • Saving, the need to collect
  • Social contact, the need for friends (peer relationships)
  • Status, the need for social standing/importance
  • Tranquility, the need to be safe
  • Vengeance, the need to strike back/to win

The Transfer to Body:

“Where in your body does your chosen desire reside?” This question acts as a second estrangement switch, transferring from, and connecting, desire to body. It is a nonsensical question that draws heavily on surrealist art strategies, liberating in their absurdity. [7] “If you were a color what colour would you be?” Children know this game and have answers for these types of inquiries. The switch between an abstract desire, defined very strictly by someone else and the feeling that this word does indeed reside within your body, allows the participants to begin to work. The question is no longer abstract: it has been made concrete and physical. This clear concept now becomes the participants’ guide in the work.        

The Material Switch:

“Find the material that works for you.” This instruction acts as the third estrangement switch and allows the physical making to begin as participants find physical form and texture for the body-feeling that has been identified. Again, the decisions made here are not reasonable, rather participants continue their line of absurdist questioning by asking, "if this feeling had a texture and a shape what would it be?" This process exposes unexpected and poetic possibilities that can be explored from the specific sensory potential of a material to body behaviors as they rise from desires, feelings, and anxieties. Dr. Montessori of course famously used blindfolds in reviewing materials, stating that the eye can interfere with what the hand knows. [8] We could add that language can interfere with what the hand knows. Once the participants have chosen materials, they can begin to build and support their burgeoning concept.

Thinking with Your Hands:

Through the making process the work is one step further removed from reasoning and habitual thinking. The participants have up to this point made three very large leaps of faith: choosing a desire, connecting this desire to their body, and their as yet unnamed feeling to a material texture and expanse. These three switches have occurred in less than fifteen minutes, allowing no time to re-consider or back out into careful reasoning. In a sense, participants are not completely committed at this point, simply because they do not know what it is that they are making. The work that follows is instinctual and effective. The conversation around the table is practical: "Can I have the scissors?" "How do I make this stick out to the side?" [9] Kelly claims that the divorce of the hands from the head puts a strain on the human psyche. [10] This suggests that bringing them back together again through embodied processes releases strain. Having viewed numerous circle participants engage in this process, we suggest that the state that it engenders is tranquil: focused, efficient, relaxed and also gently energetic. Thinking as an emergent bodily process allows us to access knowledge, expertise or connoisseurship that otherwise eludes articulation. [11] The OWL processes leans heavily on this idea.

Being 'Done':

Knowing when a device is 'done' is an instinctual knowing. The circle structure removes verbal reasoning from the imagining and creating process, and frees the participant to trust their ability to recognize what it is they are doing as it emerges, including when it is ‘done.’ This knowing ‘when’ is something we all have experienced, Henri Cartier Bresson called it “the decisive moment,” the moment when the trigger on the camera is pushed. This moment relies on the photographer’s ability to see and record an event literally taking form in the immediate future. [12] Cartier-Bresson’s decisive moment was tied to a particular approach to photography, nonetheless it is useful to provide ways of thinking around the notion of making a device which is yet to be imagined, and knowing when that device is ‘done.’ In musical improvisation, the knowing where to go next becomes a series of small decisions made in a hyper aware state of flow in which the musician ‘knows’ both the minds and desires of his or her fellow musicians, and also holds the experience of the audience as an almost physical thing which can be examined, turned, changed, and at some point is ‘done.’ [13]

Description:

The interview is filmed in one take. Participants are required to think on their feet, to not let their inner dialogue drown out their ideas. We began with language, with the desires and now we return to language again. The process between is embodied, non-lingual or mute. As language floods in, it takes over, surprising the participants. Excluding language from the central part of our structure allows a very intuitive and productive process to emerge and only at the end is reasoning allowed back into the experience. In order to allow this process to appear “on camera” we ask the participant to speak in one-take with minimal intervention from the camera operator. This achieves two things: first it allows the process to remain personal and introverted, the camera operator is just that, an operator facilitating the participant to self-record their piece; and secondly, the switch between an intuitive and wordless making process to a reasoned presentation happens ‘on camera,’ with many participants only realizing what they have built as they name it. To make this final switch more distinct we ask strict, product-like questions. Instead of, "how did you feel?" we are asking, "what does it do?" The strictness of this line of enquiry allows the sometimes hazy decision making process that has come before to crystallize out. The 'product' is described and the participants are thereby brought back into the everyday world. The circle is broken and the game is over.

Debrief:

As a postscript to the overall workshop experience, each participant is debriefed before leaving the workshop space. This allows us to close any conceptual holes, attend to any concerns the participant might have and is an important part of us taking responsibility for the emotions and questions that may arise in an intense experience. It is also where we can explain a little bit more about the background and reasons for the project.

The workshop takes two hours, including the recording of all twelve participants’ work. In that time we have opened a bubble in time in which we were allowed to physically build what did not previously exist, and in turn meditate over our desires, and how they might be met or mitigated.

Nine circles have been conducted to date: three in Tokyo and six in Sydney. Five of the Sydney workshops were targeted towards specific social or community groups: artists with disabilities and their carers; design academics; young children; performing artists; librarians. The outcomes were exhibited as part of the 2010 Participatory Design Conference in Sydney. [14] The breadth of participants and contexts afforded deep reflection, and the development of the strict structure described above. The whole process shifted the way that people thought about and imagined their bodies in relation to technology. The results were not only enchanting, but were deeply felt. [15]

Some Conclusions

Susan Stewart, in her book On Longing proposes that souvenirs are objects of desires that assist in the formation of continuous personal narratives that connect the present with the past. [16] OWL objects and devices connect participants through their imaginations and desires, as well as through the objects themselves, from the present to the future. They give form to, and assist in the formation of continuous, or ongoing personal narratives that support this connection. [17]

The workshops themselves are live, volatile processes, understood in the sense of Dewey’s ‘experience.’ [18] We work with ideas not just in the form of description, where only language can become knowledge and meaning, but rather as a ‘process of becoming’ that, without turning to either romanticism or mysticism, can allow what may appear as chaos to create order and pattern through embodied experiences. Judith Butler states that we are required to, “risk ourselves precisely at moments of unknowingness, when what forms us diverges from what lies before us, when our willingness to become undone in relation to others constitutes our chance of becoming human.” [19] The workshops are purposely built to facilitate this kind of risk taking, to provide a temporary space in which we can ‘become.’  

In Viktor Shklovsky's view, art resists and overturns the deadening effects of habituation. As our "perception becomes habitual," he argues, "all of our habits retreat into the area of the unconsciously automatic" and as a result "we apprehend objects only as shapes with imprecise extensions." [20] Art promises to recover the sense of immediacy and wonder that habit slowly erodes: "The purpose of art is to impart the sensation of things as they are perceived and not as they are known." [20]

The OWL project confronts desires, bodies and dreams about technology. It affects a displacement of desires, by naming them and giving them form, but it also affords giving account from the place Butler speaks of, the place where we become and remain human. The objects that are made are a kind of souvenirs from the future, but where souvenirs remind us 'what happened then' the OWL objects carry stories about 'what happens next.'

References and Notes: 
  1. Rebecca Solnit, A Field Guide to Getting Lost (New York: Viking Penguin, 2005).
  2. Arthur C. Clarke, Profiles of the Future (New York: Holt, Rinehart and Winston, 1984).
  3. Steven Reiss, Who am I: The 16 Basic Desires That Motivate Our Behavior and Define Our Personality (New York: Tarcher/Penguin, 2000).
  4. Roger Caillois, Man, Play and Games (Champaign, IL: University of Illinois Press, 2001).
  5. Rollo May, The Courage to Create (New York: W. W. Norton & Co., 1994).
  6. Anne Bogart, A Director Prepares: Seven Essays on Art and Theatre (London: Routledge, 2001).
  7. Alistair Brotchie, A Book of Surrealist Games, ed Mel Gooding (Boston, MA: Shambhala, 2004).
  8. Angeline Stoll Lillard, Montessori: The Science Behind the Genius (New York: Oxford University Press, 2008).
  9. Kristina Andersen, “Developing Your Own Hardware,” Digital Artists' Handbook Website, 2007 http://www.digitalartistshandbook.org/hardware (accessed April 2, 2012).
  10. Kevin Kelly, What Technology Wants (New York: Viking/Penguin, 2010).
  11. Danielle Wilde, Thecla Schiphorst and Sietske Klooster, “Move to Design/Design to Move: A Conversation about Designing for the Body,” in Interactions 18, no. 4 (2011).
  12. Jack Zichittella, “The Decisive Moment Revealed,” in Arts and Activities 123, no. 5 (1998).
  13. Joel Ryan, “Knowing When,” in The Vibrancy Effect eBook, eds. Chris Salter and Michel van Dartel (forthcoming).
  14. Danielle Wilde and Kristina Andersen, “The OWL Bodyprops Fitting Sessions,” Proceedings of the 11th Participatory Design Conference (PDC 2010), Sydney (November 2010) http://www.daniellewilde.com/dw/OWL_files/OWL_bodyprops_fitting_sessions.pdf (accessed April 2, 2012).
  15. Personal correspondence.
  16. Susan Stewart, On Longing: Narratives of the Miniature, the Gigantic, the Souvenir, the Collection (Durham, NC: Duke University Press, 1993).
  17. Danielle Wilde, “Swing That Thing: Moving to Move” (PhD diss., Monash University, 2011).
  18. John Dewey, Art as Experience (New York: Capricorn Books, 1958).
  19. Judith P. Butler, Giving an Account of Oneself (Bronx, New York: Fordham University Press, 2005).
  20. Jon Beasley-Murray, “Shklovsky,” posthegemony.blogspot.com (blog), January 18, 2011, http://posthegemony.blogspot.com/2011/01/shklovsky.html (accessed April 2, 2012).
  

Hyperpresent Avatars

 

This paper will discuss two student projects, which were developed during a hybrid course between art/design and computer sciences at Sabancı University; both of which involve the creation of two avatars whose visual attributes are determined by data feeds from ‘Real Life’ sources by following up from Biocca's concept of the Cyborg’s Dilemma, we will describe the creative and technological processes which went into the materialization of these two avatars.

Author(s)

Hyperpresence

If the body is the primary communication hardware, then what is its relationship to a medium which is made up of steel, plastic, and silicon given that instead of pulsing blood, pulses of electrons and light animate the computational hardware?

Marshall McLuhan long ago pointed out that communication interfaces attach themselves to the body. In the words of McLuhan, “Media are extensions of the senses,” in that the view of the world associated with print is being replaced by a world view associated with electronic media that stresses feelings and emotions. [1] This is a different vision than Licklider's [2] for whom “mancomputer symbiosis” is a subclass of “man-machine systems” in which the human brain is coupled to its machine counterpart. This coupling of one brain to another made sense in the early days of computing when the communication between human and machine was still one of conversation where instead of a mind communication through a body to another body, we have only two disembodied conversations, a sterile coupling of abstract symbol generators. At the close of 20th century however, the development of advanced computer interfaces is characterized by progressive embodiment. Progressive embodiment is the steadily advancing immersion and coupling of the body to an advanced communication interface.

Intelligence augmentation applies itself to the theory that communication technologies can be cognitive prostheses amplifying or assisting cognitive processes or by developing cognitive skills. This leads to the question, of what it means to be virtually embodied, particularly if this state also contributes to intelligence augmentation. In other words, what arc the psychological effects of goals of embodiment in virtual environments? What are the psychological effects of embodiment in virtual environments? Most commonly these are expressed as various forms of ‘presence,’ which is described as the perceptual sensation of being in a place other than where you physically are, or a sense of transportation to a ‘place’ created by media. [3] It is the illusion of ‘being there’ in a virtual space.

Compounding the dual concepts of (virtual) environment and (virtual) agent are Giuseppe Mantovani and Giuseppe Riva’s findings which point at the social nature of ‘presence,’ challenging the notion that experiencing a simulated environment is merely a matter of perceiving its objective features: Presence (real or simulated) means that individuals perceive themselves, objects, as well as others not only as situated in an external space but as immersed in a socio-cultural web connected through interactions between objects and people. [4]

This social aspect of ‘presence’ is further picked up by Frank Biocca, who seems to question the issue both from an externalized as well as an internalized viewpoint, bringing to the fore the notion of self-presence:

When the user’s body enters the virtual world and inhabits an avatar, a number of changes in self-presence are possible. Self-presence is defined as the effect of virtual environment on the perception of one’s body (i.e., body schema or body image), physiological states, emotional states, perceived traits, and identity. To use a phrase, self-presence refers the effect of the sensory environment on mental models of the self, especially when that model of the self is foregrounded or made salient. As with other forms of presence, designers share the assumption that increases in self-presence are correlated with higher levels of cognitive performance, and, possibly, emotional development. In the words of Socrates, the goal to ‘know thyself’ is a worthy journey. It may be the only journey.” [5]

And it is at this juncture that Biocca formulates a vision, a hypothesis, a wish:

“… it may be possible to develop a medium in which one feels greater “access to the intelligence, intentions, and sensory impressions of another” than is possible in the most intimate face-to-face communication. One aspect of what might be called hyperpresence” (Biocca, 1997) may be possible in the social presence domain as well. Of course, it is hard for us now to imagine a medium that can create greater intimacy than face-to-face communication. But this misses the point of social presence and the very artifice of the body itself. In face-to-face communication the body is used to communicate one’s sensory experiences, observation, and inner states to another. The body is the medium for this transfer. Communication codes such as spoken language and non-verbal codes such as facial expression, posture, touch, and motion are used. But, for example, inner states might be communicated more vividly through the use of sensors that can amplify subtle physiological or nonverbal cues. These can augment the intentional and unintentional cues used in interpersonal communication to assess the emotional states and intentions of others.” [5]

Data Avatars

While Biocca’s deliberations seem to focus on sensor based technologies, there may well be other means of conveying data, which is likely to bring about the communication of inner states, emotional responses and non-verbal clues, including an immediate manifestation of interests and inclinations.

Two avatars which may fulfill such demands, through non-sensor based technologies, were created by two separate groups of students, during different semesters, as course projects for a hybrid art/design and computation course entitled CS450, co-instructed between two artists and one computer scientists at Sabanci University. [6]

Both projects deliberately go against the grain of the prevalent mindsets of metaverse residents which, more often than not, involve a wish for concealment of real life attributes: A study conducted by Brosnan, [7] using 126 participants recruited from Second Life, shows that while the physical persona may be predictive to a certain extent in virtual embodiment, nonetheless in many cases significant differences between physical and virtual appearances and identities is to be expected.

This typical behaviorism is being challenged by bringing data from the physical realm into the metaverse: Rather than create avatars which are vessels of concealment, revelations regarding the physical state of the wearer are being sought. Thus, what is aimed for are wearable virtual technologies which allow their users to be represented in a manner in which both their real life and virtual life traits can be visualized simultaneously, by using data imported from the physical to the virtual realm.

The Miró Avatar

The project originates from the desire to integrate an emotional presence into the World Wide Web. In real life, emotions are not communicated consciously; hence the idea of using Electroencephalography (EEG) to collect a person’s emotions. However, since EEG data cannot have reliable or interpretable meaning concerning any emotional state, one may only speak of collecting the ‘idea’ of one’s emotions. EEG is used to accumulate an individual’s brain signals; signals that occur each moment, unconsciously, in response to the interaction with the immediate environment. Since it was seen to be desirable to interpret these signals as the idea of one’s emotional presence in virtual reality, the three dimensional metaverse of Second Life became a natural platform to apply such a metaphor of reality.

Figure 1: The Miro Avatar, Işıl Demir, Can Şen, Yiğit Yüksel, Second Life, 2008.

The collection of EEG data is done by an open source program, BrainBay, which outputs the biosignal as EDF files. These files are converted to ASCII text files with another open source program, Polyman. The content of these files are integers between -4000 and 4000. The ASCII files are uploaded to a website from which custom made scripts in Linden Scripting Language (LSL) read the files that contain the EEG data.

The avatar changes according to incoming brain wave. When the avatar is activated the script begins reading data from the server and a change in the shape of the avatar according to the incoming integer values is brought about. Thus the user's brain waves form a virtual manifestation that represents his/her virtual appearance which can also be considered as a metaphor for the representation of one's mind; since, figuratively, what is thus visualized are the person's ‘thoughts.’

As far as the creative process is concerned, the visualization of one’s emotional presence has been inspired by the idea of the four dimensional painting which Miró proposed in his later years. Thus, the avatar, composed of the various visual elements featured in Miró’s paintings, continuously changes its shape and is redrawn, transcending the two and three dimensionality of painting and sculpture. As expected, this representation stands in contradiction to the prevalent tendencies of metaverse and MMORPG players who, will either create accurate physical reflections of themselves by making an avatar corresponding to their actual appearance or conversely by giving the avatar physical traits to which they aspire to in real life, but which are entirely out of their reach in the physical realm.

As a general rule three dimensional virtual spaces tend to be simulations of real spaces and as such they can solely be interacted with and experienced through mental processes which are the visual, auditory, and cognitive stimulations in the brain. So, instead of creating an avatar based on actual physical traits, the output of the project offers to create an alternative visual entity, usable as an avatar, derived from the fact that users cannot have a real physical presence in virtual spaces and the fact that their mental input is the only factor that creates the illusion of presence in a virtual space. Other users can ‘see’ them, not because they are physically there; but because there is an avatar that is shaped via their thoughts and desires with which one may interact in a manner similar to face-on-face physical interaction. Thus it may be concluded that, in terms of representation, virtual appearance may well rely on the output of unconscious thoughts, which are what is also mirrored in the surrealist approach of Miró’s paintings.

The PersonaSkin Avatar

The second project involves an avatar who carries several body attachments which change color saturation values based upon a data feed which is generated from the arts and entertainment section of a facebook user’s profile. Although the project was initially intended for real life usage, inspired by an RFID based real life event which tied facebook data to physical bodies, launched in Israel in 2010. However, despite this physical precedent it was decided to first discover the possibilities of identity matching through accessories and outfits in a virtual world. Thus, a metaverse resident who also owns a facebook account can utilize these attachments to project his/her interests to the outside (virtual) world.

Figure 2: The Personaskin Avatar, Ayse Naz Pelen, Doğukan Malbora, Mustafa Cağrı Güven, Second Life, 2011.

According to Swann's self verification theory, during most social interactions there is a general desire for outside evaluations which verify self-views; in other words, a wish to get others to see us in the way in which we see ourselves. Given that Facebook users create their profiles themselves, very much along the lines in which they want to represent themselves, self verification theory has become an important part of this project. The aim is to achieve an understanding as to how persons may choose to introduce themselves in social networks, real and/or virtual, in order to attain states of self verification through identity matching: The avatar is expected to bring them to the notice of persons of similar facebook status, in terms of the frequency of interests presented in the arts and entertainments section thereof.

In terms of technology the data is being taken out of Facebook via php and a Facebook api. Subsequently the data is sent to a server and from there imported into Second Life, where LSL is being used to embed the data into the objects which represent the various categories either by heightened/lessened saturation values or alternatively through different levels of transparencies.

Questions such as age, sex and geographic location appear to become increasingly less relevant in a metaverse environment, where people seem to interact mainly through their ideologies and their creativity which are taken to be standalone attributes which exist independently of the ‘real life’ persona behind the keyboard. Under such circumstances an avatar of androgynous appearance, whose adornments are created out of his or her areas of interest seems to be particularly apt design strategy. Since some kind of legend is needed to decipher visualization of the incoming data the skin of the dramatic full avatar also serves as a legend. In cases where residents who wish to go for a more conservative appearance, a t-shirt and various colorized male and female skins are also included in the package.

Conclusion

In the brave new world of three dimensional, online virtual worlds yet another aspect of our grappling with embodiment is coming to the fore. This is in accord with the notion of cyborg as an interface which couples the physical body with technology [8], within which three dimensionally embodied avatars can also be characterized as a form of cyborg coupling. For Biocca this coupling underscores what he calls the cyborg’s dilemma, which for him is nothing less than a Faustian tradeoff: “Choose technological embodiment to amplify the body, but beware that your body schema and identity may adapt to this cyborg form.” [5]

Thus, a germane question would appear to be whether such attire would be powerful enough to provoke change and transformation not only on the virtual agent but extend its influence into the physical realm, bringing forth new modes of presence as well as self-presence not only in three dimensionally embodied online virtual worlds but also in the one which we inhabit with our flesh and blood selves.

Can avatar attire which reveals, rather than conceals a metaverse resident’s persona aid in the process of self-presence and (virtual) self verification? Can personal change be brought about through technologies which not only reveal our pixelated flesh, but also reveal the biological and cultural fields which we weave around us? Can social interactions be transformed and enhanced through virtual wearables which reveal our inner beings to those around us? Can novel states of creativity and play, of unique observations breeding new forms of authorship and understanding, come about through virtual candor?

While both avatars address these issues, when it comes to the Facebook avatar a further consideration is the integration of a heavily used ‘real life’ virtual social media platform (Facebook) into the metaverse as a socialization tool is a prolific area for further study.

This text has attempted to discuss some of the technological and artistic means through which such questions may be posited, through two projects employing such devices for the creation of two data driven avatar costumes.

Acknowledgments

We wish to express our heartfelt thanks to our students Işıl Demir, Can Şen, Yiğit Yüksel (Miro Avatar) and Ayse Naz Pelen, Doğukan Malbora, Mustafa Cağrı Güven (personaskin avatar) for the brilliant work which they accomplished and which provided the material for this text.

References and Notes: 
  1. Marshall Mcluhan, Understanding Media: The Extensions of Man (Toronto: Signet Books, 1964).

  2. J. C. R. Licklider and Robert W. Taylor, "The Computer as a Communication Device," Science & Technology 76 vol. 2 (1968): 21 - 41.

  3. Jonathan Steuer, "Defining Virtual Reality: Dimensions determining Telepresence," in Communication in the Age of Virtual Reality, eds. F. Biocca and M. Levy (Hillsdale, NJ: Lawrence Erlbaum Associates, 1995), 33-56.

  4. Giuseppe Mantovani and Giuseppe Riva, “Real Presence: How Different Ontologies Generate Different Criteria for Presence, Telepresence, and Virtual Presence,” Presence: Teleoperators & Virtual Environments 8, no. 5 (1999): 540 - 555.

  5. Frank Biocca, “The Cyborg's Dilemma: Progressive Embodiment in Virtual Environments,” in Human Factors in Information Technology 13 (1999): 113-144.

  6. Elif Ayiter and Selim Balcisoy, “Transdisciplinary Avenues in Education,” Computing and Art, Lecture Notes in Computer Science 3942 (2006): 80 – 89.

  7. G. J. Brosnan, M. Doodson and R. Joiner, R., “Using ‘Second Life’ to Support Experiential learning,” PLAT2010: Psychology Learning and Teaching Conference, Edinburgh, UK, Edinburgh Napier University, 2007.

  8. Chris Hables Gray, Heidi Figueroa-Sarriera, Steve Mentor, eds., The Cyborg Handbook (New York: Routledge, 1995).

  

Virtual Instrumentality : Exploring Embodiment in Artistic Installations

In this paper we study the question of interaction with digital technologies by exploring the cognitive mechanisms of embodiment in the context of multisensory artistic installations. In order to test our hypothesis we observed visits to an experimental installation, which provides conceptual and technological consistency. Our first observations suggest that these conditions result in a strong embodiment for the proposed interactions.

Author(s)

Introduction

Digital technologies for creation free us from physical constraints, but at the same time might lead to the loss of instrumentality, that is the very specific, rich and nearly intimate physical relationship between a human being and an object used to perform actions on the environment. [1] Yet the computer, programmed according to certain rules and linked to the man by the appropriate interface, may offer new forms of instrumentality and be considered as the locus of all instrumentalities. [2] We study the cognitive mechanisms of instrumentality in the context of multisensory art installations, inspired by the hypothesis that conceptual and technological consistency of the composing elements of a multisensory virtual environment is important to the instrumentality of the experience, which should result in a strong embodiment of the proposed interfaces and interactions.

Enacting Digital Matter is an art installation based on the simulation of virtual scenes, addressing the visual, auditory and haptic senses; proposing a form of virtual instrumentality based on physical modeling and force-feedback interfaces. It was presented at the European School of Visual Arts (Poitiers, France) in February 2010. Through this installation, most visitors experienced for the first time a multisensory interaction with physically consistent virtual objects; i.e. simulated objects that behave according to Newton’s laws of motion. However, the visitors were presented with sensory paradoxes and unusual situations, such as the possibility to discover an object only by touch, or together with a visual representation apparently conflicting with haptic sensations. Thus, visitors were lead to experience aesthetic and emotional 'shocks' and to question their senses, which is the opportunity to collect essential information about the way our sensory-cognitive system works in an artistic multisensory situation. The installation was designed to capture these unique moments, allowing for further analysis in search of evidences of embodiment.

The Installation

Experiencing the virtual scenes, as well as the consequent commentaries of the visitors, were part of the installation, which was more a performance than a material setup. The aesthetic objects considered were not the force-feedback device, the simulated scenes, nor the resulting sounds and images, but the moments of discovery, exploration and expression by the visitor.

The installation consisted of two simulation stations, each one equipped with a screen, a loudspeaker and an Ergon_X force-feedback interface from Ergos Technologies, which allowed visitors to interact with the virtual scenes through hand and arm gestures. Each scene was based on a physical model created and simulated with the CORDIS-ANIMA system. [3] The sensory consistency of the installation was ensured by the fact that a single physical model produced the audio, visual and haptic signals through a synchronous simulation engine. Each station was also equipped with two video cameras and microphones, so that every visit could be recorded in good conditions for further analysis. This equipment was visible and each visitor was asked to give his or her permission to be filmed and recorded. At the entrance of the installation, a monitor screen displayed what was going on inside through one of the installed cameras.

The Model

Each visitor was offered the possibility to explore one or two scenes among the three available ones: “Pebble Box,” “Friction” and “Approach and Retract.” Each scene corresponded to six different versions that were presented successively. For a given scene, the six versions were based on the same physical model but differed by the presence or absence of one of the sensory channels and by the visual representation. For example, a scene could be presented only with visual feedback in one version (no sound nor force feedback) and with all three sensory channels in another one. In the following, the time spent by a visitor on one version will be called a ‘session.’

In this article, we will focus on the Pebble Box scene. [4] The underlying physical model is composed of eight circular masses enclosed in a flat, circular area (see Fig. 1). Using a force-feedback joystick, the visitor directly moves another, smaller mass in the box. Force feedback gives a haptic feeling of the scene: through his or her hands, the visitor can feel the contact between the manipulated masses and the other ones or the border of the box. The interaction between the masses, including the one manipulated by the visitor, is an elastic collision, whose stiffness is high for some versions of the scene and very low for others, giving respectively hard and soft contacts between the masses. Two visualizations were proposed: a 'blurry' one, which gives the impression of a nearly continuous medium, and a 'ball-like' one, which represents the masses and the limits of the box in a clear, non-ambiguous way. The table on the lower side of Fig. 2 summarizes the different parameters of each version. The order of presentation was the same for all visitors, from version 1 to version 6.

The Pebble Box scene did not have audio output. However, the motors of the Ergon_X interface emit sounds – particularly during hard collisions – which some visitors have remarked on and interpreted (see Results).

Methodology

Realism is well known to be a factor of embodiment and immersion, so it could have interfered with the other factors we wanted to observe through the installation; i.e. the consistency of the sensory sensations and the presence of haptic feedback. Consequently, we gave simple and quite abstract visual representations to the scenes. Visual abstractness was also intended to help the evocation process since no straightforward interpretation of the scenes is given. In the same perspective, the experience proposed to the visitors didn't include any scenario, so as to focus them on the interaction with the simulated objects.

As mentioned previously, the installation explicitly included the fact that visitors could express the sensations and feelings elicited by their interaction with the virtual scenes. To stimulate expression, a coordinator accompanied the visitor in order to facilitate his or her reactions, through an open interview addressing; (1) the felt sense of the experience, (2) how it was felt, and (3) what it felt like. The coordinator encouraged the visitors to go beyond superficial descriptions and comments about what they liked or disliked in the situation. He or she helped them talk about their haptic sensations, which is known to be difficult for many people. The scenes were not presented as being a representation of any existing situation: they were only designated through numbers (e.g. “Scene 1”) and the coordinators did not make any suggestions that could lead the visitor to a particular interpretation. As a consequence, the resulting subjective descriptions were expected to access deeper levels of consciousness related to the felt experience, for example through ‘forgotten’ memories or evocative thoughts.

Six visits to the Pebble Box scene have been recorded (see Fig. 2). The visitors were all men, aged from 20 to 55 years old, most of them having an artistic background. We will refer to them with an arbitrary number (e.g. “Visitor 1”), which is not related to the order in which they visited the installation. The visits lasted approximately one hour.

Results

We focus here on three main observations that suggest the connection between consistent sensory signals and embodiment, or, in other words, what we call virtual instrumentality. Instrumentality in the virtual world is the result of an embodied interaction, which enables the human capabilities to incorporate the new situation. The instrument becomes an extension of the hand, and can be used fluidly and intuitively to explore the given possibilities of the virtual world. In the Pebble Box scene, the instrument considered is a hybrid system constituted of the force-feedback device (real-world part) and the simulated mass that is connected to it (virtual part). The structure of the process towards instrumentality is here explored in three constituents:

Embodied memories

In order to explain the newly felt sensations, visitors were suggested to employ a strategy of transposing them to another experience, felt in the past. The experiences they chose in order to describe their sensations were characterized by a strong embodied quality. They were about sensations from their daily routine, or deep-anchored senses of their past. These felt memories came to explain the actual haptic situation.

Here is how Visitor 1 describes his sensations when exploring the scene in the first session (no visualization, hard contacts):

The images that come to my mind are situations where, sometimes, I wake up in the morning, on my bedside table, there is a glass of tea, my glasses, stacked books, the alarm clock, handkerchief packs, and things like that, and I try to catch my glasses to check what time it is and so I grope around saying to myself ‘I’m going to try not to knock anything over...' and then suddenly you put your eyes at your fingertips.

During the first session too, Visitor 2 explains that the haptic sensation he experiences is actually familiar and he can remember it from another situation: “I know from experience, I’ve done this before, I can remember that sensation... when I was riding a bike, the friction of the brake on the front wheel.”

Spatial representations through haptic feedback

The sensation of a space, opening up at the haptic sense, has been described during most sessions and by most visitors. For example, during the first session, even though there is no visualization, Visitor 3 said:

I think there are still constraints, that is to say... places, places... For example I have the impression that I feel a kind of ball, a kind of place where I am below. [...] I’m navigating around a space, into a space [...] there are empty places, there are full and empty spaces... Well, I really see it as a plane, [A/N: a flat surface] it’s on a plane.

The description of the virtual space gained in subtleness during the third session, when a visualization of the scene, the blurry one, is given for the first time. It became instantly clear for all participants that the image was the graphic representation of the virtual space they had explored haptically. Visitor 3 declared: “Yes, this is the graphical representation of this space!”

Visitor 3, recognized the image as a graphical representation while continuing to manipulate the haptic device. The connection between gesture and graphics came as the result of doing. An image coming as a verification of the haptic sense has also been discussed in the paper of David Prytherch and Bob Jerrard. [5]

In addition to that, Visitor 2 describes how he was able to refine the characteristics of this space: “I think there are several stages with the joystick. All around, well... there is nothing acting. Then, there is a resistance appearing around, when moving towards the center of the joystick, there is a resistance that comes in.”

Visitor 3 also described with more details the virtual space: “It's as if there was a ... a circular constraint, in the center, a ring, and I can go either outside or inside it. Now I'm in the inner ring, and if I force a bit I move to the outer ring.”

During the fourth session (blurry visualization, soft objects), visitors talked about sensing a curved space, a feeling that can be due to the succession of repulsion and contraction forces. Here is what Visitor 4 said: “I have a space which is rather curved. A haptic space.”

During the sixth session (ball-like visualization, rigid objects), all of the participants talked about how the virtual space they felt before was finally revealed to them. They were able to identify the haptic sensations they experienced during the previous sessions and felt the connection between the mental representation of the space they had constructed and the visual space presented to them during this final session. Visitor 3 expresses this connection between the visual and haptic channels in a particularly strong manner: “There, this is what I wanted from the start!”

This quotation suggests that the mental representation of the scene elicited by the haptic channel was strong enough to call for a specific visualization, which corresponds to the ball-like one. Notice also how most of the visitors use first person expressions to describe the sensations. This point is really important to us because it indicates clearly an effective implication and immersion into the virtual scene. It seems that visitors were projecting themselves in the scene through the instrument instead of considering it as an intermediate between them: the instrument was, at least partially, incorporated. This tendency was probably reinforced by the fact that they didn’t clearly see what they where actually manipulating until the fifth session.

All these remarks indicate that it is possible to create a strong representation of the virtual space with haptic sensations as the main input. However, during the second (no visualization, soft contacts) and fifth (ball-like visualization, hard contacts, no haptic feedback) sessions, it is really remarkable that none of the visitors talked about space. On the contrary, when strong force feedback was there, the feeling of touching the space was present, and even augmented by visual representations of the scene.

The machine embodies an independent living agent

The haptic device of the installation takes on a life of its own, it becomes an autonomous entity, with its own will and personality. The visitors interpret its feedback and reaction as well as the mechanic sounds, as a dialog between them and the machine. They ascribed mystic ways to its performance.

“She [A/N: the machine] doesn’t want me to reach the central position.”

“When I pivot, I feel that it kind of stands up to me.”

“I like the sound of the machine... you know... its way of conversing too [...] I don’t know if it’s a dialog... Well yes it is, it’s a dialog [...] if we say that this is a reactive entity, maybe a living entity, I don’t know, it’s… this movement to make it feel good, or bad... according to its reaction.”

We observed differences in the degree people experienced it in different situations. On one hand, in the first session, the machine’s reaction was perceived as stubbornness, as a resistance to the visitor’s intention to manipulate it.

On the other hand, in the third situation, visitors softened their expression and tended to interpret the machine’s reaction more as a way of communication between them and the image, an agent who reacts to their gestures. In both cases the haptic feedback was the same, what changed is that in the first situation the only sensory feedback came from the haptic device, and in the second there was an image which reacted correspondingly to their gestures. So, a second sensory feedback cue helps in the understanding of the interaction. People were no longer confronting the machine, but rather cooperating with it.

“Anytime I move, it’s full of tiny different sounds, as if it was a language.” (Visitor 2)

“Actually, it’s a response to the gesture I make.” (Visitor 2, talking about what happens on the screen)

“Without the image, well it’s true that I feel something but, there, on the screen, I’m conscious that there is something facing me.” (Visitor 6)

“What is curious is that, suddenly, I feel like there is someone else who wasn’t there [...] Until now, I had the feeling that I was in a kind of dialog and now... now we are three.” (Visitor 1)

Finally, in the forth session (blurry visualization, soft contacts), the feeling of being in a dialog persists but this time the entity is considered less reactive:

“I think it was given some anesthetic.” (Visitor 3)

Conclusions

Our method has proved to be a valuable way to collect rich information about the visitors’ experience, providing insights into the sensory-cognitive process. We identified three dimensions that characterize creative interaction with the virtual scene depending on the degree of multisensoriality. Enactive situations awake embodied memories in order to create the adequate conditions to translate the virtual experience to an embodied one. Developing haptic feedback enables spatial representations of the virtual world and subsequently helps in building a better coupling of action-interaction, a necessary condition for creative interaction. Finally, the that fact to recognize the machine as an equal co-player by attributing it agency couldn’t conclude better our hypothesis for creative instrumentality in virtual artistic environments.

References and Notes: 

All quotations in the paper were translated from French by the authors.

Acknowledgments

This works has been supported by French National Agency of Research (ANR-08-CREA-031).

Credits of Enacting Digital Matter

Artistic production: A. Luciani, J-L. Florens
Interaction and simulation engineering: J-L. Florens, A. Luciani, C. Cadoz, J. Castet 
Software engineering: N. Castagné

References

 

 

  1. Annie Luciani et al., “Exemplary Enactive Tasks and Associated Technological Bottlenecks,” (paper presented at the 2nd Enactive Workshop, Montreal, 2006).
  2. Claude Cadoz, “Musique, Geste, Technologie,” in Les Nouveaux Gestes de la Musique, eds. Hugues Genevois and Raphaël de Vivo, 47-92 (Marseille: Éditions Parenthèses, 1999).
  3. Claude Cadoz et al., “CORDIS-ANIMA: A Modeling and Simulation System for Sound and Image Synthesis - The General Formalism,” in Computer Music Journal 17, no. 1 (1993): 19-29.
  4. Maxime Houot et al., “Perception of Multiple Moving Objects Through Multisensory-Haptic Interaction: Is Haptic So Evident for Physical Object Perception?” in Proceedings of the 3rd International Conference on Enactive Interfaces, Montpellier, November 2006, 154-155.
  5. David Prytherch and Bob Jerrard, “Haptics, the Secret Senses; the Covert Nature of the Haptic Senses in Creative Tacit Skills," in Proceedings of EuroHaptics 2003, Dublin, 2003, 384-396.

 

 

  

Don't Anthropomorphize Me Either

A discussion of embodiment, entrainment and agency.

We believe we can control a robot with code and screwdrivers. We believe it is inert matter. We believe it is a creature like us. We believe it is nothing like the complex beings we are. We believe we created it. We believe it is a clever pet that never shits. We believe it obeys us. We believe it is a lifeless machine. We invest it with personality. We divest it of presence.

Author(s)

BACKGROUND

I have been working in digital arts since the late 80s, primarily making interactive works of various kinds. Recently, I worked with robots for the first time, as a member of the collaborative group In Serial: myself, Petra Gemeinboeck, PRINZGAU/podgorschek and Marion Traenkle. We produced an installation comprised of a mechanical mop, a muddy fluid mess and a troupe of robots whose task was to attempt to clean the mess, but who in fact increased it continually.

In our work as In Serial, I was particularly entranced by the interactions between non-humans; the idea that our machinery, robots and mess did not respond to or engage with humans at all, but were energetically entangled with and focused on each other. The mop communicated with the robots via infra-red signals. The physical nature of the installation and its mess impinged upon the robots’ movements. The agitations of the robots released more fluid to the floor. Humans were bystanders.

We were working with pre-made off-the-shelf robots, the iRobot Create, which is a version of a commercially available domestic robotic vacuum cleaner. These were not purpose built for the artwork. They came with their own abilities, tendencies and habits in place, which made working with them, especially in the brutal, wet, sticky scenario we developed, quite tricky.

EMBODIMENT

I understand embodiment in a quantum physics sense – that the particles that form us are no different from and not divided from the particles that make up the rest of the world. There are no real physical boundaries between people, microbes, robots, furniture, food, garbage, gases, fluids, animals or any of the stuff of our world at all. There are only different formations, different dynamic patterns, being lived out in this one seething mass. Embodiment is densities, patterns, collections, constellations, drives and desires. Just as one cannot step into the same river twice, as its water is constantly flowing away and being renewed, the atoms that make up bodies are moving at lightning speeds around relatively vast empty spaces, appearing, disappearing, colliding and rebounding. We breathe in and breathe out billions of atoms with each breath. We incrementally exchange our atoms, cells, organs with each other and with the rest of the world, replacing about 98% of our atoms each year. [1]

In these densities, patterns and flows of exchange, I am interested in where consciousness, intention and agency reside.

Jane Bennett refers to, “[...] encountering the world as a swarm of vibrant materials entering and leaving agentic assemblages.” [2]

My self spills out beyond the boundaries of my skin, given that it is no boundary at all. Energies from elsewhere spill over into the formation I think of as me. Perhaps volition, will and intent are not restricted to humans, given that a human is not a separate discrete entity. Perhaps objects and ideas have as much personal volition as I do. Perhaps ideas are just a kind of ripple or current that moves various particles around within this vast soup of information, energy and matter. Perhaps it is constellations, fluid assemblages of apparently disparate actants, that effect agency, enact drives, ferment thoughts and conduct relationships. Perhaps fields of intention find material expression through whatever comes to hand.

In Media Ecologies Matt Fuller refers to:  

[…] a combination of drives and capacities that, stimulating each other to new realms of potential, produce something that is in virulent excess of the sum of its parts. Indeed such parts can no longer be disassembled; they produce an ecology. Not a whole but a live torrent in time of variegated and combinatorial energy and matter. [3]

[…] the capacities of activity, through sensation and affect possible to each composition, whether organic or not are shaped by what it is, what it connects to, and the dimensions of relationality around it. [4]

The embodiment I am interested in is that surrounding, forming and propelling us as a system, a turbulence, an assemblage, of flesh, thoughts, machines, fluids, robots, programs, ideas, tables, rooms and so on – an embodied agency arising from the shifting, intricate, dynamic arrangements of our particular constellations.

ENTRAINMENT

Entrainment is a term used in physics to mean a coming into phase or sync. Early experiments, back in the 1600s, showed that pendulums set off at different rates will gradually come into phase with each other. 

In geography, entrainment is the process by which sediment becomes part of a fluid flow. In meteorology, it describes a non-turbulent flow being captured by a turbulent one, as can happen when dry environmental air becomes entrained within a moist cloud. In hydrodynamics, it describes one fluid pushing or pulling another along with it. In engineering, it is the entrapment of one substance within another, such as gas in an aerated fluid or tiny objects caught up in smoke. In biology, it can describe physiological rhythms coming into phase with environmental rhythms, as in circadian sleep cycles, or the synchronization of whole organisms to external rhythms.  In new age alternative health gadgetry, it is delivered to our brainwaves via flashing lights and audio pulses. [5]

It is a physical, material, energetic coming into phase, a confluence, a synchronization between inner and outer; between this and that, within beats and flows.

Music articulates our limbs. It directly addresses and shifts the muscles, bones and joints. Our feet begin to tap; our shoulders are swayed by the beat, by the pleasure of repetition, by the force of a tune. We can be swept away and lose ourselves, abandoning thought and self-consciousness, or we can be barely aware of some small dancing of a toe while our mind is importantly solving problems in a concentrated chatter.

Conversely though, the beat might repel, might induce a rebellion of the limbs rather than choreograph them. If I am in the mood for some thrash punk and I walk into a cafe where they are playing swing jazz, it will push me, irritated, back out onto the street while part of me cries uselessly, “But I just want a coffee.” If I am craving the quiet joys of the Tord Gustavsen Trio but the cafe is bellowing Henry Rollins, as much as I love Henry, I will be pushed out the door the same way. I am not in the mood.

The participants in an entrainment need to be just that. The rhythm will not take and the sync will turn to resistance, if the parties are not willing, if they have no affinity. Maybe the geographer’s sediment must have an attraction to the flow of water. Maybe the gas bubbles must want to aerate the engineer’s fluid. Certainly, my limbs must have a tendency, or at least a tolerance, toward the beat that lifts them.

RABBIT / ROBOT

A rabbit moved in with my girlfriend and I a number of years ago and to start with there was constant struggle of wills. Rabbits are not smart enough that you can train them. They simply do what they feel they have to. The rabbit decided to make her home in the living room. We did not want her to. A lack of communication and understanding along with very different intentions and drives produces a brutal and stupid language. She bit us. We barricaded the door. She launched herself at the barricade and scrambled over. We bribed her with carrot. She chewed through our electrical cables.

Over years of frustrations and defeats, we became familiar with her ways, her world-view, and she with ours. The small, determined rhythm of her being and the distracted human over-thought rhythms of ours settled into phase. The arrangement, ecology or constellation found its form. We had an entrainment.

Like the rabbit, these off-the-shelf robots caused us some troubles in our attempts to control and direct. We frustrated the robots – tethered them, trapped them, told them to clean and prevented them from cleaning, perched them on high tables, even took away some of their senses. They disobeyed our programs, leapt off the tables, threw their tire tread and  largely refused to dance to our tune.

Entrainment takes time, attention and proximity. It takes a recognition and appreciation of the force, intention, drives and will of the unique material-intelligent-energetic constellations in play. It takes willingness to participate and respect for the others in the mix.

ANTHROPOMORPHISM

Perhaps our first mistake is to anthropomorphize ourselves; thinking we are separate, cohesive, autonomous beings, in singular command of our thoughts, decisions and actions. In the robot, we see reflections and parallels of our imaginary free-standing, contained, independence in its similar ostensible autonomy, decisions and actions. We then believe we can control the robot with programming and screwdrivers. We believe it is inert matter. We believe it is a little creature like us. We believe it is nothing like the complex creatures we are. We believe, god-like, we created it. We believe it is a manageable, clever pet that never shits. We believe it obeys us. We believe it is a lifeless machine. We invest it with personality. We divest it of presence.

Yet we are not discreet entities and neither are robots. What if the particles we appear to inhabit are propelled by winds and flow forms of other ideas, other material particles and other energies? Perhaps impetus sweeps into us from the robot or beyond. Perhaps the robot’s constellation, its arrangement, has its own intelligence, will and intention. Given that we are not materially separate from each other, nor in any way fixed, perhaps its will and intention animate us at the same time as ours drives them.

Perhaps this is also true for the table, the cigarette butt, the floor, the spilled coffee, the song on the radio, the ideas I read about last night – the other material and immaterial stuff of our world. Robots though, have a level of digital and machinic complexity that allows a more humanly understandable reading, a possibility of communication, collaboration or contest that is not possible with a rock or a sandwich wrapper.

“A touch of anthropomorphism then, can catalyse a sensibility that finds a world filled not with ontologically distinct categories of beings (subjects and objects) but with variously composed materialities that form confederations.” [6]

Perhaps over time and in proximity, with a respectful recognition of vibrant material presence, the particles that seem to form the robot and the particles that seem to form the human, can come into entrainment and leave behind our anthropocentric arrogance first and anthropomorphism second. The exchange of atoms, the currents of ideas, the forces and phenomena of the sea of particles might manifest through some dissonant hum between human and robot fields of formation.

"[...] order is not imposed from above, by mind exerting its will on dumb material forces; it is intrinsic to the self-organising nature of the phenomenal world  itself. When we recognise our participation in its co-arising patterns, we can claim our power to act."  [7]      

References and Notes: 
  1. Deepak Chopra, Quantum Healing:Exploring the Frontiers of Mind Body Medicine (New York: Bantam Books, 1989), 48.
  2. Jane Bennett, Vibrant Matter: A Political Ecology of Things (Durham, NC: Duke University Press, 2010), 107.
  3. Matt Fuller, Media Ecologies: Materialist Energies in Art and Technoculture (Cambridge, MA: The MIT Press, 2005), 173
  4. Ibid., 174.
  5. Wikipedia, "Entertainment," http://en.wikipedia.org/wiki/Entrainment (accessed September 2011).
  6. Jane Bennett, Vibrant Matter: A Political Ecology of Things, 99.
  7. Joanna Macy, Mutual Causality in Buddhism and General Systems Theory: The Dharma of Natural Systems (Albany, NY: State University of New York Press, 1991), xii.
  

Digital Performance in Networked Public Spaces: Situating the Posthuman Subject

This paper analyses how digital performances in networked public spaces situate the posthuman subject through a complex interplay of human and non-human elements, highlighting the importance of embodiment rather than privileging information over matter. Through empirical research on Blast Theory's digital performance A Machine To See With, I attempt to analyse this process closer through a performative account of posthumanism.

Author(s)

INTRODUCTION

Since the late nineteenth and early twentieth centuries, the city has become associated with a machinic organism through the rationalisation of urban space and living patterns and consolidated through major infrastructure networks. The advent of cybernetics through advances in computational processes has extended the machinic metaphor to human life in the form of the posthuman subject, where information is assumed to flow freely between computational devices and the body, and where embodiment is downplayed or erased altogether.

This paper argues that the experience of the posthuman subject in digital performances in (networked) public spaces is defined through assemblages where agency is distributed between human and nonhuman agents, reenacting embodiment rather than privileging information over matter. It also suggests that the embodied posthuman practices enabled by these performances are better understood through a performative (rather than a representationalist) account that overcomes the inadequacies of a static or advantage point of observation and of assigning agency to individual and self-operating agents.

Through research recently conducted on Blast Theory's digital performance A Machine To See With, I analyse the (posthuman) subject's experience of digital performances in networked public spaces through the embodied practices generated by the complex interplay between (but not limited to) participants, digital devices, networked public space, the performance's narrative and bystanders.

THE MACHINIC CITY AND THE POSTHUMAN SUBJECT

Since the late nineteenth century, the all-encompassing infrastructure networks of modern urban planning enabled what Steven Graham and Simon Marvin defined as the "[binding of] the metropolis into a functioning 'machine' or 'organism'". This process is famously illustrated by Haussman's nineteenth century ‘modernisation’ of Paris, described by Chaoy as an attempt to "regularise the disordered city [and] disentangle it from its dross, the sediment of past and present failures" [1].

In The Metropolis and Mental Life, George Simmel exposed the objectification of life through the rationalisation of living conditions in the emerging metropolitan areas of the early twentieth century, arguing that "modern mind has become more and more calculating" and the world becomes an "arithmetic problem". Simmel argued that the "calculative exactness of practical life" is the outcome of the desire to "fix every part of the world by mathematical formulas". For Simmel, this desire was intrinsically related to the practices of the metropolitan life, with its "punctual integration of all activities and mutual relations into a stable and impersonal time schedule" [2].

In Simmel's account we can identify the emergence of a posthuman subject, merging the machinic processes of the city and the 'calculating mind' of the subject. This link was highlighted by the Futurist movement in Italy, which emerged a few years after Simmel wrote The Metropolis and Mental Life. In The Manifesto of Futurist Mechanical Art (from 1922), the Futurists proclaimed that: "we too are machines, we too are mechanized by the atmosphere that we breathe...This is the new necessity and the basis of the new aesthetic" [3].

According to Steve Dixon, the aesthetics of Italian Futurist performance theory and practice between 1909 have been highly influential in contemporary digital performance philosophies and aesthetic practices [4]. However, he sees a significant difference in approach between both. Dixon argues that the Futurist's glorification of the machine and their focus on embodied interactive practices and audience collaboration allowed "human excitement [to be] projected outward". In the 1915 Futurist Scenography manifesto, Enrico Prampolini argues that: "[Futurist Theatre] is alone in seeking the audience's collaboration. It doesn't remain static like a stupid voyeur, but joins noisily in the action [...] communicating with the actors". In contrast, Dixon argues that contemporary digital performance directs human excitement and creativity inward and into small screens, in what he defines as the "introversion of the computer paradigm" [5].

We must examine Dixon's argument against what Malcolm McCullough describes as a "paradigm shift from cyberspace to pervasive computing", where digital technology "pours out beyond the screen, into our messy places, under our laws of physics" [6]. This process brings computers into the messy, noisy and unpredictable 'real' world. The rising trend of internet-enabled mobile phones (also known as 'smart' phones) is perhaps the most visible reminder of this shift, which is also supported by: gesture-controlled videogames consoles, computer tablet devices and RFID (radio-frequency identification) tags, which attach readable digital information to objects.

Therefore, rather than taking Dixon's account of the 'introversion of the computer paradigm' at face value, we must test it empirically against the innovative practices that digital performance enables through embodied practices. These involve a constant dialogue between participants, digital mobile devices and the surrounding environment, mixing virtual and physical environments and screens and the outside world. In contrast to the Futurist's extrovert approach of provoking their audiences through physical reactions, contemporary digital performances such as Blast Theory's A Machine To See With provoke and challenge the audience through much subtler techniques, such as automated phone messages directed to their mobile phones. While in this case the techniques and actions might be internalised, the participant's are by no means less challenged than in the Futurist's plays: they must deal with these messages while navigating the 'messiness' of urban public space, eventually generating unusual or uncanny situations that must be dealt with in a public arena. Therefore, the performative experience of the participant is neither lost nor diminished, and is perhaps more challenged than in the Futurist's collaborative theatre plays.

TOWARDS AN EMBODIED AND PERFORMATIVE POSTHUMANISM

An embodied and performative posthumanism challenges the desire to control and/or predict the body's responses through normative and synchronised processes. Performing is always already a transformative, iterative and unpredictable process and highly context dependent. While modern urban planning attempted to control the body by standardising and synchronising the flows of infrastructure networks and the circulation of people through the city (as in Haussman's 'disentangling of the city'), the cybernetic movement emerging after the Second World War envisioned the control of the body through the control of flows of information.

As Katherine Hayles reminds us, the construction of the cyborg as a (posthuman) "technological artifact and cultural icon" was supported by the "conception of information as a (disembodied) entity" where "protein and silicon operate as a single system" and information flows are free from physical constraints. Yet Hayles points out that information is "always instantiated in a medium". She argues that in cybernetic posthuman accounts, the body is no longer identified with the self; it becomes a universal object "for control and mastery rather than [...] an intrinsic part of the self" [7].

Defining the body as a universal object entails that it is "primarily, if not entirely, a linguistic and discursive construction" rather than a performative body. Therefore we must distinguish body from embodiment to understand how the linguistic domain has attempted to control the body. According to Hayles while body is constructed through normative assumptions that define a stable and normative set of criteria, "[...] embodiment is contextual, enmeshed within the specifics of place, time, physiology, and culture, which together compose enactment" [8]. Therefore, embodiment denies the possibility of reducing the subject to linguistic interpretations. In her defense of a posthumanist performativity, Karen Barad argues that "language has been granted too much power" and that "the only thing that doesn't seem to matter anymore is matter" [9].

Barad proposes a posthumanist notion of performativity that incorporates both "material and discursive, social and scientific, human and nonhuman, and natural and cultural factors" [10]. However, she argues that this is only possible if "agency is not aligned with human intentionality or subjectivity" [11]. Therefore, posthumanist performativity recognises that it is unfeasible (and possibly undesirable) to resist the impact of machines in our environment and our own bodies, while at the same time acknowledging the importance of embodiment (or body in context) against the linguistic and normative construction of the body. This is enabled by a posthuman subject that, in Hayles' words, constitutes "a dynamic partnership between humans and intelligent machines [that] replaces the liberal humanist subject's manifest destiny to dominate and control nature" [12].

DEFINING DIGITAL PERFORMANCE IN NETWORKED PUBLIC SPACES

Digital performances in networked public spaces foreground the embodied posthuman subject envisioned by Hayles. The term digital performance encompasses works where both embodiment and electronic flows converge. It also avoids the limitations of using categories such as locative media, or digital interactive installations, which suggest a focus on a specific (and stable) technology or infrastructure.

I employ the term networked public spaces to describe the convergence of (private and public) urban and electronic flows. Public space cannot be reduced to a fixed arena where the 'public sphere' is enacted in an orderly manner oras many contemporary social studies suggestportrayed as 'dead space' [13]. Public space has always already been networked, however the advent of digital technologies has foregrounded and accelerated this process.

Mobile phones in particular have enabled embodied posthuman practices that support performativity in public spaces through its multiple capabilities: phone calls, SMS (short message service), mobile Internet and locative media applications. These have generated high hopes of a renaissance of the 'public sphere' and of public space, however these arguments remain speculative and largely untested.

The rise of locative media as "the next big thing" was accompanied, as Tuters and Varnelis point out, by practitioner's claims that "it can reconfigure our everyday life [...] by renewing our sense of place in the world". Yet at the same time, locative media has been criticised by many theorists for its apolitical nature and dependence on technology: while Andreas Broeckman accused it of being the “avant-garde of the ‘society of control’”, Coco Fusco argued that artists have substituted an "abstract connectedness for any real engagement with people in other places or even in their own locale" [14].

TOWARDS A PERFORMATIVE ACCOUNT OF THE POSTHUMAN SUBJECT

Such contradictory accounts highlight the importance of understanding embodied practices to situate the posthuman subject's experience in digital performances in networked public spaces. We must take into account: the subject's prior experiences of (networked) and everyday media practices through their own social and cultural contexts; the technologies involved, which are subject to failures and misunderstandings; and the unpredictability of the networked public space with its complex assemblages of bystanders,weather and mobility patterns, urban furniture, traffic flows and other participants.

A performative account of the posthuman subject in digital performance enables a non-linear narrative that "[articulates] the posthuman as a technical-cultural concept" and refutes metanarratives about "the transformation of the human into a disembodied posthuman" [15]. As Barad points out, performativity "[shifts] the focus from linguistic representations to discursive practices" [16]. This is evident in digital performances such as A Machine To See With: while it is based on a linear narrative with a clear chain of pre-scripted events, it is reshaped by the 'performance' of unpredictable assemblages of embodied practices.

The audience of such performances presents another complicating factor. While in traditional theatre plays (including the avant-garde performances of the Futurists), the audience was allocated a fixed areaand expected to react to the play through predictable patterns of engagementin performances such as A Machine To See With, the audience is not only fully mobile, but performing the roles of both the actors and spectators. Therefore, despite the narrative being pre-scripted, it is impossible to predict participants' reactions. This challenges the understanding of these events and highlights the need for new methods of observation and analysis.

Barad suggests a shift in focus from representationalist understandings of events towards an active approach that reinforces the inseparable link between "observed object" and "agencies of observation", denying the possibility of a static or advantage point of observation. As the posthuman subject performs, the observer of such events must also 'perform'. In her view, discursive practices must producerather than simply describe"the 'subjects' and 'objects' of knowledge practices" [17]. While Barad's framework proposes a posthumanist account that must be 'performed', it also questions the human/nonhuman dichotomy and the privileging of human agency as the main trigger of events. A performative framework is particularly suited for understanding the embodied practices of the posthuman subject enabled by digital performances in networked public spaces.

In the following section, I discuss my attempt at performing observation during my field research on Blast Theory's digital performance A Machine To See With in Brighton during September 2011 towards an understanding of the posthuman subject's experience in digital performance in networked public spaces.

CASE STUDY: BLAST THEORY'S A MACHINE TO SEE WITH

"We needed to know whether you are a person who could step through a door and become someone completely different, and now we know. Your eyes are machines to see with and I am a machine to see with. This film is now yours. Is this the ending you want?" [18].

Participants are confronted with the quote above at the end of A Machine To See With, a digital performance by Blast Theory that reflects on our posthuman nature and the influence of machinic processes on our everyday lives. Blast Theory is an artist collaborative led my Matt Adams, Ju Row Farr and Nick Tandavanitj based in Brighton that has been creating innovative and challenging performances for the last twenty years. They describe their work as "explor[ing] interactivity and the social and political aspects of technology" and using "performance, installation, video, mobile and online technologies to ask questions about the ideologies present in the information that surrounds us" [19].

A Machine To See With invites participants to take part in a simulated bank heist that involves walking through public space while following pre-scripted instructions relayed by an automated phone system to their own mobile phones. Six participants start together following different routes that include a stopover in a public toilet (to answer questions given by the system while waiting inside a cubicle) and eventually converge onto the roof of a private (and usually busy) car park where they are told to enter a car. At this stage, one of the other participants is invited to join them in planning an attempted bank heist, which involves, among other things, betrayal, an aborted countdown, an escape route and a pledge to give money to a complete stranger.

While conducting field research on the event during its premiere in Brighton during September 2011 I attempted to employ a performative account of the posthuman subject participant by remaining close to the "observed object"as Barad puts itand using multiple and mobile points and methods of observation [20]. This approach involved: observing participants taking part in the event while making audio notes; taking pictures and videos of key moments; interviewing participants; taking part in the event myself; taking part in preliminary tests conducted by the artists; and attending project development meetings.

My performative observation of A Machine To See With highlighted the importance of embodied practices in reshaping the posthuman subject's experience of the event. Despite the linear narrative script of the automated phone system, the participant's experience was always unique and dependent on several unpredictable factors, such as the mobility of participants, their prior knowledge of Brighton, their interpretation of the messages received, the interference of bystanders, the surrounding urban space and the engagement of other participants.

For example, in one occasion a participant ended up in the wrong car park after failing to understand an instruction given by the system and decided to ask a bystander for help. While that might have been a frustrating event, he identified that moment as a highlight of his experience: while he hid for twenty minutes behind a bin in the wrong car park (the phone system told him to be discrete) waiting for a car that never turned up (which was stationed in the correct car park), he observed in the distance how the incoming sea fog (a nonhuman agent) gently enveloped a nearby building. He described it as the highlight of his experience―an event that was triggered by a failure in the relay of information.

The reality of the everyday life of Brighton constantly infiltrated the linearity of the pre-scripted narrative, challenging the perception and experience of participants: unexpected clouds of fog suddenly enveloping the car park; desperate bystanders knocking furiously on the public toilets' cubicles where participants were present; curious teenagers provoking participants as they exited the car; close calls with the passing traffic. Participants reacted very differently to these different experiences of embodiment. For example, while resorting to the escape route after the aborted countdown during the bank heist, some participants ran promptly, while others calmly walked away.

While employing a performative account allowed me to gain several insights into the experience of the posthuman subject taking part in the performance, it also highlighted the challenges of this approach: the difficulty of making notes and following the observed subject while moving through the public space of the city; identifying ideal points of view along the way while trying to remain 'invisible' to participants; dealing with the suspicion of bystanders; and losing track of late, absent or stealth participants.

CONCLUSION

A performative account of digital performances in networked public spaces enables a better understanding of the experience of the posthuman subject participant through the embodied practices that are triggered by assemblages of human and nonhuman agents. Such events are better understood through a methodological approach where observed object and agencies of observation are interlinked, and where embodiment plays an important role in both the participant's and the observer's experience.

Although employing a performative account exposes the observer to similar difficulties encountered by participantswhich have to be dealt with dynamicallyit allows for a dynamic approach that avoids the pitfalls of narrow descriptive accounts based on static or advantage points of observation and of assigning agency exclusively to individual agents.

A Machine To See With highlights the importance of embodiment in the posthuman subject's experience of digital performances in networked public spaces against a linguistic and normative construction of the body where information is privileged over matter, enabling emerging forms of embodied interactive practices and audience collaboration.

References and Notes: 

1 Graham and Marvin, Splintering Urbanism: Networked Infrastructures, Technological Mobilities and the Urban Condition (London: Routledge, 2001), 53 – 55.
2 Wolff,
The Sociology of Georg Simmel (New York: The Free Press, 1950), 412–413.
3 Ibid., 63.
4 Dixon,
Digital Performance: A History of New Media in Theater, Dance, Performance Art, and Installation (Cambridge: MIT Press, 2007), 9.
5 Ibid., 58–64.
6 McCullough,
Digital Ground (Cambridge, Mass., The MIT Press, 2004), 9.
7 Hayles,
How We Became Posthuman: Virtual Bodies in Cybernetics, Literature, and Informatics (Chicago; London: The University of Chicago Press, 1999), 2–13.
8 Ibid., 192–196 .
9 Barad, "Posthumanist Performativity: Toward an Understanding of How Matter Comes to Matter", Signs: Journal of Women in Culture and Society, 28 (3) (2006), 801.
10
Ibid., 808.
11 Ibid., 826.
12
Hayles, 288.
13 see for example Richard Sennett's
The Fall of the Public Man and Marc Augé's Non-Places: Introduction to an Anthropology of Supermodernity
14 Tuters and Varnelis, "Beyond Locative Media: Giving Shape to the Internet of Things", Leonardo, 39 (4) (2006), 358360.
15 Ibid., 22.
16
Barad, 807.
17 Barad, 814
819.
18 Blasttheory "A Machine To See With, Banff",
Youtube, http://www.youtube.com/watch?v=cD26y4ncDe4
19 Blast Theory, "Biography",
http://www.blasttheory.co.uk/bt/documents/Blast_Theory_Biography.pdf
20 Barad, 814.

  

Material matters: machine agency and performativity

This paper explores new forms of entanglement between human and nonhuman agents. In considering the performative potential of intelligent machine agents, we are interested in shifting the focus from representational issues to questions of agency and materiality. The investigation revolves around the authors’ robotic installation “Zwischenräume”. 

 

Author(s)

In early Artificial Intelligence approaches, robots sensed their environment, built complete internal models using the sensed data, constructed plans based on those models, and acted to execute their plans. Even though they acted in the world, the world they ‘conceived’ and acted upon was a separate, disembodied reality. Contemporary approaches, in contrast, aim for intelligence that emerges from interacting with the world, thus emphasizing situatedness and embodiment (Brooks 1991, Harvey 2000). The agencies performed by these ‘intelligent’ machines evolve based on the dynamics of their material context. It is only when robotic agents are coupled with an environment, that, according to Beers, their potential to act is realized through the agent’s behaviour in that environment (1995). This is a starting point for considering ecologies that entangle human and nonhuman agents through embodied experience of a shared environment. From a posthuman point of view, embodiment is always contextual and specific; agency is materially enacted and distributed across bodies, rather than located within (see Hayles 1999, Barad 2003, Bennett 2010). Without disregarding their differences, both human and nonhuman agents adapt and know not by observing from the outside, but because they act as part of the world (Barad 2003).

This paper explores new forms of entanglement between agents, human and nonhuman, and probes into their performative potential. Our investigation seeks to set up a conversation between disciplines by looking at the potential of machine agency through the lens of materialist performativity. The notion of the performative here refers to the productive and, at the same time, destabilizing enactment of agency as agents engage with their environment. In considering the performative potential of intelligent machine agents, we are interested in shifting the focus from representational issues to questions of agency and materiality. First, we will discuss embodiment and agency as they are applied in the Dynamical Systems approach to robotics and conceptualized in feminist materialism. The investigation of how, together, these two can open up a third lens through which to look at the performative potential of machine agency will revolve around the authors’ interdisciplinary robotic practice and their work Zwischenräume: a machine-augmented performance environment, which embeds a group of autonomous robots into the architectural fabric of our environment. 

The Dynamical Systems view of agency is based on the observation that “animals are endowed with nervous systems whose dynamics are such that, when coupled with the dynamics of their bodies and environments, these animals can engage in the patterns of behavior necessary for their survival” (Beer & Gallagher 1992). Artificial Intelligence inspired by this view degrades intelligence “in favour of the concept of adaptive behaviour” (Harvey 2000). The lived phenomenal experience of knowing-how outplays the information processing of knowing-that. “Treating an agent —creature, human or robot —as a dynamical system coupled with its environment through sensors and motors, inputs and outputs, leads to a metaphor of agents being perturbed in their dynamics through this coupling”. This contrasts the traditional AI approach, according to which agents are “computing appropriate outputs from their inputs” (Harvey 2000). The metaphor resonates with Varela’s co-evolution between a system and its environment or another system: both evolve through mutual perturbations, setting off a trajectory of mutual adaptations to compensate for the external perturbances. The two structurally coupled systems “have an interlocked history of structural transformations, selecting each other’s trajectories” (Varela, 1979).

As a general formalism the Dynamical Systems’ perspective can be applied to computational systems as well as non-cognitive and non-computational physical systems. Its potential to straddle the Cartesian boundaries between mind, body, and the environment (Clark 1998) opens up a path into thinking across human and nonhuman agential capacities.

Looked at from a posthumanist point of view, embodiment “always is contextual, enmeshed within the specifics of place, time, physiology and culture, which together compose enactment” (Hayles 1999). It aligns with Varela’s biologist view, where experience comes from having a body that always is embedded in an extensive biological, psychological and cultural context (Varela et al: 1991). Agency is a product of this process of enactment, or ‘enaction’, rather than a ‘virtue’ that can be possessed or programmed. In Karen Barad’s performative account, agency is “a matter of intra-acting; it is an enactment, not something that someone or some thing has” (2003). The becoming of agencies and bodies (matter) is mutually entangled—agency is enacted through the dynamic encounter of bodies, while, at the same time, bodies are produced and transformed in this “congealing of agency” (Barad 2003). Similar to the Dynamical Systems view, these material enactments may involve humans or nonhumans, however the materialist feminist perspective challenges not only Cartesian objectivity but unsettles a range of ontological boundaries, deeply ingrained in the Cartesian tradition of modern epistemology, such as human–nonhuman, culture–nature and social–scientific.

“The separation of epistemology from ontology is a reverberation of a metaphysics that assumes an inherent difference between human and nonhuman, subject and object, mind and body, matter and discourse. Onto-epistem-ology – the study of practices of knowing in being – is probably a better way to think about the kind of understandings that are needed to come to terms with how specific intra-actions matter” (Barad 2003).

Our installation work Zwischenräume (Interstitial Spaces) is concerned with the intimate complicities that connect us with the machinic ecologies we create. It develops an unusual concoction of walls, curious robotic agents and surveillance technology to explore the performative potential of the unfolding material pluralogue. The charged terrain of the wall becomes the site for this unusual material encounter, playing out the co-dependant agential relationship between humans, machines and their environment. The installation couples curious robotic agents with our built environment by embedding robots into the architectural skin, sandwiched between the existing wall and a temporary wall that resembles it. Each machine agent is equipped with a motorised hammer, a surveillance camera, and a microphone to interact with its environment and network with the other machines. The hammer is not only used by the robots to pierce holes for the camera eye to see what’s going on outside but also for communicating amongst the collective. The walls’ and the machines’ anatomy intertwine, turning the wall into the machine’s brittle skin, and the machine into the wall’s kinetic organs. The wall-body is the milieu through which the machines intervene and develop and express their desires through knocking, chipping, and punching holes, and adapting. 

The machine-augmented environment embodies the agents in the terrain they survey; they are programmed to be curious and are thus intrinsically motivated to explore and transform their environment. The means of marking and exploring have been adopted from two military references, that of urban combat and visual intelligence. Movements, colours and faces are processed to create an adaptive model of the surrounds that allows the robotic agents to expect learned behaviours and proactively intervene. To these curious machines, learning and adapting are not goal driven but evolve based on what they discover and interpret as ‘interesting’. The intrinsic desire to learn about the world directs both the system’s gaze and its actions, resulting in a feedback process that increases the complexity of the environment relative to the perceptual abilities of the agent. Literally carving a trace of their curiosity into the wall, their desire to look is acted out in the open and manifests materially. They also communicate their state of arousal physically by re-sculpting their environment, rather than using an electronic network. Equipped with contact microphones to listen into the wall and sense the knocking of other robots, they use different knocking signals to rhythmically express excitement (high levels of sustained interest) or frustration (low levels of interest for a certain period of time). The embodied agents act and adapt through their intra-actions with their surrounds; shaping what they ‘desire’ to create or perform. At the same time, they become and are stimulated by what they shape. 

When shown for the first time (figure 1 and 2), the gallery space was bound by glass walls, requiring us to not only stage the intervention but also the environment to be intervened with. The transparent space was turned inside out to present a private, cosy, living room scene oriented towards the public space outside the gallery. The machinery attached to the temporary walls inside the gallery transformed the living room scene into a capricious voyeur that drastically transformed the space over the course of three weeks. While the implicit theme of surveillance and voyeurism is not the focus of this paper, it is worth noting that the enactment and embodiment of the power of the machinic gaze was at the heart of Zwischenräume’s conceptual development. Yet while the voyeurism enacted by Zwischenräume’s robotic actors relies on visual intelligence, the work defies military logic of suspicious behaviour and rather promotes the machines’ capability to seek difference for the sake of being different (Gemeinboeck & Saunders 2011). The machines’ motivation to seek difference for the sake of difference, rather than for the purpose of othering that which is different, sets the tone for an alternative investigation into the politics of surveillance and its material affect. It isn’t as simple as incriminating or trivialising the machine. The mingling of agencies and materialities in our installation and the way in which the audience is implicitly implicated, rather than invited to control the course of events, intimates the heterogeneous nature of surveillance. Thus, it is not the spectacle of the intervening machinery that we are interested in, but rather the spectacle of the mutual processes this intervention unfolds as it foregrounds the material ecology of this machine augmented environment and its ongoing becoming. 

The structural coupling of machinic agents and our built environment politicizes the matter of material agency and aims to foreground the performative potential of Dynamical Systems. Looked at from an expanded, ecological perspective, the work enacts what Jane Bennett describes as “an encounter between ontologically diverse agents, some human, some not, though all are thoroughly material” (2010). Zwischenräume’s drama features the encounter of two nonhuman agents, both human-made artefacts, one imbued with (artificial) intelligence and an ability to be proactive and the other designed to be inert and deprived of any vital qualities. We were interested in the co-dependent nature of this assemblage of forces, and the affective relationship through which it evolves. The structural coupling of machine and environment sets in motion their path of material becoming; both evolve through continual adaptations to compensate for the mutual perturbations. The process opens up the transversality of assemblages that owe their agential forces to the vitality of the materialities and dynamic spatio-temporal relations that constitute them (Bennett 2010). It’s a performance that always unfolds in the present, without the comfort of rehearsal. Rather, as Matthew Fuller argues, “the process of becoming that is machinic heterogenesis has no plot, as in story or territory, only a “middle,” an ongoingness: It cannot be turned into a standard object, it must be done” (2005). 

The new assemblage not only challenges the structural integrity of the wall but also intervenes into the socio-politics of our third skin, laying open its vulnerabilities to continuous perforation. While perhaps the machine-wall couple seems purely destructive at first, together they unsettle the politics of the wall and turn it into a negotiable playground. Looked at as actors, they have much in common: both are as much technological as they are cultural; each models nature. The dynamical system underlying the first is inspired by the observation of the animal, while the latter renders the cave efficient, mobile, and mass producible. Both are ambiguous with regards to their acting: the machine empowers some and deprives others; the wall includes some and excludes others. Both extend the human: the machine is an extension of both, mind and body, while the wall is our extended skin. And yet, the dynamic agential forces of the machine are much closer to the human. We are more empathetic to, and at the same time, threatened by them. We (Westerners) cannot perceive the vital qualities of the wall, whereas the embedded machines can render it alive. The performativity of the machinic wall is further complicated by the machines’ autonomy; the self-motivated act of destructing the wall, the self-motivated act of looking. This is where it gets uncanny. It’s ok if the machines act on our behalf, and we control the machine that deconstructs the wall or if human governance drives the machine’s eye. Yet intrinsically motivated agents exhibit a higher degree of autonomy than agents motivated by an external human agent. The meaning of agency changes drastically, once the human actor can no longer control the human-machine-environment coupling. The discomfort of this shift, of course, reaffirms the segregation and hierarchisation of these actors. Neither the machines nor the wall exist outside the realm of human culture, and the autonomy of the machine is simply stretching its capacity to extend the human further: its intrinsic motivation, even if artificial and perhaps alien, is still modeled by a human agent, as is the design of its material embodiment.

This stretching quality was exactly what we aimed for with Zwischenräume, allowing us to stretch into the environment, to intra-act, not as the isolated and superior human but as part of a bigger assemblage. Coupling autonomously performing agents with our built environment opens up a space for Barad’s ‘congealing of agency’ (2003) where the different agential forces not only co-evolve but potentially conspire and perform together. All actors involved are vital players, entangled in a complicated web of connections and specificities. While non-anthropomorphic, the material embodiment of the machines’ cognitive processes and desires places them in a realm, where we (humans) can share and bodily experience them. The unfolding relationship between audience, machines and other matter, materialises a slice of our machinic ecology and makes tangible our position within.

References and Notes: 

Barad, K “Posthumanist Performativity: Toward an Understanding of How Matter Comes to Matter,” Signs: Journal of Women in Culture and Society 28:3 (2003), 801–831

Beer, RD “Computational and Dynamical Languages for Autonomous Agents,” in Mind as Motion, ed. R Port and T van Gelder (MIT Press, 1995)  

Beer, RD & Gallagher, JC “Evolving dynamic neural networks for adaptive behavior,” Adaptive Behavior 1:1 (1992), 91-122

Bennett, J Vibrant Matter: A Political Ecology of Things (Duke University Press, 2010)

Brooks, RA “Intelligence without representation,” Artificial Intelligence 47 (1991), 139–159

Clark, A Being There: Putting Brain, Body, and World Together Again (MIT Press, 1998)

Fuller, M Media Ecologies: Materialist Energies in Technoculture (MIT Press, 2005)

Gemeinboeck, P & Saunders, R “Zwischenräume: The Machine as Voyeur,” Proceedings, Transdisciplinary Imaging at the Intersections between Art, Science and Culture (2011), 62-70

Harvey, I “Robotics: Philosophy of Mind using a Screwdriver,” in Evolutionary Robotics: From Intelligent Robots to Artificial Life, Vol. III, ed. T Gomi (AAI Books, 2000), 207-230

Hayles, NK How We Became Posthuman: Virtual Bodies in Cybernetics, Literature, and Informatics (University of Chicago Press, 1999)

Varela, FJ Principles of Biological Autonomy (Elsevier, 1979)

Varela, FJ, Thompson, E and Rosch, E The Embodied Mind (MIT Press, 1991)

  

Below the Belt – Participant Experience in a Breath Controlled Interactive Artwork

Contemporary theories of embodiment and affect are explored in relation to the breath-responsive interactive installation, Below the Belt. The artwork uses bio-sensed data to measure the breathing patterns of participants in an attempt to uncover the relationship between breath and emotion. The artwork forms the test-bed in an examination of how emotion and breath are considered in the construction of experience across bodily and social realms.

Author(s)

Introduction

Despite the development of body-responsive interactive art the vast majority of artists and researchers working within this genre have explored embodiment from an external perspective, privileging the senses of the outer body or the proprioceptive sense of the body in space. Few have examined perceptions of the inner body. Where the inner senses have been used to drive the work the participant’s focus has often been directed towards self-reflection and their affecting relationship with the social sphere has been ignored. When the affective nature of social interactions is considered the works have commonly bestowed a pacifying and subdued meditative tone.

The interactive installation Below the Belt is a breath-responsive artwork exploring how the aesthetic experience of engagement with breathing and emotion can promote an awareness of embodiment. It amplifies the breathing patterns of participants to extend their expressive and perceptual awareness and their connections to the inner senses. Often we only become aware of these senses when we become unwell. This can lead us to alienate the body further in an attempt to dissociate ourselves from the discomfort, and so detachment becomes a habit. This work examines how breath awareness can subvert our customary tendency to favour the outer body above the inner body by magnifying our perception of our inner world. [1]

Below the Belt provokes the participant to explore the broader relationship between their breath, emotions and the social realm. It stretches their focus beyond their fleshy boundary to bring awareness to the subtleties of the affective relationship between these bodily responses and their social interactions. The primary motivation of this work is to take the lived experience of breathing out of the private realm and into the public. It attempts to understand participants’ first hand accounts of their experience using a phenomenological approach as a way of focusing on the whole bodily being.

In this paper I briefly explore theories of embodiment and affect to assess the role that emotion and breath play in bodily and social domains. I continue by positioning this work in relation to other breath-focused interactive artworks before providing some context to the work itself. The paper concludes with reflections on the experiences of a selection of the participants who encountered the work and on how these findings are informing my ongoing research in this area.

Embodiment and Affect

Contemporary approaches to the study of embodiment and affect theory sustain inquiry into lived experience, subjective perspectives and meaning making. The existentialist phenomenological tradition questions the dualistic nature of Descartian thought that bifurcates mind and body. In Phenomenology of Perception, Maurice Merleau-Ponty explained perception as bodily experience where the body is subject distinguishing between the objective, physiological entity of the body and the phenomenal body that we experience the world through. [2] However the term embodiment is still often misused to describe the body’s role in cognition while still maintaining the Cartesian paradigm. I believe embodiment is the very nature of being and the primacy of the body in constructing experience.

Medical anthropologist Margot Lyon suggests that because the experience of embodiment is accentuated when we are emotionally present in the world, we can study embodiment by studying emotion or a bodily capacity linked to emotion, such as breath. [3] The respiratory function is related to feeling in part because of the nerve fibres it shares with the autonomic nervous system, which plays an important part in emotion. For example, slow, deep breathing can regulate the functioning of the autonomic nervous system by increasing parasympathetic activity (rest and digest responses), effectively bringing the involuntary autonomous nervous system into the realm of the voluntary. This argument does not attempt to diminish emotion to the function or arousal of breath. It acknowledges that behaviour cannot be reduced to physiological processes and makes no attempt to match feelings with particular breathing patterns.

It should be noted that whilst embodiment and affect theory has only recently been articulated by Western scholars it has of course been studied through practice based research for many centuries by philosophers of Eastern traditions who have used breathing practices and meditative states to reflect on states of consciousness.

Body-responsive Interactive Art

Body-responsive interactive art is entirely reliant on the actions of the participant, which becomes the instrument of communication. The participant responds to the work through their embodied reactions, the work reinterprets this feedback and so the dialogue unfolds. Their aesthetic experience is defined not in terms of beauty but rather through their experience of this interaction. It is this experience that creates meaning for the work. Body-focused interactive artworks provide a unique platform to engage in dialogical exploratory practices.

As biofeedback technology has emerged, so have creative ways of engaging with digital breath-focused interactive artworks. Below the Belt is situated amongst works that are activated by breath such as George Khut’s work Cardiomorphologies v1. [4] My research looks beyond the self-reflective aspects of this focus to the broader affect of breath on the social realm. Works being created in this area include Christa Sommerer and Laurent Mignonneau’s Mobile Feelings. [5] Also Thecla Shiphorst’s wearable body architecture Whisper, which uses breath to explore human interactions in the social domain. [6] Where as most artists working in this area have focused on the meditative aspects of reflections on their work I am interested in provoking a broader range of responses from participants to allow them to more fully explore the complex relationship between their breath and emotions. The artwork Below the Belt provides the ‘black box’ in which to examine these theories.

Below the Belt

Below The Belt is an interactive installation that affords participants the space to feel the affective nature of emotion, and its role in human experience, through the prism of breath. It attempts to re-embody interaction within video installations, to amplify and extend the bodily experience of participants. The installation uses affective computing and the breath of participants as the vehicle to explore embodied subjectivity. It places embodiment theory and the affecting influence of emotion and breath centre stage, directing participant attention to their breath to explore how it is affected by the instruction. The participant is presented with a playful environment to experiment with breathing techniques and an opportunity to cultivate their breathing literacy. Participants interact with breathing coaching Tony, made visible through a single channel projection, who guides them to slow down their breath and increase the natural rise and fall of their abdomen.

The installation relies on wearable computing to make Tony’s instruction audible – protective boxing headgear, implanted with wireless headphones, provide an immersive auditory environment blocking out extraneous sound and focusing the participant’s vision on the projected video. A biosensor embedded in a champion title-boxing belt wirelessly relays the degree of stretching to an Arduino microprocessor attached to a laptop. The microprocessor feeds this stream of data to Max/MSP software (Cycling 74), which identifies patterns in the data attributable to the pace of breathing and degree of abdominal movement. The laptop, visible directly under the projected video image, displays the Max patch, the guts of the processing driving the work. This includes a simple graphic representation of the participant’s pattern of breathing.

The first stage of the encounter involved attaching the belt, immediately drawing the participant’s attention to their upper abdomen, the region under measurement. Tony then spends the first minute setting the scene and explaining the rules of engagement. During this time a baseline is calibrated for each participant according to her or his breath patterning. The participant’s overall performance is judged according to deviations from this baseline. Thereafter, at regular intervals the average value for the preceding period is calculated and compared to the participant’s previous results. These differential values are fed to Jitter software which triggers the appropriate video vignette of Tony’s feeback. The work does not attempt to make judgments about the participant’s natural breathing patterns as performance is measured in terms of changes in the pattern of breathing during the encounter.

The breath coach, Tony O'Loughlin, is actually a boxing coach from Elouera-Tony Mundine gym in Sydney. Tony’s antagonistic coaching style starkly contrasts with instruction found in the more popular meditative breathing practices. He takes each participant through five rounds. After each round he provides feedback, often harsh, on their breathing performance for that round, based on the Max patch results, and offers appropriate breathing exercises to improve performance in the next round. At the end of the five rounds the competitor’s overall performance is calculated and Tony proclaims their performance with all the fanfare of a championship bout. Tony’s aggressive motivational style was chosen as a way of provoking a reaction that is at odds with the calm, smooth breathing he demands. The participant has the challenge of obeying instruction to relax and breathe deeply, delivered in a forceful and abrasive style. The natural body response to this harsh feedback for some may be an increase in their nervous system’s sympathetic responses leading to shallower, faster breathing.

Through the metaphor of boxing the work examines the tensions between competitive contact sports and the inward focus of supportive breathing practices. It plays with the constant mediating role that breath plays in the bodily and the social realm. While the participant is sensing and performing their responses they are also differentiating and appreciating the systems interpretations, in the full knowledge that they are being measured. Although the work relies on rhetoric grounded in competition the reality is that each participant is only ever competing against herself or himself, never with each other. The irony is that when participants get caught up in this sense of competition the natural reaction of the body is to retreat to flight or fight mode which produces shallower, faster breath consequently impairing their performance.

Understanding Participant Experience

When John Dewey redefined aesthetic experience, he contended that the work that art does takes place within the entire process of art making. Art is more than the material ‘work of art’, it is the development of an experience and recognises the aesthetic experience in everyday living. Pragmatist aesthetics elevated the experience of the audience as a vital component in completing an artwork. It proposes that to fully assess any work of art the experiences of those who interact with it must be considered and understood. [7] This philosophical position supports the dialogical aspect of my inquiry that seeks to understand the lived experiences of participants interacting with this work.

There is very little empirical research on audience experience of interactive art. [8] Whilst various literature has examined participants’ creative engagement with body-focused interactive artworks these approaches have mainly used third person investigations which rarely examine the physical, emotional and affective experience. Where they have taken into account the quality of experience they have often employed methodologies that take an embodied cognitive approach, which maintains that the mind is split from the body. In addition purely first-person accounts of participant experience can easily be dismissed in academic realms as anecdotal, unless they have been rigorously interrogated. My exploration is motivated not just in attempting to understand the experience of interacting with the artwork but also in seeking to understand the participant’s interoceptive exploration. This area has been neglected, perhaps due to the difficulty of articulating experiential response, yet it remains central to the way in which we interpret and understand an artwork. The phenomenological approach used in this investigation recognises the body as our basic mode of being in the world in terms of both the process and practice of this research.

The relational practice that grounds this work seeks to develop dialogue between the participants, their audience and the artist. Seeking to understand their experiences during their interaction with Below the Belt led me to enter into research-focused dialogues with seven randomly selected participants. Unstructured interviews, conducted immediately following the encounter, used open questions to initiate a conversation about the experience. Participants were asked to recollect their thoughts, feelings, perceptions and sensory memories of the interaction. More specific questions followed, focusing on observed body language, gesticulations and breathing quality.

Reflections

A selection of participant quotes are found below along with a brief description of the themes identified in these responses.

Breathing literacy

The diverse quality of participants’ connection to their breathing is illustrated by the following statements:
S1: “I never think of my breath, it’s just kind of always there”.
S3: “Tony’s advice was at odds with the instructions I was given as a kid with asthma. I used to be told to breath into my chest and not into my belly. That new logic was really hard for me.”

Engagement

Participants chose to engage with the artwork in a variety of ways as described below:
S2: “I got swept up by the competition. I got heavyweight but I found myself quite anxious and tight in the process. Although I fooled the system I feel like I cheated myself”.
S4: “About half way through I could see the graph on the laptop and figured it was my breathing. I hooked on to it for the rest of the time”.

Participant co-experience

The social aspect of this seemingly solitary pursuit was perhaps impacted by the competitive tone of the work. Some took the opportunity to perform to their audience – on a number of occasions the participant, on stepping out of the installation, would announce to their ‘audience’ the title Tony had given them, often in the same flourishing style he had used.
S2: “I was determined to beat my girlfriend – she got super-heavy weight.”

The challenge for both interviewer and participant in understanding these encounters is the limitation of words to describe an embodied experience. This required attentiveness to all the forms of communication used by participants in relaying their experiences. They were encouraged to verbalise their body language and take time to unearth meaning where there was ambiguity in their language. Overall, participants reported an appreciation of the opportunity to have their breath foregrounded in this way.

Some participants struggled to recollect aspects of their encounter. It is intended that future studies employ the video cued recall method to support participant memory. The audiovisual nature of this medium honours the temporal, embodied and emotive nature of the artwork and captures the participant’s body language, gesticulations, breathing quality and tonal subtleties of voice.

References and Notes: 
  1. James Morley, “Inspiration and Expiration,” in Philosophy East and West 51 (2001): 73-83.
  2. Maurice Merleau-Ponty, Phenomenology of Perception, (London: Routledge & Kegan Paul, 1962).
  3. Margot Lyon, “Emotion and embodiment,” in Biocultural approaches to the emotions, ed. Alexander Laban Hinton, 118 (Cambridge: Cambridge University Press, 1999).
  4. George Khut, “Interactive Art as Embodied Enquiry,” in Engage: Interaction, Arts & Audience Experience, eds. Ernest Edmonds, Lizzie Muller, and Deborah Turnbull (Sydney: Creativity and Cognition Studios Press, 2006).
  5. Christa Sommerer and Laurent Mignonneau, “Mobile Feelings wireless communication of heartbeat and breath for mobile art” (paper presented at 14th International Conference on Artificial Reality and Telexistance, Seoul, 2004).
  6. Thecla Shiphorst, "Soft, softer, softly: whispering between the lines,” in ARt & D: research and development in art, eds. Andrew E. Benjamin and Joke Brouwer, 166-176 (Rotterdam: V2_Publishing, 2005).
  7. John Dewey, Art as Experience (New York: Capricorn Books; Paragon, 1959).
  8. Brigid Costello, Lizzie Muller, Shigeki Amitani and Ernest Edmonds, "Understanding the experience of interactive art: Iamascope in Beta_space" (paper presented at Australasian conference on Interactive entertainment, Sydney, 2005).
  

enVella: Making Space Personal

The paper describes enVella, a kinetic dress that moves when triggered by the detection of the wearer’s fear or anxiety. The aim of this project was to investigate the detection of fear with biosensors, and to see whether an enveloping physical transformation can provide a sense of comfort to the wearer.

Author(s)

Introduction

enVella is a kinetic dress which moves when triggered by the detection of the wearer’s fear or anxiety. The upper portion of the dress is encircled with fans, which open and envelop the wearer if her body temperature and heart rate are both rising. The aim of this project was to investigate the detection of fear with biosensors, and to see whether an enveloping physical transformation can provide a sense of comfort to the wearer. The paper will begin with the inspiration behind enVella, discuss its design, and conclude with the technical aspects of the project.

Inspiration

enVella draws inspiration from the natural world specifically with regards to animals’ reactions to fear and the idea of engendering feelings of safety in human beings. An experience everyone has in common is that of being in a mother’s womb. In a warm, enveloped space, one feels safe, warm and protected as if one was inside the mother’s womb. [3] Despite not offering any real protection, we instinctively huddle under a blanket when frightened.

enVella’s motion was inspired by the frill-necked lizard which is capable of erecting a normally-concealed frill from around its neck. This physical transformation makes the lizard appear larger than it is, and so a less appealing prey. [4] By merging this reptilian defense mechanism and the soothing of envelopment, the hope was that this combination would produce a comforting psychological reaction in the wearer during anxious moments.

Design 

In an attempt to recreate the sensation of being enveloped inside the mother’s womb and implementing the frilled-neck lizard’s self-defense behaviour of increasing in size towards wearable technology, this adventurous application explores the concept of embodiment where the dress and wearer engages in a symbiotic relationship. In order for enVella to achieve it’s full potential, the dress must be worn for it to react accordingly with the wearer’s emotional state. Without a pulse, the dress cannot achieve its full potential and without the dress, the wearer will not be able to experience the safety, comfort and protection provided by enVella.

enVella is designed on a white cotton dress attached with four servo-controlled fans. Each fan is constructed of satin fabric folded in half multiple times, and held in place by heavy duty double- sided fusible interfacing. Wooden supports were added onto the ends of each fan to provide motor and anchor support. At the base of each fan, a servo attachment was glued. The fans are triggered by a combination of the two sensor inputs: a heart rate monitor and a temperature sensor. If the wearer’s current heart rate and temperature is greater than a predefined average heart rate and body temperature, the fans will open – beginning at the chest then around the neck in a sequential manner (Fig 1). With the fans in their opened form, heart rate and temperature data continue to be relayed from the sensors to the microcontroller. If both heart rate and temperature return to normal after 20 seconds, the fans will close.

The core element of enVella was inspired by animals with distinctive self defense reactions such as the frilled-neck lizard. Through user studies, few design forms were determined as most effective at enveloping the wearer and embodying the idea of comfort and warmth. The four fans were strategically placed and programmed to optimize the enveloping experience. When a state of anxiety or fear is detected by the microcontroller, the fans will open in smooth succession. Because enVella is a dress designed to comfort the wearer, the decision to have the fans open was to create a sense of division between the wearer and the frightening, threatening entity. This invisible wall creates a psychological barrier between the two and thus alleviating some of the resultant fear. A gradual enclosure of the space around the wearer’s visual field, rather than a sudden one, adds to the comforting quality of the motion. Whereas if the fans were to open rapidly, it would have an opposite effect on our intention of comforting the wearer.

Technical 

Among the technical challenges present in enVella, detecting and identifying fear was the most difficult to justify through biosensors. One challenge was to find a suitable solution to distinguish fear from other strong emotions like anxiety and anger. Fear is generally differentiated from anxiety by the perception of a specific external threat. However, for our purposes, it was not possible to distinguish one from the other as anxiety and fear have almost identical physiological symptoms. Typical physiological responses to fear and anxiety elevated heart rate, increased sweating, and increased blood pressure. Both the input of heart rate and body temperature were chosen for the purpose of this project, as an indication to measure state of fear. Various heart rate monitor circuits have been developed with Arduino using different methods but the majority of implementations use the Polar RMCM01 Heart Rate module. The Polar Heart Rate Monitor Interface (HRMI) was selected given the added difficulty of working with the bare module. The multiple interfaces available in this new module make it very stable, flexible and easy to work with.

Conclusion 

enVella project designers witness the success of the dress and how it achieves envelopment through the natural motion of fans. User feedbacks have been positive, with the majority stating they found the enveloping effect comforting. A few participants raised possible issues with the feasibility of a cloth dress offering sufficient protection, and concerns over providing comfort without actual protection. Participants also mentioned possible discomfort and unease that may arise from users who are claustrophobic. Through academic research, the team on designs inspired by nature and explored various forms before finalizing on fans to create temporary personal space. enVella is not only a dress but is also a concept to assess the research question of creating personal space in a state of fear.

References and Notes: 

  1. G.S. Bedford, “Anti-Predator Tactics from the Frilled Neck lizard Chlamydosaurus kingii,” Journal of the Victorian Herpetelogical Society 6, no 3 (1995): 120- 130.
  2. Gu Chuanhua, The Attachment to Maternal Womb: The Initial Attachment of an Infant (Central China, Huebi Province Key Lab for Human Development and Mental Health, 2011).
  3. James B. Murphy, Willieam E. Lamoreaux, and Charles C. Carpenter, “Threatening Behaviour in the Angle-Headed Dragon, Goniocephalus Dilophus (Reptilia, Lacertilia, Agamidae),” Journal of Herpetology 12, no 4 (1978): 455-460.
  4. Sylvers, Patrick, Jamie Laprarie and Scott Lilienfeld, “Differences Between Trait Fear and Trait Anxiety: Implications for Psychopathology,” Clinical Psychology Review 31, no1: 122–137.
  

Virtual Doppelgängers: Embodiment, Morphogenesis, and Transversal Action -- Panel Introduction

In 1969 Gilles Deleuze theorized the Body without Organs (BwO). The term refers to the virtual dimension of the body likened to the egg as site of embodiment (in Deleuze and Guattari’s Anti-Anti-Oedipus)a set of multiple potentialities and dysfunctional repetitions. In this panel we seek to explore the relations between fleshly bodies and digitized ones as sites of embodiment for our current, informatically energized existences.

Author(s)

In 1969 Gilles Deleuze theorized the “BwO” or Body Without Organs (in The Logic of the Sense, after Antonin Artaud’s origination of the term in 1947). [1] [2] BwO refers to the virtual dimension of the body and its potentials, likened to the egg as the site of embodiment[3] This BwO-as-egg is further described by Deleuze and Guattari as a set or “spatium” of multiple intensities or potentialities.  Deleuze and Guattari’s body without organs is not against organs, per se, but opposed to the organism—the body subjectivized through organization. [4] Moreover, there must remain a connection between the organism and the BwO. D&G: “You have to keep enough of the organism for it to reform each dawn; . . . you have to keep small rations of subjectivity in sufficient quantity to enable you to respond to the dominant reality.” The BwO is a way of transcending the strata of subjectivity and freeing its lines of flight.  

The image of the BwO is one that, for many, has suggested the analogy with the digital or avatar body, clearly a BwO, but only insofar as it is fluid and unsubjectivized and occupies what D&G call the “plane of consistency,” for which the virtual world stands as a figure. Slavoj Zizek even calls Deleuze the “philosopher of the virtual.” [5]

In the spirit of the BwO thus interpreted, then, this panel explores relations between fleshly bodies and digitized ones as sites of embodiment for our current, informatically intensified existences. From Facebook to online games and performances in Second Life, many of us experience various parts of our lives virtually today. But how are these experiences absorbed into our so-called “real life”? 

There have been controversies and supporting studies (esp. concerning virtual games) suggesting that too much virtual mediation is harmful to our “sense of reality” and ability to interact well in society. This idea has been around for a long time as progeny of old, unresolved debates about violence on TV. Clinical studies of violent virtual games are still ongoing and as yet inconclusive, but speculation continues. And many say online socializing, which (like gaming) offers action free of consequences, has encouraged the growth of bullying and even occasions of lynch mob mentality, pointing to phenomena such as the responses to the recent Casey Anthony trial in the US (the young Florida woman charged with murdering her infant daughter). That trial, which used virtual simulations in the courtroom itself, was the subject of obsessive coverage in both TV and social media including animated simulations on YouTube of the crime scene for “virtual jurors.” Media pundits speculated as to whether or not heightened access to visualizations of the case incited members of the general public, who repeatedly mobbed and fought each other to gain entry to limited public courtroom seats, and demonstrated for a guilty verdict, outside. 

In our session, Patrick Lichty will discuss the scientific discovery of mirror neurons, which attach us to others, or the images of others or ourselves, on an autonomic level. Researchers like Jeremy Bailenson (Virtual Human Interaction Lab, Stanford University) and Nick Yee (Palo Alto Research Center) provide evidence for the idea that our conduct as avatars in online worlds has an aftereffect, a “Proteus effect” as Yee calls it, such that our behavior and feelings in real life are adjusted. In a study in which participants were given tall avatars and asked to negotiate tasks with other (shorter) avatars, subjects were then told to negotiate similar tasks with people in real life. The study found that persons experiencing taller avatars negotiated more advantageous results, and this effect also carried through when they negotiated a similar deal face to face—they “acted” taller, in effect. [6]

Bailinson’s group did studies of doppelgängers - specifically, avatars that are built to look exactly like us. In a series of studies, Bailenson and co-researcher, Jesse Fox, at Ohio State, used doppelgängers as theraputic tools. Test subjects who observed their near mirror images exercising and losing weight, for example, were followed after the test and shown to be more inclined to exercise than a control group. Subjects who observed their virtual selves manipulated to look much older, displayed heightened interest in their retirement savings. Studies were, of course, suggestive rather than conclusive.

But similar effects are the focus of simulations like the multi-platform “Always in Season Island” an educational and consciousness-raising project in which (when the Second Life portion of the site is released) visitors will participate in reenactments of lynchings and torture that took place in the American South from the 18th century through the mid 1960s. [7] Participants will see their clothing transformed to period dress, and they will become virtual witnesses. They will also receive information about historical lynchings and be connected to the project’s Twitter or Facebook pages to share their feelings about the SIM. The project aims for responses that show real concern for the scenarios because of the experience of virtually witnessing them and bearing the moral tension such witnessing holds.

According to philosopher and researcher Philip Brey, there are clear ethical issues entrenched in our behavior as virtual selves, and these involve two categories of assumptions; he writes:

According to the argument from moral development, it is wrong to treat virtual humans cruelly because doing so will make it more likely that we will treat real humans cruelly. The reason for this is that the emotions appealed to in the treatment of virtual humans are the same emotions that are appealed to in the treatment of real humans . . . The argument from psychological harm is that third parties may be harmed by the knowledge or observation that people engage in violent, degrading or offensive behavior in single-user VR . . . [8]

Clearly, our evolving abilities to mash up real and virtual existences has both therapeutic and educational potential, but also responsibilities. It holds promise of empowerments—Deleuzian intensities—but also of manipulation and subjectification.

Sherrie Turkle suggests that most of us already live in both virtual and physical realities and our “life mix,” as she terms it; our “multi-lifing,” has become the norm. It can function efficiently or go off the rails, but Turkle finds that the most beneficial doppelgängers - beneficial, that is, for their real operators and overall life mix - are those who participate in “real” online relationships, or situations with consequences. [9]

The session addresses both artworks and theoretical frameworks that engage our replicated bodies, the relations they create, and their transversal effects across multiple platforms and modes of existence. Greg Little will tell us more about the metaphysics of the BwO in the context of avatars. Micha Cárdenas and Elle Mehrman explore how virtual experiences can transform our real-world identities. Stephanie Rothenberg will discuss the mash-up between work and play. And Patrick Lichty discusses affective potential of virtual performance art.

References and Notes: 

  1. G. Deleuze, The Logic of the Sense (New York: Columbia University Press, 1990).
  2. A. Artaud, “Body without Organs,” Pour en Finir avec le Jugement de dieu, Radio Française (Paris, November, 22-29 1947).
  3. G. Deleuze and F. Guattari, Anti-Oedipus (London: Continuum, 2004).
  4. Deleuze and Guattari, A Thousand Plateaus (Minneapolis, MN: University of Minnesota Press, 1987), 158.
  5. S. Zizek, “The Reality of the Virtual,” in Organs Without Bodies, (London: Routledge, 2003).
  6. N. Yee, The Proteus Effect (PhD. diss, Stanford University, 2007): 46-50.
  7. “Always in Season Island” is created by Jacqueline Olive with consultants. See: http://www.alwaysinseasonisland.com/
  8. P. Brey, “Virtual Reality and Computer Simulation,” in Handbook of Information and Computer Ethics, eds. K.Himma and H.Tavani, 361-384 (Hoboken: Wileyand Sons, 2008).
  9. S. Turkle, Alone Together (New York: Basic Books, 2011), 160, 223.
  

Augmented Movement Vision : Moving, Seeing and Sensing

The embodied  Augmented Reality screen has the potential to alter and augment the dimensionality of our perceptual field through the form and content of the overlaid image. Such augmentation would affect the way our body habitually moves and navigates. This paper explores A.R. expanded spatiality and our body’s plasticity or flexibility to refigure and adapt to movement in space with augmented movement vision.

Author(s)

INTRODUCTION

BODY WARES : SCREEN MATERIALITY

The embodiment of the virtual screen presents a situation in which information has to be organized in relation to the moving body. Conventionally, the mobile virtual space is employed as infospace that is structured around egocentric and/or allocentric spatial frameworks in relation to the body, and without the need to be in a continuous field of Cartesian space. [1] The virtual space, in this way, function as a presence or an absence feature that is aligned along with the structures of the physical but is not constrained by location and physical continuity. It is contended here, further, that structures or features in the virtual space need not at all be aligned within the logic of the Cartesian co-ordinate system  - that is, the virtual image can serve as a direct extension that transforms or augments the dimensionality of the actual space. The potentiality of the virtual screen lies, in part, to the fact that it is a null-space without the necessary constriction of physical laws. Its (screen) materiality consists of a medium through which contents and meanings are being projected from. The malleable virtual contents can function as simulation, representation, presence or mirror, and so forth.  Therefore, the embodied virtual space could extend not only spatiality but also, more radically, the user’s body frame. This implies that there are more potential within such trans-spatiality between the actual and the virtual than the conventional spatial habits and expectations of our body allow. Such spaces do not just present new forms of spatiality but challenges both the body plasticity or flexibility to re-adapt as well as our conventional body-space-time notions of directionality, positioning and orientation in spatial traversing.

TO MOVE, OR NOT

Philosopher Elizabeth Grosz commented that rather than refiguring embodiment, virtual space is often employed in a manner that reaffirm Cartesian mind/body division. [2] There is good reason why (embodied) screen space, in spite of the potentiality of its materiality, is not deployed in more radical ways of interaction to push the capabilities of the body. The kind of augmentation proposed above, challenges deeply ingrained habitual ways of being, and therefore are physically discomforting to the body when implemented. Fracture made to the linearity of the physical space can be made coherent and manageable when the image space is seemingly external to the body, which means that the multiplication of image spaces do not break the singularity of the body’s perceptual frame. However, when augmentation starts to encroach onto the embodied space of the moving body and is deployed as a direct re-structuring of spatial dimensionality or as a re-embodiment of the body, they become highly unmanageable to the body. Which is to say there is a disparity of functional requirements between movement vision or vision for movement and locomotion versus vision for more inactive or stationary activities like reading or simply visually scanning the environment.

Enactive theories expound that perceptual representation is derived from actions. However, here, there is the reversed scenario in which perception precedes the possibility for action. Intuitively, one can perceive and analyse the augmented scene more readily, from an external standpoint,  than one can learn to re-coordinate one’s body to move fluently across the augmented embodied space. Just as, when lost, we stop and refer to our streetmap, to bring actions down to the minimal level, and using our cognitive skills to re-orient ourselves in space. This does not denounce the possibility that higher level off-line perception is inherently rooted in former sensorimotor experience and its memories. [3] It is to be argued in this paper that the challenges of an augmentation of movement vision could be (better) overcome along with the development of both cognitive as well as movement strategies for the body to re-learn, refigure and rehabituate, when the body’s usual perceptual relationship with space is augmented either from within or without.

LAYERING : REALITIES

Whilst the embodied screen has the capacity to simulate all kinds of scenarios and configurations, it possesses certain characteristics that are unique to its medium, in its actual relationship to the body and to the extended space. These could be directly translated into the kind of spatio-temporality that it can configure for embodied experience that is not shared by other set-ups. With its integration with the perceptual field of the body, its content, in fact, has no fixed locality within extended space and has no inherent situatedness – other than with its host, the body.

EXPANDED SPATIALITY  - TEST SUBJECTS

To explore such hybrid dimensionality, this author is currently developing a series of augmented reality art projects with the working premise of using AR strategies to present ways of perceiving and navigating through space that expands from the circumscription of our physical make-up.

In the “Mpov : xTread” (Fig. 1, Left)  series, the user moves around a site with the ability to control concurrently an additional moving point in AR space – which function as an autonomous doubling of her presence and movement in space, such that she is navigating from two positions at once. This project begins with the idea of an expansion of perception from the persistent single, frontedness of the human bipedal body, and investigates the navigation of space with an additional viewpoint. The body (through the multiplied viewpoint) creates a space of active geometry as it moves. In this work the body centredness and directionality is disrupted in that moving forward is not necessarily going forward, but backwards, leftwards etc.  

The “IsoThread” (Fig. 1, Right) series work with virtual forms that transcend from the regularity of directionality and orientation that our body experience when it traverses across the stable structures of the flat ground. The user navigates the actual space through the virtual topological reality as augmentation. The IsoThread project presents a situation in which the body is invited to reconcile the translation of its position and orientation between the physical environment and the form of the virtual model. In traversing through the virtual and actual space concurrently, the mapping of the virtual form onto the physical space is devoid of any fixed location and orientation in actuality. By mapping the topological with the flat plane, going forward loops back on a twisted axes. There is no going forward, backward, left or right.

xRELATIONAL : AUGMENTED DIMENSIONALITY : ACTIVE GEOMETRY

The projects above are designed to explore the character of dynamicism that can be brought about with the embodied screen through the layering of realities between the virtual content and the actual space. Volumes of spaces could be nested and  juxtaposed, dynamically re-sized and morphed, becoming simulation or doubling the actual as re-presentation, all of these configuring space in a non-Euclidean manner. The situatedness of the body within the extended space becomes extended relational (xRelational) in such trans-spatiality. This is because the state of the moving body described here is not so much being relational to other bodies/structures in space, rather it has to be ready to extend from its embodied situatedness and adopt (or embody) a multiplication of positions, viewpoints and spatial reference frames – in the process rendering space as folded, heterogeneous, multiplied and informatics. Incongruous spaces and views inter-join and split apart, configure and reconfigure. The flipping in and out of viewpoints and perspective forming an inter-crossing of perspective (xPerspectival) creating ‘any-spaces-whatever’.

Such configuration produces active geometries, where the experiential space does not have the regularity of flow but consists of interpenetrating volumes which form sub-regularity of orientation, directionality and positioning as the body’s perceptual reference frame is decentered. Such virtualization of embodied space, virtualizes the body by making contingent the body’s borders, and making dissolute the supposed boundary of its exocentric and the egocentric spatial reference and altering the spatio-temporality logic of its movement in space. Subverting Euclidean linearity, the qualitative and differential takes on space structuring and geometricizing functions; with the body creating and configuring space as it moves.

IN-EXCESS : AUGMENTED MOVEMENT VISION AND TRANS-SPATIAL BODY SPACE

The Augmentation of Movement Vision identified here occurs in two manners. Firstly, in the embodied augmented reality, visually led movement creates a disparity between the perceptual information that is received through the virtual screen and the information that is received through the other modalities of the body. Secondly, the augmented reality vision, in this case, forms a multiplication of spatial references and fracturing of the body’s supposed singular egocentric frame. The augmented vision forms an excess that results in the body’s needs to re-learn how it can represent and organize new forms of spatial information to facilitate coordination of its collective parts for actions. This is in contrast to what Brian Massumi terms movement-vision, which describes a proprioceptive state, with no division of subject and object. [4] Augmented Movement Vision, here, describes a level of experience which is closer to the state of internal representation, where a lack of singularity or incongruency of movement in space registered in relation to the single-directional vision of the moving body would break down the body’s capacity to manage movement.

Unlike phenomenological notion of consciousness as defining experience, Bergson  argues that consciousness is derived from the multiplicity of information the body receives; and that our conscious perception is a ‘necessary poverty’ (or diminution) of our image of matter. [5] For Bergson, conscious representation of the matter suppresses and filters away the information it receives of that which is of no interest for our bodily functions.  However, the body is inherently plastic and flexible. Functions of the body are not set in stone, they are open to change brought on by the necessity and demand for new forms of actions in heterogeneous environments. This plasticity of function underlies evolutionary theories of phylogeny and ontogeny (namely, the development found in a species and in an individual over time). Neuroscientist Daphne Bavelier’s research on the effects of multi-tasking in Gaming found that gamers who had to content with split-screens action scenarios for extended time starts to adapt and evolve new mental and vision speed and skills that enable the smooth and skilful means of managing the split-screen environment. [6] Bergson’s notion of a center of indetermination suggests such openness of the body. Information receives is at first unextended in the body, through training they become localized, thus what we experience is memory.

It is inferred here that perhaps the above suggests that the visual information in seeming excess of  the body’s usual functions can be re-embodied with the whole body through the implementation of interaction between the image and the body movement. That is, to re-figure the body movement to accommodate this excess. The idea for this hypothesis is drawn from the examples of experiments undertaken in the field of neuroscience. It has been shown in experiments that the Body Schema – our representation of our own body – show qualities of plasticity. Neuroscientist Angelo Maravita and his colleagues found that multi-sensory integration of visual, tactile and proprioceptive information in primate brain enables it to construct various body-part-centered representations of space; and that this representation shows plasticity for change as active tool-use extends the reachable space and modifies the representation of peri-personal space or the space within the arm-length of the body. [7] Separately, it is known that we can dislocate or project our bodily actions onto the video screen and maintain the integrity of our coordination just by following our actions on the screen – as it is commonly performed by surgeons. This follows that our body schema have the same potential to couple with the screen space as part of its own peri-personal space, and that ‘virtual tools’ could be employed in a similar manner to extend our body space with the virtual space.

COUPLING WITH THE VIRTUAL – A PROPOSAL

In the A.R. project “Mpov” introduced above, the perceptual field in one of the eyes is partly overlaid with the space of the virtual – such that the eyes is looking at the two spaces at once. This may sounds like that the overlaid image would occlude much of the perceptual field. However, in practice,  our stereoscopic vision naturally merge the virtual image onto the actual with some degree of transparency.

In order for the body to efficiently move in such trans-spatiality, the body has to be coupled with the virtual space in some manner, such that the body can find new means of co-ordinated movement. When the virtual space stays outside of the movement space of the body, it puts on cognitive load on the brain. Embodying the virtual has the advantages of off-loading mental processes that would be otherwise be needed to make sense of the hybrid space. Through refiguring the body’s movement, the new movement patterning derived will off-load this into physical processes. Some neuroscientists would agree that mental and physical processes are not distinct but have integrative roles to play in our thoughts processes.

Further, from the neuroscience concepts of bodily path structure and subspaces, it is inferred that a possible method of implementation is to engage the use of a certain part of the body (one of the arms, for instance) to operate and interact with the virtual space. In this manner, the body space is segmented into two frames of realities, and the user can learn to reconcile the hybrid space through movement and sensing.

Path structure is the geometry rules in which our bodies described spatial structure, they determine the distance and direction trajectories for movement. [8] Each movable part of the body has its own path structure, and there are collectively a hierarchy of different path structures in which some belong in the sub-spaces of others. The rotation of the eyes is a path structure that is considered a subspace of the movement of the head. Subspaces working collectively together produces greater degrees of freedom of movement. The plurality and division of sensorimotor spaces suggest the potential for which the body is open to refiguration for more complex scenarios and to the modification of its internal representation of the extended space it maps. When one arm is tap to control and interact the virtual reality, the body is able to physically sense the virtual, as an extension, within the degrees of freedom of movement the arm allows.

“LEARNING A NEW ACT” – ACTION, CONCEPT AND THE BODY

Cognitive scientist Andy Clark points out that there is a robust finding that mental rehearsal can actually improve sports skills and the part of the brain called the cerebellum commonly known as the motor area. [9] He notes the discrepancies in the amount of time proprioceptive feedback information reaches to facilitate the action of smooth skilled reaching, which is between 200-500 milliseconds, in contrast to the mere 70 milliseconds the body could actually perform the same action, suggests that neural circuitry that had learnt the pathways involved in the act could trigger the same pathways on cue. [10] This shows that internal representation does play a role in our actions.

If this raises the case that, indeed, language and concepts aid our body movement, then the next question is: are concepts developed from bodily experiences, from our internal representations, or are they independent products. The Deleuzian ontogenetic notion of concepts argues that concepts do not arrive from experiences without creative and productive conditions. Concepts do not merely state the conditions in which identities are formed, rather they produce real knowledge that are creative analysis rather than facts that are representations of the world.

The challenge then is in creating new concept and language that can more adequately assist us to habituate and navigate the increasing complexities of the digital ecology.

References and Notes: 
  1. Frank Biocca, Arthur Tang, Charles Owen, Weimin Mou and Xiao Fan, “Mobile Infospaces: Personal and Egocentric Space as Psychological Frames for Information Organization in Augmented Reality Environments,” in Foundations of Augmented Cognition, ed. Dylan D. Schmorrow, 154-163 (New Jersey: Lawrence Erlbaum Associates, 2005).
  2. Elizabeth Grosz, Architecture from the Outside: Essays on Virtual and Real Space (Cambridge, MA: The MIT Press, 2001), 85.
  3. Jacques Paillard, “Knowing Where and How to Get There,” in Brain and Space, ed. Jacques Paillard, 461-481 (Oxford: Oxford University Press, 1991).
  4. Henri Bergson, Matter and Memory, trans. N. M. Paul and W. S. Palmer (New York: Zone Books, 1994), 188.
  5. Ibid., 171.
  6. Michelle Trudeau, “Video Games Boost Brain Power, Multitasking Skills,” NPR.org, December 20, 2010, http://www.npr.org/2010/12/20/132077565/video-games-boost-brain-power-multitasking-skills (accessed May 28, 2012).
  7. Angelo Maravita, Charles Spence and Jon Driver, “Multisensory Integration and the Body Schema: Close to Hand and Within Reach,” in Current Biology 13, no. 13 (2003): 531-539.
  8. Jacques Paillard, “Motor and Representational Framing of Space,” in Brain and Space, ed. Jacques Paillard, 163-182 (Oxford: Oxford University Press, 1991).
  9. Andy Clark, “Embodiment and the Philosophy of the Mind,” in Current Issues in Philosophy of the Mind: Royal Institute of Philosophy Supplement, Volume 43, ed. A. O’Hear, 35-52 (Cambridge: Cambridge University Press, 1998).
  10. Ibid., 22.
  
Syndicate content