Syncretic Social Agency: Deterritorialised Robotics and Mixed Reality Data Transfer Systems

This paper adopts a syncretic approach to the gathering of disparate beliefs and ideologies in order to expand on the topic of anthropomorphic representation in order to deconstruct our relationships with agents and the architecture of autonomy.


Focusing on networked agency this investigation seeks to articulate the need for dialogue in anthropomorphic social robotics to include, in order to fully understand the breadth of relationships, particularly their effect on consciousness and identity. In this paper I refer to the notion of agency rather than the field of robotics as I believe the notion of servitude applies even to the most advanced artificially intelligent autonomous robots. Citing examples of research from the fields that include media art, telematic robotics, real time digital simulation and mechatronics, this is an attempt to analyse hybridized agency in mixed reality data transfer (MRDT) systems that follow a Deleuzian paradigm of intuitive deterritorialisation and reterritorialisation of the body in real time situations.[1] Second order cybernetics was very successful in its endeavor to explain our early relationships with robots in terms of interactivity and connectivity however the incorporation of more networked systems of autonomous/anthropomorphic based interactions have created a system of agency that is less anchored in a traditional bio-physical/electro-physical dichotomy.[2]

It is a popular belief that we are now, through a media convergent, participatory culture (integrated socially through a subnet of platforms) creating a collective intelligence that exists in this global village of knowledge (data) transfer.This perspective evades mythological notions of anthropomorphic interaction. Networked robotic systems that use real time MRDT expand autonomous robotic interaction beyond traditional bio-physical/electro-physical relationships and are integral to understanding our relationship with autonomous agents. Adopting a syncretic approach to this discourse allows for the inclusion of social networks in dialogue concerning social robotics. Syncretism has traditionally been regarded as an attempt to harmonise and analogise disparate ideologies, socio-political views and fields of inquiry. Unlike traditional sites for communication and cultural exchange, social digital platforms rely on actions and conversations to shape not only the social and cultural environments, but also the spatial environments. Such systems allow participants to physically interact with virtual (deterritorialised) biological representations and mediate (reterritorialise) through physical engagement, rather than entering traditional text or numerical based data sets and command sequences.

Robots have always questioned our stance on expressive cultural and spiritual engagement; particularly the social implications of the human-machine synergies have arisen through their development. Robotic development has generally resulted in a significantly anthropomorphic output, that fits nicely into traditional notions of agency and recent sociable robotic development is no difference. Even within the field of artificially intelligent robotics a dichotomous paradigm of ‘otherness’ (master/servant owner/pet child/friend etc.) still serves as the best model to explain our relationship with these machines. This is due to the very physical nature of both entities involved in the system.

Thorough analysis of this traditional cybernetic relationship exists in the work of Donna Haraway and Katherine Hayles that is specific to the integration of humanity with machines. Haraway relates the body’s augmentation through digital technology to the notion of the social cyborg. In A Cyborg Manifesto [3] she argues that the body can be viewed as a conglomerate where its components can be separated, combined with new elements and put together again in ways that violate its traditional boundaries. This rhetoric implies a fractured identity that articulates a cyborg reality that is socially inclusive. In Chaos Bound literary theorist N. Katherine Hayles refers to the notion of dispersed self in light of virtual bodies and narrative, arguing that by turning bodiless information into narratives, the teleology of disembodiment is replaced with contests with ambiguous outcomes: “As I have argued, human being is first of all embodied being, and the complexities of this embodiment mean that human awareness unfolds in very different ways than intelligence in cybernetic machines. Stelarc expresses a more dramatic response claiming,

Evolution ends when technology invades the body. It is no longer of any advantage either to remain “human” or to evolve as a species. Human thought recedes into the human past. The end of philosophy, the end of the human form.” [4]

These views seem to predate the integration of networked technologies within robotic systems, particularly mixed reality data transfer systems. They also are situated in a humanist paradigm that tends to neglect the other species we coexist within these realities. Of course recently Stelarc has shifted his research into networked agency with his Prosthetic Head project. This is an example of how even AI agent systems rely on human engagement still in order to be meaningfully experienced. In the work Stelarc argues that against the metaphysical notion of mind/body separation, declaring the mind to be no more an agent than the body. The work forms a metaphorical network through its intrinsic association with Stelarc’s identity through its similar semblance and behavior to Stelarc. It is in affect a post-biological mixed reality system of data transfer that acts both as agent for the audience and for Stelarc’s ongoing performative exploration of the relationship between mind, body and machine.

Networked agency shifts user-agent relationships into a new era that I believe to be far less autonomous than the physical output of such systems suggest. It is ironic that we often think of non-physically present environments (cyberspace, nanospace and psychic/apparitional space) as virtual realities or alternate/other realities, considering how hybridized they all are, through their ability to transfer data in an interconnected, intercommunicated way. For example I cannot take a rock and combine it with water to create ice in an eco-space, however I can translate an image of a rock into code, cut and paste in the code for an image of water and hybridize the data, which I can then use to create anything from a sound wave to a calibration for a freezer in order to create ice. It is important to remember the fluidity of data as it allows for new media that deconstructs and reconstructs information in a way that can articulate the entire spectrum of wet and dry, natural and artificial, embodied and distributed, tangible and ephemeral, visible and occult. [5]

In order to understand the context of this paper we must first analyse the architecture of autonomy. In the increasingly reliable Wikipedia is suggests autonomous robotics as:

“…robots that can perform desired tasks in unstructured environments without continuous human guidance. Many kinds of robots have some degree of autonomy. Different robots can be autonomous in different ways. A high degree of autonomy is particularly desirable in fields such as space exploration, cleaning floors, mowing lawns, and waste water treatment.” [6]

If one believes that these are indeed the tasks allocated to autonomous robots by definition it appears as though autonomous functions are by nature arbitrary and in no way contributing to a rich cultural discourse.

An example of a current research project that conflicts with this statement is being conducted at the Robotics and Mechatronics Centre (RMC), German Aerospace Centre. This projects developed a real-time interactive simulation and training environment used as a platform for the analysis, training and programming of on-orbit servicing tasks. Basically it is a real time haptic mixed reality data transfer system that can control robots in space. By networking haptic-enabled virtual telepresence with it, the robot does in some sense lose its autonomy, however both user and agent are able to work independently of one another in a hybridized fashion to complete tasks. It is similar to a NASA astronaut receiving commands from mission control and carrying them out, however in this case the data is transferred in an intuitive real time mixed reality state. [7]

Clearly there are experiential differences between the two and one allows for a much broader social engagement in a process usually limited to a few highly trained individuals. It also alleviates the physical conditioning usually requires to perform such tasks. Here the robot is very much the agent friend of the user as it helps them perform far beyond their biological realty, and therefore it is post-biological.

Post-biological, in this sense, refers to a redefinition of the embodied subject which encompasses their location in virtual environments as well as in the physical. This involves the creation, through art practice, of what we might term autonomous agents that are born from data but which take on the appearance of bio-forms and thus become embodied. At the same time these agents are a differential embodiment of the ‘bodies’, which first generated that data in their everyday activities.

Another way of looking at autonomous architecture is the cybernetic paradigm of action/response. The father of cybernetics Norbert Wiener describe this model in regards to how messages are communicated between involved entities in systems:

... society can only be understood through a study of the messages and communication facilities which belong to it; and that in the future development of these messages and communication facilities, messages between man and machines, between machines and man, and between machine and machine, are destined to play an ever increasing part.” [8]

Recent developments in the way we access networks require this statement to be revisited as systems are becoming more universally accessible and interacted with. Our participation in metaverses such as Second Life, and social networked cyberspaces such as Facebook, Twitter, Youtube, Blogs and Wikis serve as a precursor to our collaborative, hybridized colonisation of post-biological space. Soon we will live in a society of second order presence and identity that endlessly creates and distributes various representations of ourselves under a collaborative paradigm of socially inclusive construction. These systems link traditional virtual and biophysical systems in a way that allows free public access and contribution.

In Leonardo Educational Almanac vol.5, N. 5, May 1997, the notes of Eduardo Kac and Antunez Roca decided that robots were:

“… a new art form and they are prone to be hybridized with diverse technologies. This quality makes them transcend the category of object to be diffused into the environment…Robotic art can occur in physical places, in telematic space, in virtual environments, or any combination of these that includes an actual location.” [9]

This definition provides a much more expansive outlook on the spatial presence of autonomous robots, but more importantly it alludes to a hybridized technological engagement that is reliant on networked interactions. A good example of such a system is Ken Goldberg's Telegarden. The work depended on telerobotic users to tend the garden. Web users could view and interact with a remote garden filled with living plants, being able plant, water, and monitor the progress of seedlings via the tender movements of an industrial robot arm. The very life of a living community depended on networked transfer of data to an agent that facilitated the tasks necessary to maintain life.

The final aspect relation to the structure of autonomy is latency in open systems of engagement. All cybernetic feedback systems endure what is known as time-space inconsistency. This is the spatial difference between user and agent and occurs due to latency, bandwidth speed, the paths chosen for data transfer to occur to name a few examples. This creates a deterritorialised autonomy in that a potentially infinite number of users can participate with agents in this ‘gap’. It is in this ambiguous space that robots can truly become autonomous as they are free within the network, emancipated of control and alleviated of the responsibility to respond. While computer scientists detest the effect this has on functionality, artists should embrace this in between space. It is a new millennium version of the gap between painting and viewer, representation and ideas, but it goes beyond dichotomies. It is forever expansive in it’s invitation to be engaged with.

Edward Shanken published a very thorough overview of telerobotics: NeMe: Tele-Agency: Telematics, Telerobotics, and the Art of Meaning in which he describes a range of artworks that push the boundaries of the field. In his conclusion he discusses the difference between active-active and active-passive models of agency:

This nomenclature emphasizes the difference between those models without burdening the issue with the ethically loaded terminology of “master” and “slave” which is pervasive in the robotics literature. But that such terms can be recycled as though depoliticized from any real-world considerations demands further reflection.”

This master-slave metaphor is one that is being replaced by peer to peer through the autonomous appearance of intelligent robotic systems. A dichotomous system I would like to compare this is to a recent model developed for a digital simulation research project by Kashif Zia, Andreas Riener and Alois Ferscha at Johannes Kepler University this year. A simulation of an evacuation situation was established at Linz train station in order to analyse cognitive decision making in socio-technical systems. [10] Participants were asked to wear sensory impaired items such as blindfolds and ear muffs while being dictated by a ‘Lifebelt- a technology designed to assist in such situations. The resulting data indicated an exchange system orientated around fear and hope, both existing as feedback loops that are directly relational to intension, emotion, belief and individualism. All of these are linked to the reliance and trust one has on networked technology and the intelligence that exists at the other end of the data transfer.

Lev Manovich expresses this new relationship with technology as a migration from “the orgy of electronic cables” to a state that is

“…completely integrated and fused with the lived environment…neither threatening nor is it some outside force that has been domesticated. Rather, it is playful and playable: it brings a party to the everyday”. [11]

So taking what has been discussed into consideration how do we then define our relationships with anthropomorphic agents in real time data transfer systems that traverse multiple worlds, ideologies and realities? First we must understand the structure of such relationships and Deleuze and Guattari do very well to relate scientific discourse to such social structures. In Difference and Repetition Deleuze introduces the notion of deterritorialisation (through dispersion) as a “dark precursor” that “relates heterogeneous systems and even completely disparate things.” In order for deterritorialisation to occur there must be some form of agent that can remain constant and self-referent. Deleuze and Guatarri state that: “The alignment of the code or linearity of the nucleic sequence in fact marks a threshold of deterritorialisation of the “sign” that gives it a new ability to be copied and makes the organism more deterritorialised than a crystal: only something deterritorialised is capable of reproducing itself.” [12]

Anthropomorphic agency’s hybridization with physical and biological architecture is constructed by the methods used to connect the environments. The combination and cohesion of heterogeneous elements is generally problematic, particularly when a three dimensional space is primarily viewed on a two dimensional plane. The integration of virtual elements and physical environments rely on bridging disparate with dynamic so that are simultaneously accessible and able to be openly engaged with, edited and developed.

References and Notes: 
  1. Deleuze and Guattari, A Thousand Plateaus
  2. Ascott Syncetic Dialogues
  3. McLuhan, Extensions of Men
  4. Haraway, 1991,Simians Cyborgs and Women, The reinvention of Nature, p.149-181 Routledge, New York
  5. Hayles, N.K. (1990). Chaos Bound: Orderly Disorder in Contemporary Literature and Science. Ithaca, New York. Cornell University Press
  6. Stelarc, 1986. ‘Beyond the Body: Amplified body, Laser Eyes and Third Hand in Theories and Documents of Contemporary Art; A Sourcebook of Artist’s Writings. University if California Press, Berkeley, pp. 426
  7. Ascott, 2010. Syncretic Dialogues. Keynote Presentation at TIIC 2010
  8., accessed on September 9th 2011
  9. Wolff, Preusche, Gerndt, 2011. A Modular Architecture for an Interactive Real-Time Simulation and Training Environment for Satellite On-Orbit Servicing, Proceedings from
  10. 2011 15th IEEE/ACM International Symposium on Distributed Simulation and Real Time Applications
  11. Wiener (1954). Cybernetics in History. In The human use of human beings: Cybernetics and society (pp.15-27). Boston: Houghton Mifflin
  12. Originally published on the Web in Leonardo Electronic Almanac, Vol. 5, N. 5, May 1997.
  13. Zia, Riener, Ferscha 2011. Evacuation Simulation based on Cognitive Decision making model in a Socio-Technical System Proceeding from
  14. 2011 15th IEEE/ACM International Symposium on Distributed Simulation and Real Time Applications