In early Artificial Intelligence approaches, robots sensed their environment, built complete internal models using the sensed data, constructed plans based on those models, and acted to execute their plans. Even though they acted in the world, the world they ‘conceived’ and acted upon was a separate, disembodied reality. Contemporary approaches, in contrast, aim for intelligence that emerges from interacting with the world, thus emphasizing situatedness and embodiment (Brooks 1991, Harvey 2000). The agencies performed by these ‘intelligent’ machines evolve based on the dynamics of their material context. It is only when robotic agents are coupled with an environment, that, according to Beers, their potential to act is realized through the agent’s behaviour in that environment (1995). This is a starting point for considering ecologies that entangle human and nonhuman agents through embodied experience of a shared environment. From a posthuman point of view, embodiment is always contextual and specific; agency is materially enacted and distributed across bodies, rather than located within (see Hayles 1999, Barad 2003, Bennett 2010). Without disregarding their differences, both human and nonhuman agents adapt and know not by observing from the outside, but because they act as part of the world (Barad 2003).

This paper explores new forms of entanglement between agents, human and nonhuman, and probes into their performative potential. Our investigation seeks to set up a conversation between disciplines by looking at the potential of machine agency through the lens of materialist performativity. The notion of the performative here refers to the productive and, at the same time, destabilizing enactment of agency as agents engage with their environment. In considering the performative potential of intelligent machine agents, we are interested in shifting the focus from representational issues to questions of agency and materiality. First, we will discuss embodiment and agency as they are applied in the Dynamical Systems approach to robotics and conceptualized in feminist materialism. The investigation of how, together, these two can open up a third lens through which to look at the performative potential of machine agency will revolve around the authors’ interdisciplinary robotic practice and their work Zwischenräume: a machine-augmented performance environment, which embeds a group of autonomous robots into the architectural fabric of our environment. 

The Dynamical Systems view of agency is based on the observation that “animals are endowed with nervous systems whose dynamics are such that, when coupled with the dynamics of their bodies and environments, these animals can engage in the patterns of behavior necessary for their survival” (Beer & Gallagher 1992). Artificial Intelligence inspired by this view degrades intelligence “in favour of the concept of adaptive behaviour” (Harvey 2000). The lived phenomenal experience of knowing-how outplays the information processing of knowing-that. “Treating an agent —creature, human or robot —as a dynamical system coupled with its environment through sensors and motors, inputs and outputs, leads to a metaphor of agents being perturbed in their dynamics through this coupling”. This contrasts the traditional AI approach, according to which agents are “computing appropriate outputs from their inputs” (Harvey 2000). The metaphor resonates with Varela’s co-evolution between a system and its environment or another system: both evolve through mutual perturbations, setting off a trajectory of mutual adaptations to compensate for the external perturbances. The two structurally coupled systems “have an interlocked history of structural transformations, selecting each other’s trajectories” (Varela, 1979).

As a general formalism the Dynamical Systems’ perspective can be applied to computational systems as well as non-cognitive and non-computational physical systems. Its potential to straddle the Cartesian boundaries between mind, body, and the environment (Clark 1998) opens up a path into thinking across human and nonhuman agential capacities.

Looked at from a posthumanist point of view, embodiment “always is contextual, enmeshed within the specifics of place, time, physiology and culture, which together compose enactment” (Hayles 1999). It aligns with Varela’s biologist view, where experience comes from having a body that always is embedded in an extensive biological, psychological and cultural context (Varela et al: 1991). Agency is a product of this process of enactment, or ‘enaction’, rather than a ‘virtue’ that can be possessed or programmed. In Karen Barad’s performative account, agency is “a matter of intra-acting; it is an enactment, not something that someone or some thing has” (2003). The becoming of agencies and bodies (matter) is mutually entangled—agency is enacted through the dynamic encounter of bodies, while, at the same time, bodies are produced and transformed in this “congealing of agency” (Barad 2003). Similar to the Dynamical Systems view, these material enactments may involve humans or nonhumans, however the materialist feminist perspective challenges not only Cartesian objectivity but unsettles a range of ontological boundaries, deeply ingrained in the Cartesian tradition of modern epistemology, such as human–nonhuman, culture–nature and social–scientific.

“The separation of epistemology from ontology is a reverberation of a metaphysics that assumes an inherent difference between human and nonhuman, subject and object, mind and body, matter and discourse. Onto-epistem-ology – the study of practices of knowing in being – is probably a better way to think about the kind of understandings that are needed to come to terms with how specific intra-actions matter” (Barad 2003).

Our installation work Zwischenräume (Interstitial Spaces) is concerned with the intimate complicities that connect us with the machinic ecologies we create. It develops an unusual concoction of walls, curious robotic agents and surveillance technology to explore the performative potential of the unfolding material pluralogue. The charged terrain of the wall becomes the site for this unusual material encounter, playing out the co-dependant agential relationship between humans, machines and their environment. The installation couples curious robotic agents with our built environment by embedding robots into the architectural skin, sandwiched between the existing wall and a temporary wall that resembles it. Each machine agent is equipped with a motorised hammer, a surveillance camera, and a microphone to interact with its environment and network with the other machines. The hammer is not only used by the robots to pierce holes for the camera eye to see what’s going on outside but also for communicating amongst the collective. The walls’ and the machines’ anatomy intertwine, turning the wall into the machine’s brittle skin, and the machine into the wall’s kinetic organs. The wall-body is the milieu through which the machines intervene and develop and express their desires through knocking, chipping, and punching holes, and adapting. 

The machine-augmented environment embodies the agents in the terrain they survey; they are programmed to be curious and are thus intrinsically motivated to explore and transform their environment. The means of marking and exploring have been adopted from two military references, that of urban combat and visual intelligence. Movements, colours and faces are processed to create an adaptive model of the surrounds that allows the robotic agents to expect learned behaviours and proactively intervene. To these curious machines, learning and adapting are not goal driven but evolve based on what they discover and interpret as ‘interesting’. The intrinsic desire to learn about the world directs both the system’s gaze and its actions, resulting in a feedback process that increases the complexity of the environment relative to the perceptual abilities of the agent. Literally carving a trace of their curiosity into the wall, their desire to look is acted out in the open and manifests materially. They also communicate their state of arousal physically by re-sculpting their environment, rather than using an electronic network. Equipped with contact microphones to listen into the wall and sense the knocking of other robots, they use different knocking signals to rhythmically express excitement (high levels of sustained interest) or frustration (low levels of interest for a certain period of time). The embodied agents act and adapt through their intra-actions with their surrounds; shaping what they ‘desire’ to create or perform. At the same time, they become and are stimulated by what they shape. 

When shown for the first time (figure 1 and 2), the gallery space was bound by glass walls, requiring us to not only stage the intervention but also the environment to be intervened with. The transparent space was turned inside out to present a private, cosy, living room scene oriented towards the public space outside the gallery. The machinery attached to the temporary walls inside the gallery transformed the living room scene into a capricious voyeur that drastically transformed the space over the course of three weeks. While the implicit theme of surveillance and voyeurism is not the focus of this paper, it is worth noting that the enactment and embodiment of the power of the machinic gaze was at the heart of Zwischenräume’s conceptual development. Yet while the voyeurism enacted by Zwischenräume’s robotic actors relies on visual intelligence, the work defies military logic of suspicious behaviour and rather promotes the machines’ capability to seek difference for the sake of being different (Gemeinboeck & Saunders 2011). The machines’ motivation to seek difference for the sake of difference, rather than for the purpose of othering that which is different, sets the tone for an alternative investigation into the politics of surveillance and its material affect. It isn’t as simple as incriminating or trivialising the machine. The mingling of agencies and materialities in our installation and the way in which the audience is implicitly implicated, rather than invited to control the course of events, intimates the heterogeneous nature of surveillance. Thus, it is not the spectacle of the intervening machinery that we are interested in, but rather the spectacle of the mutual processes this intervention unfolds as it foregrounds the material ecology of this machine augmented environment and its ongoing becoming. 

The structural coupling of machinic agents and our built environment politicizes the matter of material agency and aims to foreground the performative potential of Dynamical Systems. Looked at from an expanded, ecological perspective, the work enacts what Jane Bennett describes as “an encounter between ontologically diverse agents, some human, some not, though all are thoroughly material” (2010). Zwischenräume’s drama features the encounter of two nonhuman agents, both human-made artefacts, one imbued with (artificial) intelligence and an ability to be proactive and the other designed to be inert and deprived of any vital qualities. We were interested in the co-dependent nature of this assemblage of forces, and the affective relationship through which it evolves. The structural coupling of machine and environment sets in motion their path of material becoming; both evolve through continual adaptations to compensate for the mutual perturbations. The process opens up the transversality of assemblages that owe their agential forces to the vitality of the materialities and dynamic spatio-temporal relations that constitute them (Bennett 2010). It’s a performance that always unfolds in the present, without the comfort of rehearsal. Rather, as Matthew Fuller argues, “the process of becoming that is machinic heterogenesis has no plot, as in story or territory, only a “middle,” an ongoingness: It cannot be turned into a standard object, it must be done” (2005). 

The new assemblage not only challenges the structural integrity of the wall but also intervenes into the socio-politics of our third skin, laying open its vulnerabilities to continuous perforation. While perhaps the machine-wall couple seems purely destructive at first, together they unsettle the politics of the wall and turn it into a negotiable playground. Looked at as actors, they have much in common: both are as much technological as they are cultural; each models nature. The dynamical system underlying the first is inspired by the observation of the animal, while the latter renders the cave efficient, mobile, and mass producible. Both are ambiguous with regards to their acting: the machine empowers some and deprives others; the wall includes some and excludes others. Both extend the human: the machine is an extension of both, mind and body, while the wall is our extended skin. And yet, the dynamic agential forces of the machine are much closer to the human. We are more empathetic to, and at the same time, threatened by them. We (Westerners) cannot perceive the vital qualities of the wall, whereas the embedded machines can render it alive. The performativity of the machinic wall is further complicated by the machines’ autonomy; the self-motivated act of destructing the wall, the self-motivated act of looking. This is where it gets uncanny. It’s ok if the machines act on our behalf, and we control the machine that deconstructs the wall or if human governance drives the machine’s eye. Yet intrinsically motivated agents exhibit a higher degree of autonomy than agents motivated by an external human agent. The meaning of agency changes drastically, once the human actor can no longer control the human-machine-environment coupling. The discomfort of this shift, of course, reaffirms the segregation and hierarchisation of these actors. Neither the machines nor the wall exist outside the realm of human culture, and the autonomy of the machine is simply stretching its capacity to extend the human further: its intrinsic motivation, even if artificial and perhaps alien, is still modeled by a human agent, as is the design of its material embodiment.

This stretching quality was exactly what we aimed for with Zwischenräume, allowing us to stretch into the environment, to intra-act, not as the isolated and superior human but as part of a bigger assemblage. Coupling autonomously performing agents with our built environment opens up a space for Barad’s ‘congealing of agency’ (2003) where the different agential forces not only co-evolve but potentially conspire and perform together. All actors involved are vital players, entangled in a complicated web of connections and specificities. While non-anthropomorphic, the material embodiment of the machines’ cognitive processes and desires places them in a realm, where we (humans) can share and bodily experience them. The unfolding relationship between audience, machines and other matter, materialises a slice of our machinic ecology and makes tangible our position within.