Do all artefacts hold the same cognitive load?

Are art installations, using new technologies, cognitive to the same extent as other spaces or technological artefacts?

New media art installations are not aimed at usability. They turn into artefacts that generate spaces where what has been produced is a result of the worldview of the artist



The theory of distributed cognition, developed formally by Hutchins and his colleagues, is a view of cognitive science which shares the idea that cognition is not situated in the thinking agent. By contrast, knowledge is regarded as a construction that emerges through contact with the artefacts to which people are related. According to this theory the person is an active not a passive agent, and it is also argued that spaces are spaces of distributed memory. On this basis one can infer a cognitive relationship between people and technology, where the latter refers to what human beings have produced as an extension of themselves. Consequently, cognitive artefacts can be understood as “physical objects made by humans for the purpose of aiding, enhancing, or improving cognition.” [1]

The present analysis seeks to distinguish between technological cognitive artefacts and cognitive technological art artefacts. We discuss whether all cognitive artefacts have the same properties in relation to the creation of cognition.


Over a decade ago Jan Greenberg and Gary Dickelman (2000) [2] argued that it was important to ask where and how knowledge is produced, rather than assuming that it is simply located in the head of the thinking agent. Since then the knowledge paradigm has shifted from a focus on the agent to a theory of distributed cognition, developed by authors such as Hutchins from the University of California, San Diego.

Hutchins’ [3] theory, which he terms ‘distributed cognition’,  aims to explain the acquisition of knowledge by using a concept of extended cognition, one that goes beyond the corporal framework of an isolated agent, locating it instead in the environment that involves the agent. This environment, according to Hutchins, is organized by elements such as technological artefacts.

Because the theory of distributed cognition represents knowledge as being located between agents and the world, such knowledge is no longer in a certain place. Rather, it is distributed in a mediating space between agents and artefacts, producing interactions which are able to produce transformations in the agent’s state of knowledge. Hence, it can be argued that cognition takes place in the system, not in the head of the agent or in a precise spatial location.

In line with these ideas authors such as Marshall McLuhan [4] suggest that cultural instruments are “extensions of man,” which alludes to their power to extend human capacity. Similarly, David Kirsh [5] theorizes about “the intelligent use of space,” in which objects and the environment are cognitively structured in order to improve the agent’s capacity, conferring functions of cognitive and physical simplification to space. From this perspective, cognition does not depend on an isolated act of the agent, but rather is produced through interaction with the social and cultural environment. This is especially relevant in contexts that involve technological tools which imply cognitive capacity and can modify the agent’s environment.

However, not all environments or technological artefacts seem to have been designed to be used in the way that people tend to use them. [6] For example, imagine a house that was purchased for its particular orientation and large windows. This might lead us to think that the orientation and the windows serve to provide more hours of daylight or shade, which in turn are associated with financial savings and personal comfort. However, the occupier could just as well use his home and its orientation to determine the time of the day by considering its relationship to the movement of the sun. In this sense, the functional aspect focused on this local use is more powerful, and illustrates how the functional value of artefacts is mobile and has different levels. [7]

At all events there are artistic spaces and artefacts whose cognitive nature diverges from this functional perspective. Rather than being determined by practical usability they are developed as socio-cultural spaces of knowledge whose ultimate ‘function’ lies in the possibility of extending somehow the mind of the interacting agent. Thus, the purpose of mental processes in this case is not the local use of the space or artefact but rather to expand the agent’s experience.

Installations of this kind enable mental processes to be implicated in multiple ways when it comes to the development of knowledge. To put it another way, an installation using new technologies can lead to a cognitive state which has not been intended by its programmers, just as we have seen in the case of functional artefacts or spaces. Therefore, while they may show certain variability their application is not limited to a localized usability as a cognitive tool. For instance, a new media art installation might have been created for its aesthetic or entertainment value, but could become a place of associations and conceptual fusions which provides the inter-actor with experiences that are difficult to achieve. 

Let us consider two examples in order to illustrate what is being proposed.


A.      Gravicells - gravity and resistance

The first example concerns the Japanese research project Gravicells - gravity and resistance, by the artists Seiko Mikami and Sota Ichikawa (Yamaguchi Centre for Arts and Media (YCAM), 2004).

This is an installation in which the agents interact in a GPS space through body movements in real time. The installation consists of projected images and geometric information in the same space, one in which the inter-actors can experiment with their bodies. Their movements through the installation produce changes in the space [Figure 1: Gravity and other external forces are simulated in order to generate new ways of understanding gravity, starting from the experience of sensorimotor processes, aesthetics and the interaction of the body with the simulated environment.]

The installation becomes essentially cognitive on different levels, principally due to its capacity to generate a conceptual fusion which would be difficult to experience in the inter-actor’s life. It comprises two ‘inputs,’ two spaces, with which a third space is created, integrating the first two spaces, thereby leading to a conceptual integration that enables the emergence of a new ‘mental space.’ [8]

  1. The first ‘input’ is determined by an artistically developed technological space, which simulates time-space.
  2. The second ‘input’ is generated by the static or moving images of the universe, which we are all familiar with.

This enables us:

  1. To consciously experience, through sensorimotor processes and first hand, the simulation of a barely accessible aspect of space: zero gravity.
  2. To become aware of the physical laws that establish the representation of space, based on a curved — not a flat — time-space continuum.

As stated above, this experience is produced by a conceptual fusion of two spaces or ‘inputs’ from which a third space arises. This allows the inter-actors to feel their movement as if they were each a celestial body, perceiving the geometric deformation as a consequence of their own displacements and body mass in the created space.

The author of this installation states the following:

 “This work presents the dynamic processes of interaction between gravity and resistance. It was created after reflecting upon the overwhelming difference between everyday life space and the mass of the earth. It seeks to recompose gravity by reconsidering the dialogue between the body and space.”

However, the experience of this project cannot be reduced to this alone, to the instrumentalization or usability of the generated cognitive space. On the contrary, knowledge is integrated in the cognitive processes (mental and sensorimotor) of the agent. This is a consequence of the worldview of the programming artist or the research group, and not of any type of localized functionality.

B.     De-Viewer

The second example is De-Viewer (1992), a project developed under commission by the company ART+COM. This shows a projection of a painting by Giovanni Francesco Caroto, the image of which is altered by the presence of the inter-actor and his/her viewing of the display [Figure 2, images a, b, c. (a) Installation of the system on the projected display; (b) Initial alteration of the process according to the visual action of the inter-actor; (c) Advanced alteration of the image due to the visual action of the inter-actor.]

The underlying technology is based on a system of eye-movement recognition. A computer is used to analyse the view of the inter-actor, calculating the coordinates of the viewpoint on the display. These coordinates are then sent to the graphics system, which alters the image as the viewpoint moves over the image. There can be no identical movement and, as a consequence, no identical alteration of the image. Once the inter-actor stops looking at the image, the alteration disappears and the image returns to its original state.

The artwork is presented in an open space where the agent/artefact relationship creates a hermeneutic environment. The experience of the artwork leads us to question the concept of reality without the presence of the perceiving agent and the very act of knowing. These aspects become evident when we perceive the effect of the visual action upon the perceived ‘object’. The view, seen as the action of the inter-actor, questions the notion of reality as something existing without the presence of the inter-actor. Furthermore, what is emphasized is knowledge based on inter-relationship within an organized system. This holistic emergence of knowledge comprises the relationship between the agent and his/her environment.

What this space does, therefore, is enable the agent to activate cognitive mechanisms with which to experience an abstract theory about reality, which was perhaps not the original idea behind this artwork.

According to its creators this art project was designed:

“… as a reaction to the general attitude to computers as tools rather than a medium, still prevalent at the end of the 1980s. (…) This installation was designed to promote one of the most crucial qualities of computers as a medium, their interactivity or mutual dialogue.”

By contrast, the experience of the work brings to the fore the following aspects:

  1. The possibility to experience consciously the effect of our presence as a constructive element of reality, in this case through eye movement.
  2. The awareness of the interdependence between the object of perception and the perceiver. Space emerges through the experience of enactive cognition [9] in relation to the phenomenon. The agent and the object/artefact are inseparable parts of the same given reality. Hermeneutic and semantic knowledge arises between the ‘other’ and the agent.
  3. The modulation of consciousness in the course of evidencing the worldview — and artistic and cultural point of view — of the programming artist or research group. This aspect is related to the theory of constructing reality through the exhibition of artistic practice.

Thus, we are not dealing here with a functional artefact of mobile instrumental usability, but rather the worldview of the programming artist or research group as an extension of their mind. This is a worldview based on distributed cognition, one in which cognitive mechanisms — such as memory, metaphorical associations or conceptual fusions, sensory, cerebral or sensorimotor processes – produce a holistic mixture that enables the generation of extended and integrated cognitive spaces that endure in the mental structure of the agent.


In light of what has been argued above it would seem important to clarify the concept of cognitive spaces or artefacts as used by Hutchins. [10] Our proposed way of doing this involves two categorizations. The first, or strong category, relates to the increased and permanent integration of new knowledge (of ideas) within the agent’s mental structures. This implies a different modulation of consciousness due to a worldview of cognitive artefacts or space that is neither functionalist nor instrumental.

The second or weak category is related to the priority given to the functional aspect of the artefact or space, which is situated and instrumental. Hence, this category is determined by the abilities which can be generated through the interaction with these artefacts or spaces and, as a consequence, in relation to their local usability and functionality.

In the earlier example of the home, knowledge could be defined as localized, as specific in regard to its usability. Therefore, once the possibility of interaction with the agent has disappeared, the tool that is able to optimize functionality also vanishes. This is not the case, however, of the two examples of art installations. This is because even after they have disappeared or the agent finds him or herself beyond their reach, the knowledge that was generated by the sensorimotor or mental action persists in time by being embedded in memories as an abstract idea. The consciousness that is generated by the worldview, expressed through the cognitive space or artefact, endures within the mental organization of the interacting agent’s knowledge.

Obviously, the worldview that regulates the new media art installations in relation to the inter-actor can show different levels of permeability depending on the theoretical load, the experience and the state of the inter-actor’s attention. This is a state that enables an increment, with different level of inscriptions, of the cognitive experience of the enactive phenomenon. [11] However, in all these cases the incorporation of the phenomenon into the agent’s consciousness tends to last over time.


In conclusion, we propose the need to differentiate between cognitive artefacts or spaces by focusing on two aspects: the modification of cognitive structures and the immediate functional usability of the object. This is related to two possible categorizations, which are termed strong and weak. The criterion of organization for the strong category would be the persistent incorporation of knowledge into cognitive structures, which results from a global worldview in the production of the cognitive artefact or space. This worldview derives from the particular view of the programming artist or research group responsible for developing the cognitive artefact or space.

The weak category is defined in relation to local and instrumental cognitive processes. In this case, functional social, cultural, economic and other local abilities can be acquired provided that the given cognitive artefact or space remains active. This implies a process that takes place in the context of immediate cognition and, therefore, neither modifies cognitive structures nor leads to their integration.

The categorization of cognitive artefacts or spaces as either weak or strong thus depends on the form of cognition with which they are associated, i.e. one that is locally contextualized and immediate (weak) or, in the case of the strong category, one derived from a global worldview which becomes integrated within the agent’s cognitive structures and, therefore, is able to endure beyond its locally contextualized use.

References and Notes: 

  1. E. Hutchins, “Cognitive Artifacts,” in The MIT Encyclopedia of the Cognitive Sciences, ed. R. A. Wilson and C. K. Frank, 126-127 (Cambridge, MA: The MIT Press, 1999).

  2. Jan D. Greenberg and Gary J. Dickelman, "Distributed Cognition: A Foundation for Performance Support," Performance Improvement Vol. 19, No. 6 (2000): 18-24.
  3. E. Hutchins, Cognition in the Wild (Cambridge, MA: The MIT Press, 1994).
  4. Marshall McLuhan, Understanding Media: The Extensions of Man (New York : New American Library, 1966).
  5. D. Kirsh, “The Intelligent Use of Space,” Artificial Intelligence 73, no. 1-2 (1995): 31-68.
  6. D. Norman, Emotional Design:  Why We Love (or Hate) Everyday Things (New York: Basic Books, 2004).
  7. Ibid.
  8. G. Fauconnier and M. Turner, “Conceptual Integration Networks,” Cognitive Science 22, no 2 (1998): 133-187.
  9. F. J. Varela, E. Thompson, and E. Rosch, The Embodied Mind: Cognitive Science and Human Experience (Cambridge, MA: The MIT Press, 1991).
  10. E. Hutchins, “Cognitive Artifacts,” in The MIT Encyclopedia of the Cognitive Sciences, ed. R. A. Wilson and C. K. Frank, 126-127 (Cambridge, MA: The MIT Press, 1999).
  11. F. J. Varela, E. Thompson, and E. Rosch, The Embodied Mind: Cognitive Science and Human Experience (Cambridge, MA: The MIT Press, 1991).

 Image captions

De-Viewer (1992), Linz, Austria, by ART+COM. Rear projection display with infrared eye-tracking system, 100x60 cm. © ART+COM; Photos ART+COM (Used with permission)

Gravicells - gravity and resistance. (2004)  by Seiko Mikami & Sota Ichikawa realtime-interactive installation GPS system 6m x6m floor composed with  sensors, transparent  screens, projectors, speakers, computers
© Seiko Mikami + Sota Ichikawa (Used with permission)