Reengineering of the Sensorium and Imaginary Landscape: Mixed Reality

The landscape is understood as the space of experience dominated by the embodiment of subjects and objects. Thus, the space is part of the body. Body and space are blended, mixed, impregnated because the spatial structure of landscape belongs to the description of our surroundings. Geotagging transforms the territorial configuration of the landscape by creating an abstract projection of connection nodes in a global cartography.


Art and Landscape: Mixed  Reality

The landscape as the mediator between different levels of perception and action is also a testimony of the civilizing process that is taking place today in the Occidental Arts. The art has not always resulted exclusively of visual processes, it has also been conceptual throughout its history, reveling the typical phases of development in which it will come to pass in time. The idealization of natural elements resulted in the creation of gardens, as appropriate places for the admiration of nature and it has also resurfaced the Garden of Eden’s myth.

Centuries after, the painting presents great panoramas, totalizing visions, geographies of places and the human attitude against the ground space, the construction of a miniature world in scales perceptible with the body in the ground, as in the landscape paintings of Pieter Brueghel (1525 -1569).

In Europe of the sixteenth and seventeenth centuries, both artists and cartographers used the same vocabulary to describe the geographical representations. Jean Marc Besse [6] states that ‘painters and geographers share the same cognitive attitude and visual acuity,’ they depart from the empirical experience of understanding the world’s signs, driven by the same interests: the rock colors, the movement of river water and the wind. The cartographer and the painter share not only the perception of the earth's surface, but mainly its subject – the landscape. It is important to emphasize the relevance of Leonard da Vinci, who, during his several journeys, studied the geological formations of the areas he visited. This work is expressed in the local maps and feature richness of detail considered in greater degree to cartographers. The artist painted the rivers, settlements and valleys in such a realistic way that we have the impression of looking at a landscape from an airplane.’

In the late 50's – another representative historic moment – artists are once more interested in the landscape with the purpose of relating it to artistic productions. Land Art offers a way of relating art to the countryside in the context of Earth Art, and the ‘protagonist of the aesthetic experience is the environment, space in which individuals interact, while the other, in the broadest sense, in which the self is confronted.’ [3] It also means a rupture with the traditional position of the viewer, who in land art no longer gazes upon the work of art, but inhabits it, expanding the notion of a conceived, perceived space. Instead of mapping roads and landscapes, the art intends to record processes of translating contemplation into human action next to the landscapes that it constantly modifies.

In this century the work with technologies, the 3D world of computers has the ability to take the viewer back to the scene. They allow the creation as action, the works are interventions of embedded data in the urban and cultural spheres, creating geographies of emergency and transience, updated in time of the communication context. Louise Poissant, [8] regarding the interactivity in the Arts, states ‘that it is a response to the feeling that the audience has to be aware of: they want to move the representation registration to that of the action.’ In fact, interaction prevents the user to maintain the distance, the intellectual distance, the criticism, compared to the trompe-l’oeil of the view paintings, in which three-dimensional perspective functioned as a visual recovery of a tactile experience.

Interactivity art has also demystified the role of the artist as a creative author, giving him/her the function of a context designer, the one who prepares the ground to the reception. The work presents itself as a field of co-creation in which users interact with the device and create a renewed aesthetic experience.

The work 14 Bis is an intervention in Augmented Reality (AR) done collaboratively, in honor of the 50th anniversary of Brasília (Brazil). It had as a poetic reference the song Tropicália, by Caetano Veloso, which deals with the construction of the new capital of Brazil, inaugurated in 1960 and idealized by the architects Oscar Niemeyer e Lúcio Costa. Recreating Santos Dumont’s (1873 - 1932) invention as a synthetic image, the project was idealized with the intention of honoring the Brazilian cultural thought.

So that the work exists in Mixed and Augmented Reality, it was necessary to  measure the point of appearance, by using a GPS (Global Positioning System) and inserting the geodesic coordinates in the Layar software, a free browser for augmented reality, compatible to the android operating system used in cellphones and tablets.

The Mixed Reality (MR) is the hybridization that connects us to the physical and digital worlds, the ‘real’ and the ‘virtual.’ Domingues [4] states that ‘with today’s locative, ubiquitous and mobile technologies, men realize their fictional desires of living in parallel worlds, in paracosmos that mix physical materiality and data immateriality, placing them in what we call cybrid realities to cybrid existence.

In loco, this cybridization is reflected in the level of the sensory perception of a combination of information from the physical environment, from the user and the data generated by a computer system and from the mobile and locative interface. Thus, we have the anthropomorphic proposal of Peter Anders, [1] based on the human body, in the experience, between space and information. Domingues [4] states that the subject cybridizes himself/herself in the subject-object-environment flow.

The boundaries of our body

Human sensory changes in the speed of the development of technological devices. Inineteenth century industrial capitalism, the machines actually assume the leadership of the economy. Transforming cities, taking farmers from their lands and turning them into workers, change customs and the relationship with the world. Charles Chaplin criticized this poetically in the film Modern Times, whose protagonist works continuously on a treadmill, making himself an integral part of the machine, his consciousness remains attached to the mechanical movements. With the advent of high-tech machines, computers, robotics and telematics, in post-industrial capitalism, machines get an amazing power and influence over subjectivity. This happens because these new technologies, besides their ubiquitous nature, possess a high degree of sophistication that promotes, with the wishes production system, characteristic of this model of capitalism, a high degree of interaction and relationships. That is, machines are not only literally negotiating human relationships through a variety of portable devices such as telephones, television, computers, and the Internet, but also promoting the emergence of a direct human-machine relationship in the re-engineering of senses and of life.

We model our sensuousness in accordance with the mechanic production of subjectivity that behaves like mechanic gears on the interactive interfaces. Roy Ascott [2] states that the interaction presupposes a reciprocal effect, so that one thing affects the other, adds or transforms, and if we are dealing with a work of art, a ‘transformation of consciousness’ that emerges from the experience, in which the opened meaning models new relations to reality, new ways of feeling in a context interactions with artistic systems. In mixed reality worlds the interaction between art objects and protagonists allows a multisensory that carries the redefinition of the human.

In cyberculture, artistic creation with new interactive media shows a rejection of old art categories, in its place, new fields that pervade the human-computer interaction (HIC), imposing ‘shareable affections’ within a set of sensory experiences that coordinate signals and that trigger the body's perception and proprioception. Diana Domingues in the article- Ciberadão e a magia das cibercoisas pervasivas-, speaks of abiology of interactivity’ in which the technological apparatus adapted to the biological alters profoundly the sensory synthesis, because they allow symbiotic processes that expand to virtual worlds the body's and environment’s natural signals in regenerative changes. The author alludes to the book Symbiotic Man: Perspectives for the Third Millennium, by Joël de Rosnay, that describes the evolution of the interfaces between humans and machines and speaks of numerous functions that are being studied from biological signals, access roads funding and decoding of information.

Interacting on the landscape where the plane 14 Bis is located, the body falls into places, searches territories, compares landscapes and explores relationships. In this sense, the landscape is contaminated by this being-in-the-world, ontological condition of coexistence. And so, Geography, as the science of concrete things, allows itself to be invaded by external processes, categories that transcend disciplinary boundaries and the landscape defines itself in categories of natural and artificial objects that make up space and time for the present and past relations of human work, confirming the assumptions of Milton Santos [9] that ‘nothing is fixed nor motionless in the landscape’. Therefore, the human experience of occupying the existential space creates natural and artificial streams, vital directions and of interactions with communication networks, demonstrating the existence of the symbiotic relationship between human - landscape - communication network, and in ‘the frequency of shifts reveals in the human, a body with a materially sensitive certitude.’ 

Merleau - Ponty’s [7] Phenomenology of Perception, aiming to reach the things themselves, describes the phenomena as they are experienced by consciousness, understands that subject and object are interrelated in the process of dialectical knowledge. He assumes that world knowledge, even in scientific terms, is given from the subject's own experience in the world also as is a body in the world. The body is the subject of perception, seen as the source of the senses, as significance of the relationship of the subject in the world, and the subject seen in its totality, in its structure of relations to things around it. The author draws attention to what is perceived by man, the phenomenon, occurs in a field which he belongs. The emphasis is to demonstrate that the relationship in the world is physique and always significant. 

From this perspective, consciousness is not separated from lived experience, it acquires a meaning and is defined as perception, so that there is no separation and opposition between the rational and sensitive data in the act of apprehension of things, and our experiences are the source of all knowledge acquired in the world and the world comes into existence only when we assign it a meaning. Thus, consciousness is continually tuning itself to the world.

Derrick de Kerkhove [5] speaks of how media edits the environment and therefore the user, our sensorium is being edited by the daily media, by electronic devices: cellphones, computers, tablets affect our strategies of information processing. The author also speaks of a ‘biotechnology,’ this experience can be observed with the use of sensors and devices attached to the body as a means of projecting emotions, media that act as interfaces between language, body and world. In the book culture the consciousness subjectivity performs a psychosensorial synthesis within the body. With the screen, there is a reversal of the consciousness to the outside, it is projected on the screen, we enter in the information.

Interaction in the mixed landscape of the 14 Bis reveals the state of presence-absence as a result of computer vision of cellphone cameras and of post-extrusion biological human vision, the act of looking is shared between the eye of the satellite in the sky and the human eye projected on the screen of the phone, neuropsychophysiology expanding human perception that characterizes the biocybrid condition. Our cognitive and motion models are expanded in the landscape, geo-referenced by a Geographic Information System (GIS) and by computer codes information. And the body attached to the mobile device translates the reengineering of sensuousness.

The living space in this context is presented as historical and mixed reality in which man is a modifying agent. And time updates the immateriality of the work process which, when completed, returns to the void in its timelessness. Because thereal’ time, in 14 Bis is just the point in time in which it updates the system that is composed by the user’s input. In this sense, the concept of time is linked to interactivity, the presence of the object hic et nunc before the perceiving subject. 

In the art’s field, the landscape intervention in mixed and augmented reality represents a cultural transgression that moves from the excessive, the unknown, the hidden, to clarity, to order, the revealed in the becoming of the cybrid landscape, the one that results from the sum of natural elements with the cyber data, which the ordered set of categories creates a new meaning to the everyday experience in the space in the reengineering of life.

References and Notes: 

  1. Peter Anders, “Ciberespaço Antrópico: Definição do Espaço Eletrônico a Partir das Leis Fundamentais,” in Arte e Vida no Século XXI: Tecnologia, Ciência e Criatividade, ed. Diana Domingues (São Paulo: UNESP, 2003).
  2. Roy Ascott, “Avances de la Telemática  y el Arte Interactivo,” in Arte, Ciencia y Tecnología, ed. Espacio Fundation Telefonica (Buenos Aires: Espacio Fundación Telefónica, 2009).
  3. Giulio Carlo Argan, Arte Moderna (São Paulo: Companhia das Letras, 1992).
  4. Diana Domingues, “Ciberestética e a Engenharia do Sentido na Software Arte,” in Estéticas Interativas: Novos Modos de Sentir, eds. Lucia Santaella and Priscila Arantes  (São Paulo: Educ, 2008).
  5. Derrick Kerckhove, “A Arquitetura da Inteligência: Interfaces do Corpo, da Mente e do Mundo,” in Arte e Vida no Século XXI: Tecnologia, Ciência e Criatividade, ed. Diana Domingues (São Paulo: UNESP, 2003).
  6. Jean Marc-Besse, La Sombra de las Cosas: Sobre Paisaje y Geografia (Madrid: Biblioteca Nueva S. L, 2010).
  7. Maurice Merleau-Ponty, Fenomenologia da Percepção, trans. Carlos Alberto Ribeiro de Moura (São Paulo: Martins Fontes, 1999).
  8. Louise Poissant, “Ser e Fazer sobre a Tela,” in Arte e Vida no Século XXI: Tecnologia, Ciência e Criatividade, ed. Diana Domingues (São Paulo: UNESP, 2003).
  9. Milton Santos, A Natureza do Espaço – Técnica e Tempo, Razão e Emoção (São Paulo: EDUSP, 2004).