The image-object notion and art practices using mobile screens.

Whether it is a question of the object supporting the image or of the object represented by this image, the relationship between images/pictures and objects has evolved throughout art history. In this paper, we summarize three different approaches of the image-object notion in order to contribute to it and propose an extension of its field of application in artistic practices using mobile devices.

Author(s)

 

The image-object notion in visual arts

Every visual representation needs a physical medium so that it can be apprehended by its viewers. Since the very origins of visual arts and crafts, artists have been creating artistic objects by manipulating physical materials in various ways. Pictures can be seen because they are inscribed on or within a surface, even when this picture is made up of a very brief phenomenon. Bas-reliefs and sculptures produce images because they take a tangible shape that makes them visible. As a last example, cinema films can be seen by the audience thanks to the projection of its frames on a screen through a light beam. These statements, although obvious, show without ambiguity that an image is linked to the object it is shaped with (paper sheet, canvas, wood piece, etc.), whatever its plastic qualities or aesthetic objectives. Whether it is a question of the object supporting the image or of the object represented by this image, the relationship between images/pictures and objects has evolved throughout art history. Several analyses can be found in theoretical research about the relationship between images and objects. These analyses use the term 'image-object.' We summarize here three different approaches of the image-object notion in order to contribute to it and propose an extension of its field of application in the artistic practices using mobile devices.

Image-object in the Middle Ages

The first direction of research leading to the image-object notion is suggested by Walter Benjamin: "Artistic production begins with ceremonial objects destined to serve in a cult. One may assume that what mattered was their existence, not their being on view." [1] One thing that the author suggests here is that religious pictures and images should also be considered, in their physicality, as objects. This problematic idea has been developed by Jérôme Baschet [2] who sets out the notion of image-object as follows:

In the Middle Ages, there is no image that is not an object at the same time or at least, that is not attached to an object of which it constitutes the scenery and supports its use. [...] We suggest the notion of image-object, in order to highlight that an image can’t be separated neither from the materiality of its medium, nor from its existence as an object, being acted and acting in specific locations and situations, and within the dynamic of social relationships and of connection with the supernatural. [3]

In the specific context of the occidental Middle Ages, an image-object would be "at once created as an image and an object" [4] and this notion "forbids to think it as the simple conjunction of an image and its medium. The image-object should rather be taken as an indissociable whole." [5] This first definition of the image-object, explicitly linked to the social and religious context of the European Middle Ages, is an image shaped by the means of physical material, a medium that gives it the capacity to exist as an object in a religious and social network of activities which determines their functionality.

Manet and the tableau-object

In his analysis of Manet's painting, Michel Foucault [6] uses the notion of tableau-object. [7] If the term 'image' is not used here, this notion shows many similitudes with the image-object notion introduced by Jérôme Baschet and slides it into the context of the painting at the end of the 19th century, when it was more independent of religion. Here also, the medium's materiality of the picture is a major aspect but with the idea that the painting is closely linked to its direct environment, and therefore its exhibition space.

What Manet has done is to make re-emerge, inside the painting's representation, the properties, the qualities or the material limitations of the canvas itself that the pictorial tradition had, in various ways, avoided or hidden until then. Manet re-invents, or maybe invents, the tableau-object, the painting as a materiality, as something colorful that is illuminated by an exterior light and in front of which the spectator turns around. [8]

Foucault's proposition consists in the demonstration of how Manet managed to restore the materiality of the painted artwork by considering, in the visual construction of his paintings, the physical characteristics of the canvas, the way it is shown and the position of the viewer facing it. Foucault seems to accentuate the sculptural aspect of Manet's paintings. As every volume placed in space, the painting becomes sensible to exterior contingencies, and becomes similar to an object that should be observed from various points of view. This idea is confirmed by Foucault's own words about Un bar aux Folies-Bergère (1882):

Here is the very last technic of Manet : the property of the painting to not be a normative space but a space in front of which we can move. The viewer is mobile facing a painting that exterior light strikes directly, verticals and horizontals lines are endlessly doubled, depth is erased. Manet didn't invent a non-representative painting but the painting-object in its material elements. [9]

Image-object and interactivity

The term image-object is frequently used in the context of computer software. Many high-level programming languages use this term to name pictures that are included in a program. [10] More generally, object oriented programming (OOP), by its fundamental principals, naturally lead to the use of this vocabulary. OOP consists of the definition and the interaction of software bricks called objects. An object represents a concept, an idea or any entity whose structure and behavior have to be defined in a way that it can communicate with other objects. In this context, pictures/images becomes objects that can have a particular behavior. Beyond this technical approach, this image-object presents conceptual specificities that Jean-Louis Weissberg raised about interactive images. As it becomes an interactive object, an image can respond to our solicitations through physical interfaces (mouse, keyboard, etc.) because its specific behavior has been programmed. "The image becomes an existence mode for the object and an access to its creation, its transformation, its manipulation." [11] This relies on the digital nature of this image-object. Variable and programmable in time and space, the digital image-object takes place in a simulation, in a time that runs on because the spectator/user interacted with it, a "flatten time" [12] or "uchronic time" [13] that we usually call the 'real time.'

Here is constructed the reality effect of simulation, not in illusion nor substitution, but in intervention. At the end of this process are generate hybrids, intermediary beings, more figurative than images, more functional than objects. Let us accept to call these image-objects. They are not anymore a matter for re-presentation but for presentation/simulation. [14]

Mobile screens: from image to object to image...

With the new generations of smartphones and tablet computers we assisted the gradual disappearance of the physical keyboard. For most of these devices, the main human interface is a multi-touch screen that makes the keyboards virtual, one of many other elements in a graphical user interface. The virtual keyboard of mobile devices is one hint, among others, of the radical compression of traditional human interface input/output devices into one single object: the screen itself. Mobile devices can then be thought as mobile screens. Our research into mobile screens and their use in artistic and interactive creations led us to another variation of the image-object notion. This variation does not exclude the three categories we stated above, but assembles them while concurrently extending the notion toward another meaning. SensorGirl, an experimental artwork we created in collaboration, [15] is a useful tool to illustrate this fourth image-object category.

SensorGirl: mobile screen's image-object model

Accelerometer sensors, primarily made to measure the linear acceleration of the mobile it is mounted on, were popularized by the Nintendo Wii remote controller, some time before the gyroscope, made to measure angular position (orientation). These sensors, nowadays embedded in almost every mobile device available on the market can be used to compute the relative rotation of the device itself as it provides 6-axis motion sensing. In the daily use of, for example, an iPhone or iPad, the current orientation of the device returned by these sensors is used to automatically rotate the main user interface: if the device is physically turned in landscape orientation by the user, the GUI rotates in landscape mode and accordingly with portrait mode. GLGravity, an Apple iOS SDK sample code, [16] demonstrates to developers how to apply the rotation matrix from the sensors to a 3D teapot in real time. This technical demo shows how one can manipulate the mobile screen to see the 3D model from various angles : the model position looks like it is fixed in the actual space, and by rotating the iPhone physically around this virtual object one can see it from the top, the left, the bottom, and so on in real time. The demo is technically well done and shows a new type of gesture, specific to mobile screens and their potential, but it does not aim at any artistic ends.

SensorGirl attempts to give more significance to this special gesture. In this iPhone/iPad application, a 3D feminine figurine wearing a short white dress can be seen in a similar way to the teapot demo. The choice of a feminine figure dressed like this fits the idea of creating a strong analogy between the observation of this 3D model through the device manipulations and a real world situation: to see underneath the dress of an actual doll, one needs to either turn the object around or lower one's head in order to see what is hidden while looking from above. The manipulation of the device to discover what lies underneath is at the same time a pretext, a motivation and an invitation to manipulate the screen. In addition, by a subtle modification of the virtual camera's position according to the current model's rotation, the face of the 3D character can never be seen. Looking out for the face without success brings the spectator naturally to manipulate the screen in every possible angle. In the end, everybody ends up in a near-voyeuristic activity: holding the screen above their head, looking at the underwear of a 3D doll.

In SensorGirl, the physical state of the screen has a direct influence on the represented 3D object. It is by grabbing the device and turning it around that one can interact with the representation. What is discovered by this manipulation can make one laugh, amused, feel uncomfortable, or even offended. [17] In any case, the screen manipulation happens and leads to a result. The gesture of rotating the screen is directly transmitted to the simulated object without the help of any third party interface device. The displaying medium and the interface device are not linked, they are the very same object. Mobile screen image-objects obviously inherits the property of the interactive images presented by Jean-Louis Weissberg: the capacity to be manipulated through their own visualization medium, a mobile screen equipped with embedded sensors. This image-object manipulation, therefore, depends on the material and technical properties of this medium (size, weight, sensor type and precision, etc.), which reminds us of the tableau-object Foucault analyzes in Manet's paintings. Finally, mobile screens are deeply inscribed in our contemporary societies, founded on information exchanges through communication networks. In a similar way to the image-objects of the Middle Ages, they are actively engaged in the social fabric and practices of a particular era. Mobile screens are the place where the three image-object notions we discussed earlier are working together as one notion, with one extension demonstrated by SensorGirl: the screen object itself and the image-object melts together to become one single entity.

A possible interactive grammar?

The gesture involved in SensorGirl is adapted to the interactive visualization of a 3D model in real time and reveals a new image-object model. However, other kinds of relationships with interactive images, in other words other interactive forms, are also a matter of the image-object. Following the path of non-mobile computer image-objects, that developed several well-known interaction gestures (drag and drop, point and click, etc.), mobile screen images-objects can produce various interactive 'figures.' Touch based, and some accelerometer based, interactions have been integrated in mobile device operating systems (iOS, Android OS, etc.) since their initial design, but many other 'image-object oriented' interactivity possibilities lie in these devices and can be used to create interactive artworks designed specifically for these devices. Book Tales [18] is a series of iPad projects based on a simple protocol: every application is an interactive scene using photographs of a book and explores one or more image-object oriented interactivities that mobile screens offer. Every application/book has its own title. Les bonbons, for example, consists of a blurred photograph of an opened book with candies placed on it. By touching the screen, the spectator adjusts the focus only around his fingertips to discover that he can read some words only through the candies, other parts being too blurry to be read. Another book, Temps perdu, shows the picture of an opened white book with a child's marble placed on it. Only inside the projected shadow of the marble does the book text appear and the spectator has to physically tilt over the screen to make the marble roll in the direction of gravity in order to read the text: an impossible and non-functional reading system. Another, Petals, shows an opened book with cherry flower petals spread out on it. Blowing on the device's microphone blows the virtual petals, which twirl and move from their starting position to reveal a blank area on the book page: the text's ink has been absorbed by the petals and is reversed as in a mirror. Other multitouch, gyroscope and accelerometer based gestures are used in other applications of this ongoing series, which can be considered as an artistic application series and a mobile screen based image-object interactivity catalog. This work is part of a larger research project into artistic practices using mobile screens and makes use of a specific programming language we developed called Mobilizing. [19] This tool, currently available on iOS devices, has been conceived with the idea to help artists to prototype and create art works for mobiles screens by providing a simplified programming tool. Partially inspired by Processing, [20] Mobilizing is an ideal tool to create small prototypes of image-object artistic projects and has already been used in various workshops. The result of this research may eventually be the construction of a kind of manual for mobile screen image-objects that makes the inventory of interactive figures available on mobile devices, a kind of interactive grammar particular to these devices.

References and Notes: 
  1. Walter Benjamin, The Work of Art in the Age of Mechanical Reproduction, http://www.marxists.org/reference/subject/philosophy/works/ge/benjamin.htm (accessed September 2011). The French translation uses "ceremonial images," not "ceremonial objects."
  2. Jérôme Baschet, "L'image-objet," in L'iconographie médiévale (Paris: Gallimard, 2008), 25–64.
  3. Ibid., 33–34.
  4. Ibid., 38.
  5. Ibid.
  6. Michel Foucault, La Peinture de Manet (lecture in Tunis, 20 May 1971), http://foucault.info/documents/manet/ (accessed September 2011).
  7. Tableau is the French word used for painting, but it also bears the meaning of picture, canvas and board, which is why we choose to keep the French word in this notion.
  8. Michel Foucault, La Peinture de Manet.
  9. Ibid.
  10. Image-object shows up on many websites about HTML and JavaScript, among others.
  11. Jean-Louis Weissberg, "Sous les vagues, la plage," in Paysages Virtuels. Image Vidéo, Image de Synthèse, Anne Cauquelin, Florence De Meredieu, Anne-Marie Duguet, Jean-Louis Weissberg, 21(Paris: Dis Voir, 1988).
  12. Ibid.
  13. Uchronic time: Edmond Couchot, Des images, du temps et des machines... (Paris: Jacqueline Chambon, 2007).
  14. Jean-Louis Weissberg, "Sous les vagues, la plage," 21.
  15. Dominique Cunin, "SensorGirl," http://dominiquecunin.acronie.org/sensorgirl/ (accessed September 2011).
  16. Apple iOS Developer Library, "GL Gravity," http://developer.apple.com/library/ios/#samplecode/GLGravity (accessed September 2011).
  17. Dominique Cunin, "AppStore reviewers rejected SensorGirl," http://dominiquecunin.acronie.org/sensorgirl/ (accessed September 2011).
  18. Mayumi Okura's personal website, http://mayumiokura.acronie.org/ (accessed September 2011).
  19. Dominique Cunin, "Mobilizing," http://dominiquecunin.acronie.org/mobilizing/ (accessed September 2011).
  20. Official Processing website, http://processing.org/ (accessed September 2011).