Skediomata: guinea pig and performer

warning: call_user_func_array() [function.call-user-func-array]: First argument is expected to be a valid callback, 'phpmailer_preview_access' was given in /var/www/web/html/includes/menu.inc on line 454.

We present Skediomata, a low cost robotic platform dedicated to drawing sketches. Skediomata is described both as a tool for research for the AIkon-II project and as a performer (in various embodiments) in art installations. We present a brief technical description of the system as well as a summary of two recent art installations where Skediomata was  employed as a live performer.

Author(s)

Introduction

Skediomata, the robotic entity dedicated to the study of drawing was born in research as a guinea pig. Perhaps not surprisingly, it and its siblings have been found to be gifted leading performers when participating in art installations. The original Skediomata is being developped in the context of the AIkon-II research project, an investigation into sketching.

The AIkon-II main research objective is to gain a better understanding of the emergence of style in observational sketches. The methodology deployed to shed light on this complex activity consist of developing a computational model of the processes at play during the sketching cycle.

Interestingly, when designing a robot that interacts with physical reality, the issues encountered are of a very different nature than if the system is solely computational. It is one of the reasons that lead actors from the artificial intelligence community such as Rodney Brooks at the MIT [1] do consider that disembodied artificial intelligence is essentially flawed. Looking at drawing as a complex sensorimotor activity brings new insights into the processes we are investigating and into ways to model these. Furthermore, the type of software architecture that supports communications by distributed concurrently running processes as used in contemporary robotics is well adapted to the simulation of an activity  such as sketching recognised has being the result of the interaction and cooperation of multiple processes [5].

Apart from being an essential and influencial “guinea pig” that furthers our research, due to the fascination that robots exerts on the public, Skediomata has proven to also be an excellent ambassador to promote AIkon-II's work. Furthermore due to its low cost and the type of software architecture developed, groups of Skediomatas feasibly collaborate as performers in art installations. 

In this paper after a brief historical introduction of drawing machines and laying out possible avenues to explain the performative qualities of Skediomata, we present a comprehensive description of the Skediomata platform. We then describe two recent artistic installations presented in 2010 and hint at the future.

Drawing and Robots

There has been a long tradition of drawing automata or machines, which we can trace back to at least the 18th century, e.g., with Maillardet's automaton which was able to draw seven sketches and write four poems. Closer to us there are important links between the origin of computational art and drawing machines. The Algorists [2] who were pioneers of the field made extensive use of early drawing machines namely pen-plotters. A notable member of the group, Roman Verostko who was celebrated in 2009 by ACM Siggraph with the attribution of a prize honoring achievements in digital art, still uses pen-plotters. Although Harold Cohen has for some time now used large format ink printers as output for AARON, until the early nineties he used custom drawing/painting machines. Interestingly the first versions of AARON were using a drawing robot, a type of mechanical turtle that allowed for large scale drawings [3]. This early robot was performing live notably at Documenta 6, 1977, Kassel in Germany. From Cohen's descriptions we can recognise that the performative quality of the installation had a strong impact on the audience; an effect that later Cohen would judge distractive. 

A drawing or even a sketch is made to be appreciated when finished. It is a static image. As often noted by artists, the process of drawing is a private intimate activity, seldomly intended to be shown. Often cited in the literature are the time, movement, process, intention, capturing the quality of the drawing activity. For the observer, the sketch is the trace of the gestures, the paths that the draughtsman's arm/body/hand has taken. It is a memory, a direct witness of the draugthsman's action. The manner in which traces lay on the paper,their positions, curvatures, harmonies or discordances, will influence the observer's perception. The recent discovery of the mirror neuron system (MNS) has lead Gallese and Freedberg [6] [4] to speculate that aesthetic experience in the viewer in front of an artwork can be explain through the MNS activity. Mirror neurons, found in the premotor area of the cortex, are activated when either performing an action or observing another individual performing the same action. The existence of mirror neurons would explain phenomena such as empathy, and reading intentionality. Interestingly, MNS activity could explain the strong effect some robotic art installations exert on audiences. For example, consider the fascination exercised by Edward Inhatowicz's Senster [9] or more recently Ricardo Nascimento's Suffering robot. In these installations there is no attempt at making the robot look organic; on the contrary their artificially is not hidden, their mechanisms are laid bare. It is only through their limbs' layouts, reactions and movements that the audiences feel empathy. 

When we are exhibiting installations where Skediomata is a performer we are exploring on one hand the performative nature of drawing and on the other the perception the audience has of the artistic practice and the artist. We remain safely away from the uncanny valleys; Skediomata does not pretend to be human. It is only an obsessive drawing entity. It has eyes but no head. The arm's limited freedom makes it only able to trace or erase. Yet, it is displaying some form of attention when focussing on a person and some form of intention, as for tracing a line there is a need for intention. The presence of intention is even more striking if the action slightly fails, such as when the arm is attempting to draw a straight line but not managing to do it perfectly. 

Technical Description

Skediomata is a robotic hand-eye system conceived to be used as a guinea pig for experimental research during our investigation into the processes involved when sketching faces. As such the general design objectives for the Skediomata system are to: a) fulfill some minimal requirements necessary for experimental research into the sketching activity, b) be suitable for public demonstration, c) be employed as a performer in art contexts. We are describing here the current version (at the time of writing, in January 2011).

Arm

The arm is conceived as a planar arm with an extra revolute joint allowing the vertical movement of the pen. Having 4 DOF brings the arm's architecture closer to a humanoid arm. This increased morphological proximity influences the observer's affective relation with the robot. To further increase the association with a human arm we have set the joint's angular limits in such a manner that the arm's freedom will resemble that of a human left or right arm. 

The actuators used are Dynamixel AX-12 servos manufactured by Robotis. Each such servo includes an integrated 8 bit micro controller. The servos are addressed with an 8 bit ID that can be networked in a daisy chain. Commands are sent by writing some values in registers. Servos states (for feedback) are queried by reading values from registers. Commands include velocity, position, compliance, maximum load. Feedback includes position, velocity, load, voltage.

Eye

The camera system provides visual data about the sitter and feedback about the drawing gestures and sketching evolving results. These multiple functions entail different requirements: i) motor control to bring the focus either on the drawing or on the sitter, ii) precision to look at the sitter and drawing, iii) speed of capture to observe the hand drawing in real time. A pan and tilt system driven by two ax-12 actuators is used to control the focus of attention's (FOA) direction.

Software

Contemporary robotic software architecture is based on communication between concurrent distributed processes. In recent years we have seen the development of open source robotic software frameworks such as ROS and YARP. These frameworks help organize communication between sensors, processors, and actuators. One of the advantages of these frameworks is that they facilitate components’ reuse and have a large ecosystem of research teams that use these and publish new components that are reusable for other projects. Skediomata is currently using YARP as a framework. An overview of the Skediomata framework is presented in fig. 3. One of the advantages of this architecture is that depending on what Skediomata is used for, AIkon or as a performer, only the Sketcher’s component is specific. Furthermore it is very easy two have multiple instances of the system running concurrently and communicating. Using YARP also provides a web interface that allows for the systems to be monitored and controlled remotely.

AIkon Pre-alpha 

 The AIkon pre-alpha version was exhibited for the first time at the Kinetica Art Fair held in London in early February 2010. The system drew over a hundred visitors' faces during the weekend, the role of the human operators (the authors) being reduced to mechanically changing the paper. The requirements we had set for the system in view of the exhibition was to have AIkon capable of autonomously do live face skeches of visitors (fig 1. and fig 2-I). Our approach is to have the system follow a strategy, method and manner as would be deployed by a human when sketching. In this instance, the manner in which the system produced sketches was inspired by the idea of approximately drawing the structural lines that could be used as initial steps in a more complete sketching cycle.

Sketching Cycle of Aikon-II Pre-alpha

  1. Scan the environment by moving the eye until a face is detected. Uses the Haar feature detector that is part of the openCV library.
  2. Focus the eye onto the sitter.
  3. Limit the region of interest (ROI) to a close-up framing.
  4. Segment figure/ground, simply by applying a flood fill algorithm, with seeds located at coordinates heuristically known to be ground. 
  5. Convert the ROI to gray levels.
  6. Convolve the ROI with a difference of gaussian (DOG). Difference of gaussian are known to be a good approximation of the transformation that occurs in the human in the lateral geniculate nucleus (LGN). 
  7. Draw salient lines with increasing precision.
    • Convolve the image with a Gabor filter banks with diminishing spatial frequencies and tuned to an increasing number of orientations. Gabor filters are known to be a good approximation of simple receptive cells found in the human visual cortex V1 area [8]. Simple and complex cells in V1 are known to have peak excitation (firing rate) to lines with given orientation and high contrast [7]. 
    • Extract the stronger responses from the Gabor filter response images.
    • Compute the location and direction of straight lines corresponding to selected responses.
    • Point to point trajectory planning in the Cartesian space.
    • Resampling and transformation of the trajectory in the Joint space using analytic inverse kinematics.
    • Execute the gesture, sending the succession of Joint positions to the arm's servos controller.

  8. Execute the signing script.
  9. Human operator mechanically and boringly detaches the paper, gives it to the sitter and puts a new sheet of paper while AIkon cools down and waits for the next sitter.

Ladies and Gents at Unleashed Devices

Ladies and Gents was a site specific installation presented at the Unleashed devices exhibition, which was held in September and October 2010 at the Watermans Center in Brentford, London. The idea we developed then was in response to a promenade in the Watermans Center space. A problem with the AIkon installation as exhibited at Kinetica was that it required the presence of a human operator to act as a paper changer device. This was unsuitable for a two month long exhibit. So we decided to work on a site specific installation with the additional constraint of being wall-based, and to produce drawings in response to visitors' presence. The only suitable wall space we found was just in between the Ladies and Gents toilet entrance doors. This location provided inspiration for the piece. The wall based drawings would echo the idea of graffiti as found in lavatories and the gender would provide guidance for the drawings.

Ladies & Gents is an installation employing three Skediomata performers acting respectively as Gent, Lady and Cleaner. Gent has an eye in the Gents lavatories, Lady an eye in the Ladies and Cleaner has two eyes watching the tiles that are located on the wall between the doors outside the lavatories, located in the center's ticket hall. 

Every time a new person enters the public lavatory, some measures are extracted from the captured silhouette. Depending on which room the person is in, either Gent or Lady uses the gender-based measurement to create a somewhat humoristic graffiti. This grafitti is drawn either by Gent or Lady's arm on the tiles located under the Cleaner's eyes. As soon as a graffiti is finished, Cleaner wipes clean the tile, but Cleaner's eyes have previously captured the drawing process, in the form of a video being automatically compressed and uploaded to YouTube and posted on a dedicated blog.

Conclusions and Future

We have presented here Skediomata, a low cost versatile robotic platform used in a research context as well as in art installations. Each time a Skediomata systems was exhibited it attracted the public and press attention, which has enabled us to have a rather full program of commisions and exhibitions for 2011. Even if our focus is on the Aikon-II research project's objectives, the versatility of the Skediomata system allows us to take part in such public engagement activities. Over this coming year, we have planned for most of the work to be on the refinement of the AIkon-II core system, with for main objective to provide AIkon better capacities at using visual feedback to correct or modify a sketch in progress. We foresee that AIkon-II will be exhibited sometime in 2012.

References and Notes: 
  1. R. A. Brooks, "Elephants Don’t Play Chess," in Robotics and Autonomous Systems 6, no. 1-2 (1990): 3-15.
  2. L. Candy and E. Edmonds, Explorations in Art and Technology (London: Springer-Verlag, 2002).
  3. H. Cohen, "What Is an Image," Proceedings of the International Conference on Artificial Intelligence (ICAI 1979), Tokyo, (1979): 34-46.
  4. D. Freedberg and Vittorio Gallese, "Motion, Emotion and Empathy in Esthetic Experience," in Trends in Cognitive Sciences 11, no. 5 (2007): 197–203.
  5. C. Frith and J. Law, "Cognitive and Physiological Processes Underlying Drawing Skills," in Leonardo 28, no. 3 (1990): 203-205.
  6. V. Gallese and D. Freedberg, "Mirror and Canonical Neurons Are Crucial Elements in Easthetic Response, Mirror and Canonical Neurons Are Not Constitutive of Aesthetic Response," in Trends in Cognitive Sciences 11, no. 10 (2007): 411-412.
  7. D.H. Hubel and T.N. Wiesel,  "Receptive Fields of Single Neurons Is the Cat’s Striate Cortex," in Journal of Physiology 148, no. 3 (1959): 574-591. 
  8. J. P. Jones and L. A. Palmer, "An Evaluation of the Two-dimensional Gabor Filter Model of Simple Receptive Fields in Cat Striate Cortex," in Journal of Neurophysiology 58, no. 6 (1987): 1233-1258.
  9. E. Kac, "Origin and Development of Robotic Art," in Art Journal 56, no. 3 (1997): 60-67.