Cell Tango: An Evolving Interactive Archive of Cellphone Photography

‘Cell Tango’ is an interactive multimedia artwork consisting of a series of animated visualizations. The visualizations are based on a dynamically evolving collection of cellphone photographs contributed instantly by the public. These images, and the accompanying tags that categorize and describe them, are projected large-scale in the gallery, continuously shifting as new contributions are added.

Author(s)

'Cell Tango’ is a dynamic artwork presented as an interactive installation in fine arts museums, in galleries, and at special events. The public visiting the exhibition is invited to interactively participate as a contributor to the project through the submission of cell phone images. These images become the primary content source of the Cell Tango artwork, and are stored online at the Flickr photo management and hosting website.

The gallery display consists of a large cinematic projection that presents the contributed cell phone images in four animation configurations ‘Cell_Bin,’ ‘Cell_Clusters,’ ‘Cell_Burst,’ and ‘Cell_Finale.’ The public is asked to add descriptive tags in the subject heading along with each submitted image. These descriptive tags function as search queries for retrieving other images from the larger Flickr database of public photography and also as organizational devices to create thematic clusters of images. The intent is to explore the potential of unexpected juxtapositions where common semantic labels can generate interesting visual relationships.

The animation ‘Cell_Bin’ consists of visualizing the most recent images in the database on the screen using an algorithm that selectively places large images first and then gradually fills in blank spaces with gradually smaller images until all of the empty screen spaces are filled. ‘Cell_Clusters’ consists of thematic clusters of found images based on contributors’ tags placed around each incoming contributed images that are marked with a yellow frame. The ‘Cell_Burst’ animation throws images on the screen that then open like bursting fireworks, placing the tags once the image appears, followed by the found Flickr images associated with each tag. ‘Cell_Finale’ concludes the visualization sequence by rapidly placing on screen all of the contributed images in the database one after the other in the spectacle action of fireworks exploding in the sky.

The artwork is dynamic meaning that what is featured is continuously being generated in real-time according to the rule-sets of the computer code. There is a conscious approach to software development, and thus it is a form of authorship. The artwork has been designed in such a way so that the software implementation that drives the project expresses the concepts, aesthetics, poetics, and artistic intentions and philosophies of the artists. The software development involves significant engineering and problem solving, but it is driven by cultural concepts and aesthetics. In other words, it was designed with a creative pull as opposed to technological push.

Works of art tend to coalesce concerns, goals, and meaning, all of which evolve at multiple stages in the work’s evolution. These stages include the initial planning and design phase and the continued iterations of transforming the design during production. Many times implicit meanings are recognized only once the work is complete, either by the artists or experts in the field whose job it is to identify what the artwork may mean. Cell Tango addresses a variety of concerns. It is an artwork that is about the creation of an archive, or collection of images in this case limited to cell phone images. The project is based on the participatory, public contribution of data. Each exhibition represents a collection that is the sum of all contributions made by the public during the length of the exhibition. So at the start, the collection is empty; and at the end, the collection is closed. Each time the project is exhibited, the sum of all images collected during the exhibition’s lifetime becomes the embodied visual archive specific to that event.

The research literature that relates to Cell Tango’s foci of interests includes Kindberg’s discussion on the ubiquitous transformative role of cell phone camera usage. [1] Lehtimaki discusses the increased use of picture taking from specialized occasions to the documentation of ordinary life. [2] Miller addresses the social network aspect of image storage sites such as Flickr. [3] Van House articulates a prioritization for image sharing as story telling, self-representation, self-expression, and documenting of everyday life over the long-term archival, collection opportunities that the photo management site offers. [4] The artist Golan Levin [5] has assembled a useful reference page of cell phone art projects, however most projects use the cell phone as a sound instrument in group performance, or else as control devices by which to activate events in site-specific locations, for example by turning on architectural façades with embedded LEDs. [6] The artist Jonah Bruckner created an early cell phone image-based work titled “Phonetic Faces” that consists of a mixed collage of images and portraits on a display contributed by the public present at the exhibition. This work used the cell phone to activate and control an onsite video camera that did the visual recording to a computer that assembled the images. [7]

The Cell Tango project may be understood as an exploration of the process of constructing and conveying meaning through the organization of visual elements (the images) according to a set of rules or conventions defined in the algorithms (the syntax) used to create the visualization. This positions the project within the context of a structuralist, cinematic tradition, as defined by the French film theorist Christian Metz in his analysis of semiotics and film. [8] Cell Tango follows the structuralist film theory model of constructing meaning through juxtaposition, adding an uncertainty component as the aleatory selection of accompanying images are delivered according to common tag labels. The outcome of the selection covers a continuum from a pure literal, analogous matching visual content to visual content that may be totally unrelated at the literal, similarity level, but which may have implicit metaphoric or associative relevance. For instance, a tag that says “circular” might result in an analogous image of a tire, whereas another image which is tagged less literally with words like “yummy” or “sweet” might result in images of cell phones, cupcakes, chocolate drippings, images of a mug, a doodle, et cetera. One of Structuralism’s fundamental principles is that underlying structures lie beneath the surface or the appearance of things. [9] In Cell Tango, the algorithmic processes express the projects’ structure through its programmatic rules, and this functions as the binding by which the juxtaposed images in Cell Tango are organized. When the meanings between juxtaposed images are evident, viewers’ experience in fact becomes less interesting than when the meanings are harder to interpret. Humans naturally attempt to create and interpret meanings out of any information coming their way, and the less evident pairings turn out to be the most stimulating in the viewing experience.

The structuralist approach to art-making and information classification had significant influence in the 1960’s. This approach was initiated through anthropology (Levi-Strauss), and migrated to cinema (Metz), literature (Barthes), architecture, and art (conceptualism, Lewitt, Darboven, Baldessari), et cetera. Over time, the structuralist approach was challenged and rejected for its absolutist approach to the world, but it becomes of interest again considering that computational processes do function at the structural level to generate narrative forms through the articulation of rules.

Cell Tango features the interplay between two image sets: a) the ‘known’ system of the visiting public’s cell phone contributions, and b) the ‘unknown’ open system of tag retrieved associative images from the larger Flickr archive. At the start of the exhibition, the collection is limited to a few starter images, but then builds up throughout the length of the exhibition, and by the end of the exhibition, the collection culminates into the unique record representative of that installation’s specific set of conditions, visitors, cultural and temporal contexts.

Our interests in opening an account at Flickr were primarily utilitarian driven. Flickr is able to processes incoming cell phone messages from all the various international standards. Flickr also includes a filtering mechanism by which sensitive content is managed to a degree. Additionally, we use the site for additional editing and for storing each exhibition as a set.

Cell Tango evolved out of an earlier museum-based data collection installation project titled ‘Pockets Full of Memories’ (PFOM) commissioned by the Centre Georges Pompidou National Museum of Modern Art in Paris in 2001 which then traveled to seven other cities (Rotterdam, 2003, Linz, 2003, Budapest, 2003, Helsinki, 2004, Manchester, 2005, Frankfurt, 2006, Taipei, 2007) each having its own variation based on cultural content defined through location and audience. [10] It consists of a data collection stage where the public first submits personal objects using a scanner and then describes and categorizes the object by answers a series of questions using a data entry kiosk station. These scanned items and their associated data are arranged using a self-organizing mapping algorithm. The algorithm spatially clusters the collection of contributions whereby each object is surrounded by other objects that have the greatest semantic compatibilities. It becomes a form of emergent organization where local relationships create a global order with each object is spatially positioned to every other object in the set according to their individual semantic value. Cell Tango grew out of the artistic, cultural, and computational concerns of PFOM, but aims to alleviate the burden of site-specific data entry. There is no need to be physically present in front of a data entry kiosk station at an art installation. Using the cell phone, anyone, from anywhere, can submit a contribution and interact with the project.

The artistic requirements of Cell Tango necessitated the creation of a robust software implementation that allowed effective user interaction via the cell phone, animation of the visual elements within the different scenes, and the extensibility and adaptability to add new scenes and to easily relocate the project in different environments. Cell Tango visualizes an evolving database of cell phone photography and it is itself an evolving project that changes according to the venue, but also in order to explore new ideas and representations.

The system is separated into four processes that run simultaneously: the data gatherer, the visualization modules, the local network management component, and the graphics renderer. Communication between these processes occurs through writing and reading from concurrent data structures, which efficiently and safely allow multiple threads to access and update the same information asynchronously.

Because the main user interaction with the system is done through submitting cell phone photography, a primary technical goal of the system is to retrieve new photos as soon as they are available. Another goal of the project is to explore the relationship between the user-submitted photos and the related public photos. In order to facilitate the rapid retrieval of new photos and the exploration of related photos, a data-gathering component is placed in a background thread that continually checks the Flickr database. The process gathers both user-submitted photos and tags, as well as related photos from the public Flickr photo pool that are related to the user-submitted photos via their folksonomic tags. A local data store is kept in memory that holds the most recent user-submitted photos and well as selection of photos selected from the pool of other, less recent, user photos.

The system uses a custom 3D graphics framework called 'Behaviorism' [11] which loads each of the photos on to hardware-accelerated textures and also provides a robust scene graph and a sophisticated set of animation and timing techniques that control the layout and narrative of the different visualization modules. Each of the visualization modules processes the data that has been placed within the local data store by the data gatherer. Visual elements based on this data are placed on the scene graph, and then rendered via a dedicated rendering thread at sixty frames per second. Animations of the movement, size, color and structure of visual elements, as well as the movement of the camera view, can be programmatically defined by adding behaviors to the timing graph. These behaviors update the objects in the scene graph at the start of each loop of the rendering thread. In addition to using hardware-accelerated textures for rendering the photos and text, each visualization module is able to use OpenGL for advanced graphics techniques, including binding to custom GLSL shader programs. Because we have had to evolve and adapt the Cell Tango project to new environments (and expect to do so further in the future), the software system is designed to make it easy to add new visualization modules that transform and visualize the photographs and tags in novel ways. Additionally, we have experimented with the sonification of the visualizations, and have developed a networking component for propagating visual or data events to another computer that transforms these events into algorithmically generated composition.

Following the 2006 premier at the International Society of Electronic Arts in San Jose (under a different title, ‘Global Collaborative Visual Mapping Archive’), Cell Tango has been featured in a variety of venues that include: an opening event at a national theater in France; a month in a commercial gallery; a long media arts exhibition at a West Coast research university gallery; a 2 month exhibition at a public Midwestern university; and a 4 month exhibition at an East Coast private university. While Cell Tango functions primarily as an artwork, it also is a research project in the study of how the functionality of photographic image-making is changing through cell phones. Each exhibition is a data collection environment as the public submits images, which are then stored for later analysis. Each venue has its own socio-economic/cultural context with some noticeable differences. User responses revealed that images were expressive of local community interests, social perspectives, and varying degrees of implementation of the functionality of photographing, from recording special events, to recording phenomena, or idiosyncratic expressions. Additionally, there is the play between the collection of submissions through the artwork’s context, which is contrasted to the public images retrieved from Flickr through the associated tags submitted by users.

Since the Cell Tango exhibitions act in one sense as a process of data collection, the next obvious step is to carry out the analysis of this collected data, which we plan to do in the future. We expect Cell Tango to continue on its exhibition life, each time new adding new datasets of user-submitted images. Cell Tango came into being at a time when we are witnessing the exponential changes in cell phone technology. A key component of this project will be the analysis of the various datasets to reveal the correlation between the cultural and technological elements as they change over time.

References and Notes: 

  1. Tim Kindberg, et al., “The Ubiquitous Camera,” IEEE Pervasive Computing 4, no. 2 (2005): 42-50.
  2. Kirsti Lehtimäki, et al., “Documenting the Ordinary,” NordiCHI (2008): 499-502.
  3. Andrew Miller, et al., “Give and Take: A Study of Consumer Photo-sharing Culture and Practice,” CHI (2007): 347-356.
  4. Nancy Van House, “Collocated Photo Sharing,” International Journal of Human-Computer Studies 67, no. 12 (2009): 1073-1086.
  5. Golan Levin’s Web Site, “An Informal Catalogue,” 2001-2004, http://www.flong.com/texts/lists/mobile_phone/ (accessed Aug. 2011).
  6. Media Architecture Institute’s Web Site, “Media Facades,” 2011, http://www.mediaarchitecture.org/ (accessed Aug. 2011).
  7. Jonah Bruckner-Cohen’s Web Site, “Phonetic Faces,” 2003 http://www.mee.tcd.ie/~bruckerj/ (accessed Aug. 2011).
  8. Christian Metz, Film Language: A Semiotics of the Cinema, trans. Michael Taylor (New York: Oxford University Press, 1974).
  9. Alison Assiter, “Althusser and Structuralism,” British Journal of Sociology 35, no. 2 (1984): 272-296.
  10. George Legrady, “Pockets Full of Memories,” Visual Communication 1, no. 2 (2002): 163-169.
  11. Angus Graeme Forbes, et al., “behaviorism: A Framework for Dynamic Data Visualization,” IEEE Transactions on Visualization and Computer Graphics 16, no. 6 (2010): 1164-1171.