Designing a way to visualize the invisible through the dynamic illustration of conversations

This paper discusses the creation of a multi-modal data driven prototype application called the Conversation Viewer. Designed to visually represent the evolution of a conversation through a dynamic touch based graphical interface, it illustrates various elements of participants’ email, text and voice messages as they seek to find a mutual agreement around a meeting date.



The rise of social media and the sophisticated development of third party applications, in combination with an ever-increasing range of mobile devices, makes the immediate communication of ideas between individuals, different communities of practice and the wider community possible. While we have the ability to send and receive vast amounts of information, status updates and social commentary about different events there is a need to capture dynamic systems of communication in a way that illustrates the less visible relations between participants of a conversation. This is significant to situations where the consequences of numerous interactions can change the perspective and direction of different courses of action. For example, this holds relevancy for participants in distributed environments involved in the collaborative development of long-term projects. Since digital communications commonly involve direct communication between individuals responsible for sending and receiving information, knowledge resulting from group discussions or informal corridor conversations is not as easily shared amongst participants in distributed environments via email, project documents and websites.  

Rather than seeking to integrate multiple systems of communication made available through aggregation tools such as Netvibes [1] or Hootsuite, [2] this research focuses on assisting others to contextually build relations between their different communications. Here the intent is to supply individuals with a comprehensive understanding of the circumstances surrounding the progressive development of different thoughts and actions that lead to mutual agreements. This approach positions human participation, negotiation and understanding at the heart of dynamic communication systems and looks beyond the notion of interaction as the mere sending, receiving or aggregation of disparate messages. For the purposes of this research, the term 'dynamic situation' refers to a set of circumstances in which one has an awareness of their surrounding environment. In particular, this can be understood as a situation that enables individuals to understand not only their contribution to an ongoing conversation but also that of other participants involved in the same discussion.

In light of this, Bell Labs France researchers specializing in hybrid and social communication, intuitive collaboration and applications design worked together to develop a prototype application called the Conversation Viewer. Its role is to make visible the abstract relations that exist between participants of a conversation. A touch-based graphical interface is used to interact with and visualize the progressive development of conversations drawn from participants’ email, text and voice messages. The data driven prototype application was initially designed as an iPhone application that was later developed for the iPad (See Fig. 1).  

The impact of functionality on form

The goal in designing and developing the Conversation Viewer was to create a way to deal with the often-fragmented experience of understanding individual and group intentions expressed by voice and text-based data. In doing so, the Conversation Viewer aims to provide greater awareness surrounding the overall actions and interactions of participants engaged in evolving conversations. This is where time as a factor of interaction in the past, present and future can be used not only as a reference to illustrate the history of a conversation but also to study the value of interactions.   

The key function of the Conversation Viewer is in the utility it offers participants to interact and develop conversations. Interaction forms the basis of conversations in which elements of dynamic situations are negotiated between participants to develop a desirable outcome. The impact of functionality on the form of conversations is in the approach taken by participants during the course of interaction. It is important that understandings are not communicated; instead they are built collaboratively through conversation where participants derive meaning from their interpretation of a discussion. A new-formed understanding is then offered to participants for further interpretation and comparison to the original, which eventuates in mutual understanding and agreement. [3] In this way, it is possible to understand how the collaborative development of a dynamic situation facilitates the collective learning of different objectives between participants through a shared process of negotiation, understanding and agreement. Hence, the Conversation Viewer offers a means for individuals to inquire about the evolution of dynamic situations and exchange information through a series of interactions to reveal and resolve contradictory ideas.

Fundamental to this is the function of an observer as an accepted participant in the act of observing that allows for subsequent understanding from such actions to be derived. [4] During the development of interaction, individuals are accepted as mutual participants in the act of knowledge creation. In doing so different individuals, considered as observers and participants, become necessary elements in the development of dynamic situations. This in turn enables them to act subjectively. By interacting with various participants involved in conversation, understanding is created through the exchange of ideas that lead to mutual agreement. This involvement is interactive and productive so that individuals affect and are affected by the interactions in which they participate. The interaction should represent the culmination of the participant’s interpretations. Significant to this is that, “The language of the conversation must bridge the logical gap between past and future, but in doing so it should not limit the variety of possible futures that are discussed nor should it force the choice of a future that is unfree.” [5] This is important because the form of a system directly influences its usefulness as a tool to support the evolution and understanding of dynamic situations.

The useful and visible functions of a system

The expression of individual intentions and the broad circumstances associated with various events and actions that take place within dynamic situations underpins the useful function of the Conversation Viewer. Creating a tool to achieve this becomes challenging, since there are no established interaction design patters or use cases for the design of dynamic communication systems. Nor is it possible. The Conversation Viewer, therefore, seeks to provide end-users of the system with a contextual timeline of events that visually flow together with the collaborative evolution of conversations over time. Building the relations between these activities means that individuals communicating with and though the Conversation Viewer are not required to search through numerous email threads, text messages and voice mails to gain a quick overview of the current state of conversation. Instead, the data driven system illustrates conversations at both general and detailed levels of information in the form of a visual narrative. The 1MC Viewer also opens up the potential to express the character of individual actions or those of a group. To achieve this a technical component called a sentiment analyzer interprets the mood of an individual’s communication and illustrates it in the design of the ambient interface. While applications such as the iPhone Tracker [6] provide a contextual and historical narrative of an individual’s physical movements from location-based data captured by the iPhone, the Conversation Viewer represents a visual narrative of (1) participants’ relationships towards reaching an agreement, (2) general and detailed information about the terms of agreement, as well as (3) the emotional disposition of participants during a conversation, as an organized integrated whole. 

To give further context to the application’s use, the following scenario briefly describes the potential interactions that could take place during the discussion, understanding and confirmation of an agreement. For the purposes of this example the terms of agreement are to find an appropriate meeting date. Fig. 2a represents the first interaction in a conversation. It illustrates a message from John in brief format that has been reduced in detail for the purposes of simplicity, as a function of the system. When touched, the message expands to show its detailed contents if so desired. At present, the agreement point is unclear, which will become more focused when a precise meeting date is proposed. Laura and Steve are represented in grey while John is represented in black, as he is the first to participate in the conversation. The participants of the conversation are positioned as neutral with respect to the agreement point. This is indicated by the background location rings in the design of the interface to help visualize each participant’s relationship to the terms of agreement. As each participant of the conversation interacts with one another, their visual appearance transforms from grey to back. Simple emotions are also represented by the system’s interface, based on semantic information found in each communication. 

Halfway through the discussions in Fig. 2b a specific meeting date has been suggested. This is visually confirmed by the clarity of center point at the core of the agreement. Here both John and Steve are closer to the agreement due to their repositioning with respect to the location rings. We can also see that Steve is happy about this arrangement while John remains neutral to the current proposal. Finally, after much discussion the meeting is confirmed (see Fig. 2c). This is visually represented by the participants close proximity to the agreement point and its solid appearance. By presenting conversations in this way, it is envisaged that participants of the conversation can gain an understanding of their development more easily than reading or listening to each individual email and voice message that forms the basis of their representation. Furthermore, with the added functionality of (1) a global slider in a customized version of the Conversation Viewer shown in Fig. 3 or (2) the gestural activity of swiping along the vertical list of communications on the left hand side of the iPad application in Fig.1, an animated view of the conversation visually evolves at the same time each moment of the discussion is discovered.


The overall purpose of the Conversation Viewer is to facilitate different ways of observing and participating in conversations and to offer end-users of the system greater contextual understanding of the evolution of dynamic situations. This is achieved through conversation where multiple viewpoints are expressed, visualized and internalized by those engaging in the discussion. As a result, a shared understanding of what is known from that which was previously unknown is created. Essential to this communication is that participants enter into the conversation with different perspectives and individual understandings that are distinct from any others. Given that without difference, there is no basis for exchange or discussion among participants that leads to the mutual understanding of something new. [7] Communication ceases to be productive without a context of difference or conflict to initiate change. [8]

It is envisaged that the manner in which dynamic situations can be experienced through the active engagement and manipulation of changing circumstances in the Conversation Viewer will assist the diverse consideration and development of mutual agreements. This is where interaction is the product of an individual’s capacity to communicate, and develop an understanding of their actions with respect to the thoughts and knowledge of other participants’ terms of agreement. Interaction is not considered in a mechanistic sense visualized by the re-positioning of graphical icons that gravitate toward the point of agreement in the Conversation Viewer’s interface. For that is the resulting material outcome of interaction.

The notion of interaction in the context of this research is centered upon the actions required to reach a mutual goal. This in turn drives the purpose of communicating with different individuals through the Conversation Viewer; especially where the development of a conversation is as a result of a comprehensive understanding of a dynamic situation. The significance of interaction is therefore accomplished through the act of doing, which in turn enables an individual to cultivate a shared understanding of the current terms of an agreement.

Ideally, interactive products or services should offer participants, in its system of communication, the freedom to choose how they may express and fulfill their goals rather than being forced to accomplish a task by way of a limited system of interaction. This is where the idea of participating and communicating through a product or service is much greater than its physical manifestation and transcends the materiality of a product. Here the notion of an organized, integrated whole that interconnects people with their environment becomes important. Interaction in this sense is largely concerned with interconnectedness between all the elements of the design situation. This can be graphic signs and symbols, material objects, activities, services, organizations, environments or systems. What is significant is the active participation of these elements with one another as an organized integrated whole. When all the parts of a design solution are connected, everything is in harmony. This holds significance for the dynamic treatment of a product’s content with respect to the form of design outcomes; particularly in the design of products or services like the Conversation Viewer that seek to support the changing conditions of dynamic communication systems.

References and Notes: 
  1. Netvibes Web Site, “Netvibes. Dashboard Everything,” May 2011, (accessed January 10, 2009).
  2. Hootsuite Web Site, “Social Media Dashboard,” 2008-2011, (accessed May 17, 2010).
  3. Natalie Ebenreuter, “Dynamics of Design,” inKybernetes: The International Journal of Systems & Cybernetics 36, no. 9/10 (2007): 1318 - 1328.
  4. Ranulph Glanville, “Thinking second-order cybernetics” (paper presented at the Cybernetics Society 27th Annual Conference, London, UK, September 14, 2002).
  5. John Christopher Jones, Design Methods (2nd edition. New York: Van Nostrand Reinhold, 1992), 407.
  6. Pete Warden’s Web Site, “iPhone Tracker,” 2011, (accessed April 27, 2011).
  7. Ranulph Glanville, “And he was magic,” inKybernetes: The International Journal of Systems and Cybernetics 30, no. 5/6 (2001):652–673.
  8. Dean C. Barnlund, “Communication: The context of change,”  Basic Readings in Communication Theory, ed. C. David Mortensen, 6–26 (2nd edition. New York: Harper and Row, 1979).