Now, everyday life gestures are all around us with the advances in ubiquitous technologies. While this will help more intuitively used interfaces, it will also bring more standardization. And one step ahead, wide use of standard gestures will be limiting creativity for case specific creative solutions. This paper shares our experiences on creativity triggering practices on visualizing actions.


The Need for Authenticity in New Interaction Design

The “desktop metaphor” has been occupying our lives since 80’s. Thus we learned to live with mice, icons, menus, button like graphic objects, desktop, alternative navigation paths like mental model abstractions, which all needed to be learned. While we all had to learn this metaphoric digital world, today’s children are opening their eyes into a digital world and they easily learn to use these apparatus. We can even say that they do all the things by intuition.

But now, things are changing with the advances in ubiquitous, wearable, tangible technologies. A field called Natural User Interface (NUI) [1] has arisen. We interact with the computers using gestures through devices we wear, cameras, and detectors. Apart from other advantages of the NUI, [2] this means that we can control them with our daily life knowledge and habits. While gestures makes things easier with affordance, predictability, learnability, WYSIWYG, [3] this situation might cause problems in some cases like it might not be that easy to conduct the gesture in several situations. 

With all this data in mind, when we look from the designer side, things are not that easy. An up-to-date interaction designer has to know this new gesture library and the standards of it so that he can design functional solutions. Hence, it will be a challenge to find authentic, in a more design oriented term, “creative” solutions regarding case specific needs. For the clarification of creativity in interactive media, we can use the “bringing something into being that is original (new, unusual, novel, and unexpected) and also valuable (useful, good, adaptive, and appropriate)” definition by Osche. [4]

Here, in this paper, we share our experiences of constructing the basic IMD education with action oriented information design. There are several methods [5] we use, one of which concentrates on visualizing actions as described in this paper. The main idea relies on ontological metaphors [6] as mediating artifacts which is a widely used education method for ideation and visualization process. But here, we focus on interaction based issues like action-reaction relations, predictability, WYSIWYG. 

Our Basic Interaction Design Education Process

Since 1999, for triggering IMD students’ creativity, we have been working on “Breaking the Rules” education methods like;

  • Re-reading traditional cultures 
  • Obstructions (auditory/haptic/visually limited interface design) for Breaking the Rules in Interaction Design [7]
  • Participatory Design and Emerging Technology Case Studies [5]
  • Random objects’ unfamiliar interactions 

First three practices are based on complex design problems for the third and fourth year students. In the light of these experiences, we decided to start this visionary practice from the beginning of four years education as the problem seem to be more like a fundamental structure of interaction thinking. 

First thing we did is to start this fundamental process from the second years with the “random objects and unfamiliar actions” practice. We urge the students to "re-define" or "re-form" artifacts or objects, with unfamiliar actions applied onto them. The main idea of the practice relies on the idea of ontological metaphors to be used as a mediating artifact. This process shouldn’t be named as basic personification but the object is re-designed as a more intelligent whole. The students have to develop a whole interactively working mechanism from a natural object. 

While above explained process helped us with focusing on the complex structure of interactive mechanisms without being bound to technological constraints, there was still something missing regarding creative yet working solutions about action-reaction visualization of action messages. It was hard for the students to explain; 

  • what the message meant, 
  • how you should use it,
  • what is going to happen when you do it,

in a one frame image. 

This time, turning back to our first year students, we restructured our basic interaction design education and looked into the practices of message visualization. We’ve been giving cases studies of combining one random “noun” and one random “verb” in one still image (Figure 1).The results didn’t express the action enough and as a hard case study students couldn’t focus on developing alternative solutions to action-reaction relations. Thus we developed the course structure with below explained study of visualizing action oriented image visualization. 

Implementation of the Action Oriented Image Visualization for Basic Interaction Design Education

During the years 2008-2011, we studied with 45 first year interaction design students within the basic interaction design course which runs simultaneously with the basic design course and each week we give different ideation practices to the students.  

With the above stated knowledge of previous experiences, we added another case study of considering one random noun with eight predefined nouns; to open, to close, to save, to delete, to copy, to group, to move, to cut (Figure 2, 3).

Students develop ideas and sketch as much as alternatives of visualization of these ideas. And the works are evaluated by a jury regarding these criteria:

  • To give the right information on what happens if you use the image, how to use,
  • Creativity,
  • Consistency,
  • Alternative amount.

Rather than evaluating the general performance of all the students we examine every single idea and discuss whether it contains any authentic outcome from the action-reaction relation point of view. We conduct the same study for 3 weeks. 


As a starting point for the evaluation of this experience, we should clarify some facts;

  • Instead of considering this practice as an icon design case study, it should be considered as an information visualization sketch for basic interaction. (Fact 1)
  • Main idea is to develop the fundamental vision of action-reaction relations for interaction design. (Fact 2) 
  • This vision will help the students with any kind of further study in information design of interaction. Especially, for the emerging technology case studies in which the students are not familiar with and there is high standardization of interaction (Fact 3)
  • By not limiting the students with neither interface, nor technological constraints, we want them to stay out of “cliché” ideas (Fact 4).
  • Using one random “noun” with these 8 actions helps them to start thinking of consistency while they are sketching for case specific solutions (Fact 5). 

Students participated in this practice so far, have shown us that this is a considerably easy task for them and they can produce many alternatives when they are not bound to technological and interface constraints. They feel free to imagine basic but creative relations between the object and the 8 actions normally assigned to interface interactions. 

We accept that these out of standard ideas might not work when applied to interface design, but this is not our focus. What we are trying to formulate is the well thought conceptual ideation of action-reaction relations in interaction design and the ability to visualize these thoughts. Students with this experience, coming to the further steps of our interaction design education structure, are easily adapting to the case studies of natural objects’ unfamiliar interactions and also the interaction for the emerging technologies case studies in the next step. Our studies on gesture based interactions for natural user interface, augmented reality and tangible interaction case studies with these students have been much more effective than our previous experiences.  

What we aimed at adding such a practice to our basic interaction design education was to strengthen the first step of “breaking the rules” structure and form the basis of creative ideation process in interaction design. With this point of view, they are moving to the next stages in which we try to prepare them to a harsh design environment of emerging technologies, standards and “cliché” designs. What we should do now is to follow the effects of this study throughout the next years of their education and see what kind of objects and actions are more effective. 

References and Notes: 


  1. Matthias Rauterberg, Morten Fjeld, Helmut Krueger, Martin Bichsel, Uwe Leonhardt, and Markus Meier, “BUILD-IT: A Planning Tool for Construction and Design,” Proceedings of the Conference on Human Factors in Computing Systems (CHI 98), Los Angeles, (April, 1998): 177-178.
  2. George Fitzmaurice, Hiroshi Ishii, and William Buxton, “Bricks: laying the foundations for graspable user interfaces” Proceedings of the Conference on Human Factors in Computing Systems (CHI 95), Denver, Colorado, (May, 1995): 442-449.
  3. Apple Inc., “Apple Human Interface Design Principles,” Apple Developer Web Site, (accessed January 15, 2008).
  4. Richard Osche, Before the Gates of Excellence: The Determinants of Creative Genius (Cambridge: Cambridge University Press, 1990).
  5. Evren Yantaç, Oğuzhan Özcan, and Ayşe Emengen, “Participatory Study in Interactive Media Education: Disabling Environment and Augmented Reality Case,” in Digital Creativity 22, no. 1 (2011): 40-48.
  6. George Lakoff, and Mark Johnson, Metaphors We Live By (Chicago: University of Chicago, 1980).
  7. Oguzhan Özcan, Evren Yantaç, and Mary J. Lou, “Breaking the Rules in Interactive Media Design Education,” in Digital Creativity 20, no. 1-2 (2009): 115-124.