Building Accessibility Stories: Enabling Multi-Sensory Experiences within the OLPC program in Uruguay

This paper shows Enabling Stories, a multimodal, interactive storytelling application, designed for children with physical disabilities, as well as children with normative development, that runs on One Laptop Per child’s computers.

Enabling Stories is a tool for stimulating the development of specific cognitive functions and skills, as well as promoting digital inclusion, and improving social, emotional and motivational aspects on its users.

Author(s)

Introduction

The One Laptop Per Child (OLPC) program aims to provide each child in developing regions with a connected laptop to address the digital divide and to allow them “to become connected to each other, to the world and to a brighter future.” [1][2]

Uruguay, through its governmental project –Plan Ceibal– was the first country to achieve the ‘full deployment’ status, after successfully delivering a laptop (named XO) to every public schooled child between 6 and 12 years old. [3] The acronym ‘Ceibal’ stands for Conectividad Educativa de Informática Básica para el Aprendizaje en Línea (Basic Computing Educative Connectivity for Online Learning; more information about Plan Ceibal is available at http://www.ceibal.org.uy). Plan Ceibal is now targeting secondary education (youngsters from 12 to 18 years old) as well.

Although Plan Ceibal successfully enabled internet access to all its users, by claiming that all the child population of Uruguay is now connected it ignores the fact that access to technology transcends the mere physical immediacy. There are users with disabilities that difficult or prevent them from fully take advantage of the Plan’s potentials and opportunities. An ethnographic study conducted in the primary school Dr. Ricardo Caritat – the only public school in Uruguay for children with motor impairments – observed that a combination of factors, including the XO ergonomics, the software interaction design, the lack of suitable software accessibility helpers, and the limited availability of accessibility peripherals, undermine the XO’s accessibility. [4]

In spite of the fact that the Plan Ceibal program explicitly states that the children are the true owners of the computers, the accessibility problems discourage physically impaired children from appropriating this technology, and therefore, difficult their access to the knowledge society.

Accessing Ceibal

Thanks to OLPC laptops coming with an integrated webcam, a reasonably fast processor and a speaker, it is possible to develop multimodal interfaces to improve accessibility. [4] Potentially, multimodal interfaces accommodate a broader range of users than traditional graphical user interfaces (GUIs) or unimodal interfaces, allowing for the inclusion of users of different ages, skill levels, cognitive styles, sensory impairments, and other temporary or permanent disabilities. [5]

With the aim to address these problems, and to foster an authentic appropriation of this technology by children with motor impairments, an innovative interaction schema to access the XO was proposed. [4] The new schema avoids the “keyboard, mouse, screen” paradigm by allowing for the execution of actions that are triggered when the user shows an image to or occludes it from the XO’s camera, and by also providing multimodal sensory feedback.

This images can be of any kind (our implementation utilizes printed images created with a standard printer), as long as they meet certain requirements of resolution, contrast between foreground and background, and uniqueness to be easily identifiable and differentiable from one another. Images that can be considered for the interaction include pictographs (such as representations of animals, people, or means of transportation), ideograms (resembling concepts, actions, or instructions), words, letters, numbers, two-dimensional barcodes, among other symbols. In addition, the images can be of any size or shape provided they fit in the field of view; can be printed in paper, cardboard, stickers, or other printing media; can vary in color; and can be glued or attached to different objects.

Since the images can be arranged in different supports, they can afford different user actions, like touching, rotating, or holding. This versatility gives designers and application developers the freedom to create different games or applications based on diverse interaction strategies. 

From an educational point of view, this approach can potentially be employed in developing cognitive, affective, and psychomotor educational aspects.

Multimodal activities

Three different types of applications (in XO jargon, activities) were created to explore different educational strategies that could help develop children’s motor and cognitive skills, improve their self-esteem, augment their autonomy and foster their relationship with the environment and with other persons.

The activities are as follows:

Activities focused on developing specific cognitive functions and skills involve presenting users with a problem and waiting for the right answer. The solution could demand sorting images (e.g., distributed on several printed cards with a rigid base), or picking, from a set of printed images, the one that contains the correct answer. For example, to develop children spatial relationships, the layout could consist of a set of images illustrating the relationships up-down, in-out, or open-closed. The user’s goal is to identify the abstract relationships and matching them with the corresponding cards in the deck.

Activities focused on promoting digital inclusion and enhancing autonomy allow for the user to trigger predefined actions in the XO –e.g. opening an activity, unrolling a menu, controlling a game character, moving the mouse cursor, or selecting a tool– by choosing from a number of options printed on a proper media, such as a sheet of paper. Three activities are being developed: a faux keyboard that associates drawings to some keyboard keys (e.g. the arrow and enter keys), the bookmark where the user can navigate to pre-defined web sites by touching its corresponding drawing, and the launcher, a menu from where the user can run XO activities.

Activities focused on encouraging social, emotional and motivational aspects incorporate musical, recreational and narrative elements. One of the prototypes under development is an interactive storytelling application, which will be discussed in the next section.

The proposed activities share some common characteristics:

 

  • Beyond the cognitive challenge associated to the process of triggering a specific action from the computer, all the activities demand children to reinforce their motor skills, either by seeking a printed card from a deck, pointing at (or occluding) an image, or raising an image to show it to the camera. 
  • The simplicity and versatility of the input device –e.g. with a piece of paper it is possible to create a keyboard– allow the teachers to create their own exercises, encouraging them to effectively use the XO as an educational tool for their classes.
  • Regardless of the fact that the framework was designed for children with motor impairments, nothing prevents it from being used by children without those impairments. 

 

By the massive use of these prototypes we expect to obtain data about their usability within this target population. Our observation indicates that the activities are perceived as highly attractive by its users, showing that it is possible to design new ones with a high motivational component.

Interactive Storytelling

Storytelling -or the art of telling stories- is a fundamental characteristic of mankind. In the words of Hamilton and Weiss, “storytelling is the oldest form of education”, since “people around the world have always told tales as a way of passing down their cultural beliefs, traditions, and history to future generations.” [6] The process of listening, creating, and telling stories is particularly important in the development of children: stories can help them to understand their world and to share it with others. [7] In early development stages, storytelling reinforces communication, recognition, and recall skills, favors social relationships between peers and adults, and provides meaning to their living experiences. In later stages, storytelling improve linguistic and literacy skills, and promote interpretation, analysis, and synthesis abilities. [8]

In addition, storytelling has been used as a method to work with children’s conflicts and concerns, helping  them to “objectify and, at times, to personify the problems that they experience as oppressive.” [9] For children with disabilities, different storytelling approaches have been used to support the development of cognitive functions and skills. [8]

In recent years, interactive storytelling has become increasingly popular in the field of human-computer interaction. In 2010, the full-day workshop of the ACM’s (Association for Computing Machinery) Ninth International Conference on Interaction Design and Children (IDC 2010), titled “Interactive Storytelling for Children”, aimed to bring together researchers from different fields that share a common interest in the subject. [8]

Compared to traditional storytelling, interactive storytelling applications allow the audience to actively participate in the drama, by altering -during the course of the story- the action, character, dialogue, and narrative performance. As a consequence, interactive storytelling can multiply the narrative possibilities, upgrading the role of the audience from listeners to participants of the story. As in traditional storytelling, there exists the role of a storyteller, either human or synthetic, whose challenge is to make sure that the story maintains a certain level of quality.

This quality can be measured in terms of consistency, enjoyability, memorability, among other parameters. When designing and developing a digital interactive storyteller, these requirements can be difficult to accomplish since it is difficult to find a proper tradeoff between the story narrative and user interaction. [10]

Research in this field includes specific applications, such as the construction of furry storytelling robots to motivate children with developmental disabilities, augmented reality displays to communicate scientific information in museums, and tangible interfaces for storytelling using physical objects (see, for example, [11]); studies about the educational value of authoring interactive storytelling in formal education, or how children can develop narrative skills by programming interactive storytelling games; and studies about using interactive storytelling for helping children with special needs. Garzotto, et al, provided a survey of papers related to interactive storytelling. [8]

Enabling Stories

Enabling Stories is a multimodal interaction-based interactive storytelling application, designed for children with motor or cognitive disabilities, as well as for children with normative development, that runs on OLPC’s XO computers.

The application is not only an interactive storytelling game but also a tool for the stimulation of the development of specific cognitive functions and skills, as well as for the promotion of digital inclusion, and the improvement of social, emotional and motivational aspects on its users.

Enabling Stories performs the role of the storyteller. It is responsible for directing the story plot by modeling key aspects of the narrative, such as characters, places, possible interactions between characters, etc., and to present the story to the user.

Interacting with the storyteller –by setting scenes up or by answering to specific questions– the user is allowed to modify the story, building new ones on real time, and therefore actively participating in the drama. For example, the storyteller would describe a scene by using spoken text, images, music, video, etc. and then prompt the child for information. 

The application supports an input modality based on the interaction schema mentioned before, as well as traditional input gestures, such as selecting a displayed option with the mouse. When an event requiring user interaction occurs, the storyteller waits for the user input. A rather trivial example would be: “the girl is taking a hike in the forest, who does she meet?”, the child, then would take one of the printed images (let’s say, one with the drawing of a dog), and puts it in the space seen by the computer’s camera. The storyteller then continues with the narration using the user input. Depending on the type of event, the storyteller could time out the scene and then continue with the following one.

Conclusions and future work

Uruguay’s OLPC program, Plan Ceibal, aims to provide a connected computer to every child and teenager of the country. Different studies show that the provided computers are not accessible for children with motor or cognitive impairments. To foster the appropriation of these computers, we proposed different types of applications that explore different educational strategies, based on an interaction schema that avoids the traditional “keyboard, mouse, screen” paradigm.

One of these proposals, named Enabling Stories, is an interactive storytelling application supported by a multimodal interface. Enabling Stories is aimed to develop children’s motor and mental skills, enhance their self-esteem, improve their autonomy, and favor social relationship between peers and adults. 

Beyond the application characteristics that make it attractive, we speculate that children with motor impairments can benefit from the physical and cognitive challenge involved in the action of showing the printed images to the computer.

Future work

A new stage of the work has commenced in June 2011, under the project “NEXO: New Interaction Modalities for the XO”. The project involves the study of the incidence of different types of applications, including Enabling Stories, in children development.

The study will be conducted by UDELAR graduate and undergraduate students of cognitive psychology and computer science, during a six-month period.  

This stage includes the design and development of different types of applications, focused on developing cognitive functions and skills, enhancing children’s social domains, and improving emotional and motivational aspects.

Within this study we expect to evaluate if the proposed applications effectively represents a positive impact on different developmental domains (physical, cognitive, social, or emotional domains).

References and Notes: 
  1. J. P. Hourcade, et al, "Early olpc experiences in a rural uruguayan school," in CHI Conference on Human Factors in Computing Systems, Florence, Italy — April 05 - 10, 2008.
  2. T. Vázquez, Americas Quarterly [Online], 2009, http://www.americasquarterly.org/node/370 (accessed September 2011).
  3. D. Hirschfeld, SciDev.net [Online], October 2009, http://www.scidev.net/es/news/uruguay-logra-la-meta-de-una-laptop-para-cada-escolar.html (accessed September 2011).
  4. M. Bonilla, et al, "Designing Interfaces for Children with Motor Impairments: An Ethnographic Approach," in Chilean Computer Science Society (SCCC), Antofagasta, 2010.
  5. S. Oviatt, "Flexible and robust multimodal interfaces for universal access, Universal Access" in the Information Society Vol. 2, No. 2 (May 2003).
  6. M. Hamilton and M. Weiss, Children tell stories: teaching and using storytelling in the classroom, 2nd ed. (Katonah, N.Y. : Richard C. Owen Publishers, 2005).
  7. A. Wright, Resource Books For Teachers: Storytelling With Children, 2nd ed. (Oxford: Oxford University Press, 2009).
  8. F. Garzotto et al., "Interactive storytelling for children," in Proceedings of IDC '10 (New York, NY, 2010).
  9. M. White and D. Epston, Narrative means to therapeutic ends (New York: Norton, 1990).
  10. Y. Cai et al., "Fuzzy cognitive goal net for interactive storytelling plot design," in Proceedings of ACE '06 (New York, NY, 2006).
  11. C. Lathan et al., "Therapeutic play with a storytelling robot," in CHI Conference on Human Factors in Computing Systems, New York, 2001.