NeuroBodyGame: the design of a wearable computer for playing games through brain signals

This paper has as a main objective to present the design aspects involved in the development of the NeuroBodyGame that consists of a wearable computer that allows the user to play games using their brain signs. It is a wireless interface for brain interaction with games loaded into the system. Both games and wearable computer react to the emotion of the user at the moment of interaction.

Author(s)

The advances achieved in recent years through ubiquitous computing, “as the method of enhancing computer use by making many computers available throughout the physical environment, but making them effectively invisible to the user,” [1] are responsible for transforming significantly how men and machines interact. This way, the body presence of the interactor, distended by its gestures, is sufficient to establish the communication flows between these systems, until then only mediated by the physicality of tangible interfaces.

Mouses, keyboards and joysticks make room for the user’s body, free to move about in the digital space miming his actions in the three-dimensional physical environment. Loose or limited movements, gestures and voices construct the expression of the visible body, in a continuous dialogue mediated by the invisibility of intangible interfaces. The configured mobility thus announces another condition for interactive existence in which games stand out as universe of hybrid reality – physical and digital.

This context gains even more dimensions with the addition of concepts from affective computing, in which the machines presenting affective abilities focused in recognition, expression, modeling, communication and in response to user's emotion [2] and bio-interfaces, by providing a differentiated condition for interaction, governed by the user’s biology. [3] In other words, the interactor’s emotion literally comes into play so an interactivity of another nature can present itself.

Within this scope, NeuroBodyGame is a wearable computer that incorporates the concepts of wearable affective computing and of the bio-interfaces (functional biometric interfaces and the brain-computer interface) to provide an organic interaction between humans and games and thus lead these systems to a co-evolutionary relationship in which the games and the wearable computer are changing according to the thoughts and emotions of the users at the moment of interaction. Thus, this paper has as a main objective to present and to discuss the principal design aspects involved in the development of this wearable computer focused on allowing the users to play games using their own organism.

Art, Science, Technology, Fashion and Games: a transdisciplinary design

Initially, it is important to underscore that the creation and development of the entire NeuroBodyGame (NBG) system involves a transdisciplinary team comprised of artists, designers, doctors and engineers, which provides the encounter and unique exchange of knowledge between specific areas of training that present themselves fully articulated and integrated to the product’s final result.

NBG integrates wireless interfaces for brain and biometric interactions with games loaded into the system, which allows the users to play games using their neurological and physiological signals. Thus, the functional biometric interfaces are responsible for checking the player’s ANS (autonomous nervous system) variability and for providing information about his physical status or behavior, gathering the physiological data in a continuous manner, that is, without having to interrupt user activity, in this case, his playability with the game. In this system, two biosensors are used as input channels for the interactor’s physiological data: the galvanic skin response sensor (GSR); and the blood volume pulse sensor (BVPS). The brain-computer interface employed transforms the electrophysiological signals of reflexes from the player’s central nervous system activity into messages to be sent to the game being played. In other words, in NBG, the player’s physiological information acts as data to configure an interaction that corresponds to his organism’s status, specifically at that given moment of his relationship with the game in question.

In order to achieve all ages, two games that are being used with the NBG: NeuroBodyGame Dragon which aims at a light user and has a less complex playability and NeuroBodyGame Car which aims at a more experienced user and presents a complex playability. Both games are open source – a fundamental characteristic for providing full remodeling of the programming and integration with the games’ controls and the interactor’s brain and physiological commands.

Like the frequencies of the brain waves, acquired from the brain-computer interface, the player’s following physiological parameters are also read during playability, by the functional biometric interfaces: emotional variability; anxiety control; emotional response; sympathetic and parasympathetic nervous system; functional oxygen; and cardiac frequency. The mapping of these brain waves and these parameters are done and associated in real time to game features, which begin to react in accordance with the player’s physiological state. In other words, the feedback obtained by the interactor from the game as well as from the wearable computer results from his emotional state during the interaction with the entire digital system.

Thus, the playability can get easier or more difficult according to the brain wave frequency and the physiological state of the user at that very moment. The wearable computer interprets the brain activity and the emotional state of the user and reacts to it by changing the colors (back and front) and by applying vibrations (back). A really calm user, extremely careful and focused will have his playability enhanced and the NBG will mostly react by showing the color blue. If the user is just calm and focused, the color displayed is green. A tense user, if a bit unfocused or even nervous, will have his playability worsen and the NBG will react to it by turning into yellow and applying a soft vibration in the area of the back. And a really tense and unfocused user will have his playability worsen and the NBG will react by changing its color to red and by vibrating really intensively.

In relation to the piece of clothing that carries the wearable computer, the starting point for its investigation and creation follows the same principles that govern the conception of pieces for a fashion collection, in which for defining its formal elements, the following aspects must be considered through experimental research: fabrics, textures, silhouette, finishing, processing, color, comfort and adaptation of these to the design for each piece. In this sense, the project procedure for the NBG proves to be identical. However, differently from a piece for a collection, in NBG, the project does not focus all its efforts in external appearance of the clothing, but rather in the structuring of its interior. In other words, it focuses on the incorporation of a series of necessary devices for operation of the wearable computer, which requires specific and appropriate adjustments for their implementation and consequently for obtaining satisfactory results.

The objectives desired by NBG, as previously described, require it to have a series of cables and electrical wires; LED tapes, rechargeable batteries, vibrators/massagers; keyboard; sensors and electrodes, for acquiring vital signals; as well as a controller system for all information acquired and interpreted in real time, which enables user-computer interaction. For such, as a first step, the guiding concept for distribution of all items in the NBG lies in considering that its internal structure reproduces the interior of the human body, taking the brain as the main point from where the nerve stimuli responsible for body functioning depart and arrive. That is why, the electrodes and sensors responsible for measuring the user’s neural and physiological signals are located in the front view of the wearable computer.

Thus, with the human body as a reference, along the spine and medulla, the NBG reproduces the chain of electrical conductors which allow sending information to other areas of the wearable device. In a second instance, the distribution of other devices and circuits is carried out, allowing the operation of all wearable computer items. In the third step, support points are created for each of the internal components while the final design of the piece of clothing is established. The fourth step, in turn, focuses on the choice of fabrics, which have basic divisions and classificatory subdivisions that refer to the types of each in the set. There are two main divisions: texture – which influences the look of the cloth, the cut and the modeling – and the framework, which establishes the foundation on which the yarns are supported and indicate the sequence the loom will work, providing them with different aspects and uses. [4]

In order to build the NBG, different textile structures can be considered for receiving all the components mentioned above. At the same time, the user’s comfort must be taken into account during the entire process for installing this wearable computer since poor adjustment of the piece to the player's body can hamper playability. Furthermore, NBG’s design aims at uniting comfort, sensations and emotions of the user, also related to the tactile experience with the wearable item, because “although touch is not exactly an emotion, its sensory elements induce neuronal, glandular, muscular and mental alterations that together we call emotion.” [5]

Thus, the design of the wearable item considers the application of three raw materials that, together with modeling, unite the intrinsic aspects of its making, comfort and esthetics. Thus, the first fabric, besides defining the inside of the piece of clothing, also constitutes the structure on which all circuits and devices are supported. Also, considering the need to carry out their maintenance, a second fabric is used to accommodate and provide stability to the applied components, preventing them from slipping during use of the NBG. And the last fabric, while granting the appearance of the wearable computer, also provides for the light emitted by the LEDs installed inside to be revealed without exhibiting the devices and circuits that comprise the interior of the piece.

For the process of choosing the raw materials, which meets the needs of the wearable computer, it is necessary to observe that the weight, fitting, elasticity, movement, adherence and texture are qualities of the fabrics as are the color, print, weave, structure, shine, opacity or transparency, flexibility and malleability. [6] Thus, these textile characteristics or qualities, which also deal with the behavior of fabrics in relation to touch, guide the definition of materials and respective colors used while also adding quality and providing the desired results for the operation of NBG, such as: (1) allow transparency and noninterference in the projection of colored light hues emitted inside the clothing; (2) in its external appearance, translate visual comfort and shine as key needs; (3) the notions used in making the clothing must also be in chromatic harmony. From these definitions, the design of clothing considers the use of materials in their compositions that alternate the use of yarns and fibers like cotton, polyamide, polyester and synthetic foam - the latter responsible for the piece's structural base.

Associated with that, a concept of significant importance and which also permeates the making of the piece as a whole is found in the possibility of it being used by bodies of diverse biotypes. For such, the shape of NBG was defined based on a vest, since it permits adjustments to different bodies by extending or reducing the piece using adapters and Velcro applied to the internal face. It means that the wearable computer can be expanded or contracted in order to fit the user’s body. Its main challenge lies in the fact that it tries to preserve the user’s comfort. Once each and every possibility of discomfort may alter the neurophysiologic signs and by doing so, it would compromise the organic information acquired. The same criteria was used in the composition of the strap that is attached to the head – for acquiring the player’s brain signals – and also considered for designing the wrist, where the keyboard used for the game is installed.

Another important moment to emphasize, considering its impact on the design of the wearable computer, guiding the making of needed adjustments, involves the usability tests conducted with NBG users throughout the entire process. These tests include the analysis of aspects related to comfort, mobility, and adaptability of the user to the functional biometric and brain-computer interfaces and the integration of physiological functions of the player's organism to the functionalities of the game in question.

Also, the player/NBG relations observed during interaction of a significant number of users – more than 5,000 – during exhibit of the wearable computer at FILE (International Festival of Electronic Art) 2010, leads to the conclusion that the use of biological information from the interactor to configure organic playability with the games constitutes a fertile field of research, considering the immersion potential the brain-computer and functional biometric interfaces provide when associated with affective computing concepts and applied to interactive digital systems, such as computer games.

Game Training and Neuro Action Stations

NeuroBodyGame’s design also extends to physical space, taking into account greater immersion of users with the proposed games. Conceived as two different environments, also called Game Training and Neuro Action stations, respectively, its interactive space configures an installation.

In a first moment, and still without use of the wearable computer, the interactor learns about the two games. This occurs in the first station, Game Training, which consists of a space dedicated to necessary training for the user to become familiar with the games indicated before interacting with the wearable computer. This training occurs directly on a desktop computer.

Neuro Action, in turn, consists specifically of user interaction with the games through their brain and physiological signals, based on use of the wearable computer. In this instance, the games are designed on an 80-inch screen that integrates game visualization to neural activities and other physiological parameters of the player that are being acquired by NeuroBodyGame during playability. The projection is made at a comfortable height for game visualization by people of different heights, meeting the ergonomic parameters and also keeping a minimum distance between the screen and the interactor.

Thus, from all these aspects presented and discussed above, in our future studies we consider the design of wearable computers that provide increasingly complex levels of interaction between the user's organism and the elements that constitute a game - characters, scenarios, feedback and playability - in order to configure effectively co-evolutionary communication between both systems: biological and technological.

References and Notes: 
  1. Mark Weiser, “Some Computer Science Issues in Ubiquitous Computing,” Communications of the ACM 36, no. 7 (1993): 75-84.
  2. Rosalind Picard, Affective Computing (Cambridge, MA: MIT Press, 1997).
  3. Rachel Zuanon, “Bio-Interfaces: Designing Wearable Devices to Organic Interactions,” in Biologically-Inspired Computing for the Arts: Scientific Data through Graphics, ed. Anna Ursyn (Pennsylvania: IGI Global, 2012).
  4. Gilda Chataignier, Fio a Fio: Tecidos, Moda e Linguagem (São Paulo: Estação das Letras, 2006), 43.
  5. Ashley Montagu, Tocar: O Significado Humano da Pele (São Paulo: Summus, 1988), 131.
  6. Andrea Saltzman, El cuerpo Diseñado (Buenos Aires: Paidós, 2004).