Play with Fire | A Real-Time Video Experience for Sustainability

Play with Fire is an interactive art installation that proposes participants to ignite generative fires over live streaming video of selected forests. This experience paradoxically encourages playing with fire to stimulate awareness and prevention of fire related damages to the forests. Our goal is to raise awareness for human causes in forest fires and effect attitude change towards environmental protection.

Author(s)

Play with Fire is an exploratory research project that proposes an interactive installation to engage the audience senses in unconventional ways. It is a performative, immersive experience that invites people to interact with real-time video from selected forests by playing with virtual fires through gestural interaction. 

We envisage the installation triggering controversial feelings by combining the "beauty and danger" of a forest on fire. This duality becomes part of the experience, and raises concerns in the audience mind, such as the pleasure and excitement of playing with fire versus its effects on a natural resource such as a forest. The experience concludes with visuals of a forest virtual regeneration process underlining the message: the forest will eventually grow again, but what is the price to pay?

In order to foster awareness and stimulate activism, we decided to conceive and design Play with Fire as a digital art experience that happens over three different moments: an introduction to the installation and its theme with an invitation to interact with it; the performative active part where the participant engages with the gestural interface, and a reflective part taking the form of a mobile application, which will stay with the participant for a long time after the installation experience.

An ARTiVIS experience

Play with Fire is part of ARTiVISS (Arts, Real-Time Video and Interactivity for Sustainability), an exploratory research in digital media related to sustainability [1]. 

At the intersection of Art, Science and Technology, the research engages a multidisciplinary team that follows a collaborative approach, where artistic practices are supported through technology. Regarding the impact and the potential of art and technology on society and the environment, we aim at creating a “forests showroom” experience through digital media, in order to inspire change in the lifestyle of the public. The outcomes of ARTiVIS include an online platform, interactive installations – Hug@ree, Treeellucinations, B-Wind!, Enchanted Forest, MAicro, and this Play with Fire experience – and the design of a DiY surveillance kit prototype.

Motivation

Living in places that have always been extremely exposed to forest fires, makes us very sensitive to the destruction of the forest patrimony by fire hazards, which also applies to a world scale. This interactive experience paradoxically encourages playing with fire to stimulate awareness and prevention of fire related damages to the forests. Ultimately, we seek to pose a constructive approach to the destructive dynamics of fire that aggravate climate change. Can digital art foster awareness and respect for nature?

Related work

Play with Fire is a transdisciplinary project and, as such, requires a diversity of references. Involving digital media, the environment, and real-time video, this interactive experience congregates references from the Arts, Sciences, and Technology. The selected case studies presented in this section are examples that in some way inspire the concept, its features and future developments.

Climate change and surveillance are very relevant and currently discussed topics in the areas of digital art and design. The role and potential of design research in the transition towards sustainability is being discussed in mainstream design and digital media art events, such as “Repair” and “Goodbye Privacy” at Ars ElectronicaChanging the Change [2], and Transmediale putting the threat to the sustainability of our planet in perspective with “Perish in Beauty? Climate Change as Cultural Demand”. The interactive installation Play with Fire also takes us into forest fires issues, specifically: distribution, detection, effects, causes, consequences, prevention, and forests surveillance.

Digital Artists such as Tiffany Holmes, have since a few years engaged in eco-visualizations projects [3] in order to sensitize audiences towards sustainability issues and climate change. Our approach enhances the eco visualization aspect of such an engagement with the activist component triggered by the gestural interface, which reflects the participants actions directly on the real-time video of the forest.

Interactive experiences engaged with the use of technologies are increasingly embodied in video based environments. Whereas real-time video has been mainly used as a functional tool for surveillance, for informational and safety purposes, the use of this resource has an enormous potential for artistic exploration. Some initial steps in this direction have been made by projects such as Funky Forest, an interactive ecosystem where children create trees with their body and divert the water to the trees to keep them alive [4]. Parque is also an interactive installation with an ecological message: “the growth of a forest is determined by the amount of attention it receives”[5] – the system recognizes vertical movements as inputs for making trees vector graphics grow. Petros Vrellis’ Fire installation [6] poetically displays the compositing of generative fire propagation and destruction effects over static backgrounds and Float4’s FireFX demo shows how performative and responsive generative fire effects can become [7]. Hand from Above [8] is an experience that playfully challenges our perception of spaces and objects, enabling virtual and real to coexist in real-time, demonstrating participants' immediate engagement, and it also evidences how determinant scale is.

The Games for Change project hosts an archive with many good examples of the expressive power of the medium for environmental awareness [9]. The work of artists like La Molleindustria [10] that appropriate and subvert videogames as an interactive medium for persuasive purposes has also influenced our work.

In Play with Fire the participant is confronted with the information of the burnt forest together with the images of it. Regarding visual effects, we can find virtuous examples of fire effects with cellular patterns in Fedkiw's computational fluid dynamics [11] and Horvath’s high quality artist-directable GPU implementation [12].

Finally, the inspiration towards acting rather than watching comes from compelling and controversial movies such as Stanley Kubrick's Clockwork Orange (1971), and David Fincher's Fight Club (1999). These works stand out as references to recall regarding the construction of experiences that subvert reality and common sense in order to convey a message.

Interactive installation setup

In Play With Fire, a surveillance camera will be setup at a selected forest location, transmitting real-time video to the installation space, a room inside of which the performative gestural interaction is contained.

One semi transparent wall is where the mash-up media and information about fire hazards and forests will be displayed. From the outside, the participants are seen as silhouettes of the arsonist – “fire wizards” performing inside the installation room forming a choreography with the changing media displayed. This is also functioning as an attractor for the audience to enter the interaction space. When the participant enters inside the installation room, the system asks him or her to place his/her mobile device on a special deck. Ready to face the screen, the audience member is presented with the start of a fire, triggered by positioning the phone over the deck, and he/she can get ready for the performative interaction.

The participant faces the main screen. A whole body gestural interface based on a depth camera captures the participant's movement. He/she goes straight into the interaction with the real-time video of the selected forest projected on the whole large scale wall. The sound of crackling fire attracts attention to the action of the fire and how it is spreading. As the user is prompted to act upon the virtual fire, it will react through real-time generative graphics and fire animation according to the gesture just performed. Intuitive and natural gestures for controlling fire (starting it, growing, concentrating, moving it, extinguishing) have been tested and selected in order to map the user engagement, still allowing further performative explorations.

After a certain amount of time, the fire takes over the trees and forest and develops its own behaviors. If at this point the audience does not intervene by performing gestures trying to extinguish the fire, the forest fire climaxes to a point of no return. Fire flames follow the trees structures. Fire has developed its own behavior and spread through the forest. The scene becomes a spectacle for the audience. Once the fire has climaxed, the amount of virtual damage to the forest will depend on how much the audience has played with the fire, how much it has watched the spectacle of burning trees go on and if he/she had ever tried to stop the flames from taking over. A desolating burnt landscape is the resulting scenario.

The interactive experience ends with the participant being presented with a screen showing information of the resulting damage in the style of a game score: as a measure of the damage generated in the Play with Fire interaction along with a snapshot and general data about how the performance fire damages, regeneration times and possible effects on climate change.

The participant picks up the phone from the deck where he/she had placed it when entering the room and is prompted to open a URL that contains a Play with Fire mobile web application that displays the damaged forest the participant leaves behind and its slow regeneration process. The duration of the forest regeneration and the length of time the application will accompany the participant will in fact depend on the data generated during the performance. Participants will carry it as a memento of their experience for a duration determined by the real-time interaction with the installation.

Technical challenges

The implementation of this experience as described above brings some important technical challenges, namely the capture of user gestures, the real-time rendering of fire effects, the illusion of destroying a forest over a real-time video stream, and the mobile component of the experience.

Capturing complex user gestures with fidelity and low latency is a challenging prospect. During the project’s pre-production we developed a library of gestures to be discovered by the participants that would trigger specific behaviours of the fire simulation. Although these gestures involve full body movement they can be fully described by arm motion. Taking this into account we are using a Kinect 3D depth sensor and mapping user arm movement to TUIO input. This allows us to prototype the installation interaction using common multitouch hardware and software tools.

An impressive and responsive real-time fire simulation is crucial for user immersion into the experience and hard to get right. We are using Horvath’s high quality artist-directable fire simulation model developed for film special effects [12] and implementing it in real-time.

Virtually destroying a remote forest presented as a real-time video stream presents some interesting technical challenges. First there’s the need to extract approximate tree structures from the video to build a “fuel map” where the fire simulation will take place. This is complex to do in a fully automatic way, so we’ll be using a semi-automatic process where the rough tree strutures are drawn over the feed in a calibration step and then the optical flow of image features is used to deform these structures over time.

The next challenge is how to composite the effects of fire destruction over the streaming video. We will have to take some artistic license in the rendering of this destruction. We looked into Melek’s work on rendering fire damage [13] and start by blending a black matte over the fuel map to simulate the charring of the wood, and then add smoke, ember and spark effects where appropriate to add some visual impact to the result.

One last challenge that presented itself during development was the uploading of a native mobile app to the user’s phone. This approach presented problems related to mobile phone security and having to develop and support a different app version for each phone. The solution we found was to make the mobile application a web application, that would only require that the participant’s phone have a web browser in order to check his “score” and the regeneration state of his forest.

Conclusion

Play with Fire as an artistic experience has been designed, from the beginning, to be ambiguous, even "wicked". Questions raised by the audience in public presentations have focused mainly on its paradoxical nature: "Won’t Play with Fire inspire people to be arsonists instead of forest caretakers?” In his book “Persuasive Games”, Ian Bogost introduces the concept of procedural rhetoric to discuss how games and interactive simulations can be used to teach a point of view and contribute to effect attitude change [14]. By playing with the system's rules, the users become familiar with it and gain a deeper understanding of its mechanics, allowing them to confront their assumptions and beliefs with this new understanding, through a process of cognitive dissonance conducive to changes in personal attitude [15].

We have designed this interactive experience for persuasive purposes, inviting people to engage in forbidden and dangerous actions in a controlled environment, in order to confront them later with the long term consequences of their own choices. In Play with Fire, participants are initially invited to experience a realistic model of something forbidden, usually outside their scope of possibility. They are allowed to experiment with what is wrong as a learning experience in a game-like environment. This (so called) magic circle [16] – where the consequences are negotiable, played in a protected environment – works not as reality abstraction, but as reality protection. 

Collaboratively developed by artists, activists and technologists, Play with Fire is an innovative approach - with a challenging technological component - that comprises a strong dimension on social and natural sciences converging New Media Arts and Sustainability. After the conception, design and initial basic prototyping of Play with Fire, the main implementation of the interactive installation is currently underway and is scheduled to premiere in the Fall of 2011.

 

ACKNOWLEDGMENTS

This research is done in the framework of the PhD in Digital Media of the UT Austin-Portugal Program funded by FCT (ref. SFRH/BD/42555/2007). We thank CIEBA for supporting conference participation. We also thank AZ Labs for the project development support.

 

References and Notes: 

 

  1. Mendes, M., Correia, N., (2010) RTiVISS, MM'10 Proc., ACM Multimedia 2010, Firenze, Italy
  2. Cipolla, C., Peruccio, P. (2008) Changing the Change – Conf. Proc., Torino, Allemandi, Conference Press
  3. Holmes, T. (2007) Eco-visualization: combining art and technology to reduce energy consumption, C&C’07 (2007)
  4. Watson, T., Gobeille, E. (2007) Funky Forest, http://www.theowatson.com/site_docs/work.php?id=41 
  5. Madeira, R., (2009) Parque, http://vimeo.com/4789754
  6. Vrellis, P, (2010) “Fire”, http://www.youtube.com/watch?v=nKNm_VosA3M 
  7. Float4, (2009) “FireFX”, http://www.youtube.com/watch?v=DJ31zxr21Mc
  8. O’Shea, C. (2009) Hand from Above, http://www.chrisoshea.org/hand-from-above
  9. Games for Change, http://www.gamesforchange.org
  10. La Molleindustria, http://www.molleindustria.org
  11. Fedkiw, R., (2011) Computational fluid dynamics research, http://physbam.stanford.edu/~fedkiw/ 
  12. Horvath, C., Geiger, W, (2009) Directable, high-resolution simulation of fire on the GPU, ACM Transactions on Graphics,  ACM
  13. Melek, Z., (2007) Interactive simulation of fire, burn and decomposition, Texas A&M University
  14. Bogost, I., (2007) Persuasive games: the expressive power of videogames, MIT Press 
  15. Festinger, L., (1957) A theory of cognitive dissonance, Stanford University Press
  16. Huizinga, J., (1966) Homo ludens: a study of the play-element in culture, Beacon Press