It’s Behind You: The parameters and process in the creation and presentation of panoramic moving images

During September 2010 ICCI (Innovation for the Creative and Cultural Industries) University of Plymouth, UK, organised a 360ْ film, arts and performance festival in Plymouth city centre. Using the festival as a case study, this paper reflects on the parameters for and the processes involved in the preparation of creative content for the festival, particularly focusing on the issues and concerns pertinent to 360ْ film making.

Author(s)

During September 2010, ICCI (Innovation for the Creative and Cultural Industries) and the University of Plymouth, UK, organised a 360 degree video film, arts and performance festival in Plymouth city centre. The rational behind this festival was to revive and re-engage with the popular nineteenth century tradition of touring panoramas. 

Drawing on this traditional context, the ICCI festival also aimed to investigate the potential of panoramic spectatorship, utilising new innovative technologies of a projection screen format of six metres by twenty metres in diameter, high quality digital HD projection, digital surround audio and a performance space/auditorium housed within a demountable dome structure

The concept of the festival had been developed in line with the aspirations of LOCOG 2012 Cultural Olympiad in the South West of England; exploring the potential of a touring venue for the display of media and performance content. Using the festival as a case study, this paper reflects on some of the parameters and criteria established for the preparation of creative content for presentation within the Arena, particularly focusing on the issues and concerns pertinent to 360 degree film making. To do this we will review the display and presentation of film within the festival’s 360 degree auditorium from three key aspects. First, we will reflect on factors that relate to the audience experience of panoramic content. Second, we will discuss how particular spatial and visual environments effect both the production and experience of presented work. And third, we will look at how these factors might be understood within a particular context.

As a result of the growth in panoramic photography [1] and recent developments in 360 degree video [2] there have been an increasing number of practitioners internationally who have been exploring this format as a creative environment. However, until recently the majority of the work produced has been limited: content has usually been displayed as interactive panoramas for computer screens, employing a computer mouse or touchpad as a means to navigate a 360 degree photographic or video space that is normally presented as a spherically mapped environment on a single screen. An emphasis for the festival was placed on the presentation of panoramic photographic imagery and video filmmaking rather than computer CGI animation, although examples of all methods were displayed.  

The primary position for viewing the 360 degree image is from the centre of the auditorium. For the filmmaker and the film editor, this geographical positioning of the audience is both exciting while at the same time potentially problematic.  The excitement emerges from the prospect of being able to place the viewer within the action - a step closer to Borger’s map, [3] the replication of reality that might be considered a holy grail for certain artists, particularly those filmmakers seeking to comment on, experiment with and play within what is real and its representation.

A simple and perhaps obvious notion is that the image is surrounding the viewer in a way similar to the viewer’s experience of ‘real life’ and therefore the 360 degree experience could be considered to be more realistic than other methods. This concept of immersion is perhaps where the audience position becomes problematised, in that the viewer in real, not filmic, life is perceived not as being immersed in something but participating in an experience of the world from a particular place at a particular time.  Immersion within a constrained environment, be it the virtual environment of a screen or screen goggles or the more geographical positioning within an enclosing visual environment like the 360 degree dome, is not the same as the more everyday experience of ‘reality.’ The constraint is the frame within which the experience purports to offer a new experience or perhaps understanding. A principal paradigm of film viewing and editing is the position of the viewer in relation to the screen, which is determined by the viewer’s location in the arena and the direction they are looking; there is always a front and back to the screen area (although, during the ICCI 360 Festival some viewers were pressed against the wall, to gain better visibility or view of the whole screen). From this position the viewers witnessed distortion that mitigates the illusion generated by the 360 degree image. The geographical position of the audience determines their perception of the illusion; ideally they need to occupy the central space of the auditorium for optimal viewing experience. Inevitably this limits the audience members who have an optimal experience, as central auditorium space is limited no matter how large the auditorium. 

Deriving perhaps from the proscenium arch of the theatre, we look into screens as we look into a book to read.  Early panoramic paintings relied heavily on the inclusion of architectural structures, either ‘faux terrain’ or painted, to frame the images they depicted.  The frame afforded by the screens represents a single field or point of focus and for the most part this is taken for granted in viewing. One of the most dramatic adjustments that needs to be considered when producing - or for that matter experiencing ­- 360 degree cinema, is the idea that, although one is working with a much wider frame, the viewer is not able to see and experience everything at any one time. While, particularly for a standing audience, there is the opportunity to move one’s eyes, head and body to experience the complete screen, the viewer, schooled for many years of image consumption through various sizes of flat screens and by every filmic example previously experienced, holds onto single screen mentality and may struggle to reduce the screen horizon into a watchable frame. Unfortunately, in doing so elements of the presentation may go unseen behind the viewer.

This 360 degree horizon within which the audience is placed seemingly has the potential to offer unlimited opportunity to encapsulate a scene, an environment or a moment in time. In the natural landscape, our perception of the horizon is most usually associated with an unlimited field of vision. We might experience awe in landscape vistas, perhaps due in part to the prospect of infinity, outside of and beyond where we are standing Other factors possibly mitigate against a similar perception within the Arena.  For example due to the organisational constraints, the base of the viewing screen within the 20 metre diameter structure was placed 2 metres above ground level; the top of the screen was therefore 8 metres above head height. Our evaluation of this configuration determined that the screen required the audience to continuously look up to experience an on screen display. During the previous tests, conducted in a smaller 12.5 metre diameter structure the screen base was 1.4 metres above ground level with a 3 metre high screen terminating 4.4 metres above ground. The lower screen configuration and perhaps the smaller space provided an improved sense of being immersed within the film or photographic environment. With the larger diameter screen, the experience of gazing at panoramic views of the sort alluded to above, one was not awe-struck in the same way since their promise of infinity was quite pragmatically restricted to the geography of the structure, the horizon of which ends with the screen. Yet an idea or expectation of unrestricted horizon is built into the idea of the 360 degree image.  This duality of ideas and experience is an interesting and unexpected conundrum of the 360 degree processes. The Projection within the 360 Arena offers a sense of an immersive experience but is also a framework for construction of that experience - the viewer has work to do.

It might be said that an important function of the relationship between viewer and screen in cinema is the viewers’ anticipation and expectation of ‘what next.’ The events that are unravelling on screen pre-suppose a series, with one event or situation replacing another. As viewers we are in a state of expectation that will be relieved by the appearance of ‘what next,’ but this relief is immediately replaced by a new ‘what next.’  What if the ‘what next’ might be happening behind you?  What if the ‘what’ also brought with it a question of ‘where?’  What if it were possible to have protagonists between scenes appear at different parts of the auditorium, within the horizon of the 360 degree screen?

In allowing for the illusion of placing the viewer within the geography of the action, the filmmaker has also to consider the time it might take the viewer to shift position to see something happening behind them.  To an extent the filmmaker has lost control of the framed image and cannot predetermine the focus of the viewer’s attention. In this respect, the production and editing of a 360 degree work allows for new ideas in editing that perhaps use things the viewer can see; such as movement, colour and contrast. And in addition to this, devices the viewer cannot see (but might hear). Teasing, perhaps. Does the video editor allow for a time lag in cuts/transitions so the viewer might adjust their viewing position, or run the risk of the viewer missing parts of the action?  Should a shift in action and visual focus around the 360 degree screen be signalled, indicated or aided perhaps by a shift in sound – particularly if one is working with surround sound?

The editing of all film and animation for the festival took place using a five screen linear format in preparation for the five-projector display method that was employed in the 360 Arena. This was not an ideal solution; inevitably screen three became the centre screen for editing purposes with screens one and five (which in projection would be joined) at opposing ends. The consequence of this linear format highlighted a number of interesting editorial issues. There was great pressure for screen three, due to its central position to become the focus for editorial decisions particularly those based upon visual composition. The natural and human desire for symmetry during the editorial process became a factor that had to be consciously overcome if the link between screens one and five was not to appear disjointed and the images were to retain the benefits of a 360 degree layout. A number of the works composed of still images used the stitch process of blending a number of images to produce a whole panorama.  This stitch effect was also used in a montage way by a number of filmmakers as they compiled a number of different images to produce a whole composite. An example of this would be; “Cortical Songs,” movements one & two, by John Matthias and Nick Ryan, (Matthias and Ryan 2008) performed by the String Ensemble of Trinity College of Music, conducted by Nick Pendelberry. [4] This 360 projection piece constructed by the co-author David Hilton was made up of single camera recordings that were edited onto 5 separate screens to remain in sync with the music.  The effect was successful, largely due to the splendid camera work of Robin Cox directed by Andrew Graham Brown, which deployed balletic craning and tracking camera motions.  Perhaps because the orchestra was not seen in the round there was no single geographical centre for an audience member to experience the film, instead he/she was able to access the separate screens within any part of the circular screen that was visible to them. The structuring of the of the work started with all screens showing the same image repeated, following the expectations associated with viewing a single screen. Gradually different images or camera views were introduced on separate screens. As the work progressed, each screen would present a different view of the performance with close ups on performers from different parts of the orchestra. In this way, the editing worked with a narrative both of the progression of the musical piece and the audience’s engagement with it. Gradually, the audiences were encouraged to see what might be happening behind them and make full use of the 360 degree environment. [5]

The Ladybug 3, [6] 360 degree video camera was used for a number of productions presented within the ICCI 360 arena. This device captures video using six cameras: five around the horizon and one above to produce 80% of a hemispherical image. Within the ICCI Arena, only a cylindrical slice of this image was used for projection on the wrap around screen, the screen did not extend to the full dome ceiling. The Ladybug 3 offered the obvious choice for film that recorded full 360 degree panoramic video imagery. Only a few of the works used the Ladybug 3 as the sole source of imagery.  Perhaps the most effective use of the camera was in the production of Panoptica by Craig Whyte, Andy Banks and Jo Plant. In this work, the producers combined live action single camera shots with live action shots using the Ladybug camera and C.G.I. effects and environments within which the protagonists of this science fiction film interacted.


The ICCI 360 screen marries both theatre and cinema and creates an entirely new experience. In conceiving the 360 film. I first thought about early cinema, largely due to the technical limitations the 360 Ladybug 3 camera created. For instance, in early films such as those created by the Lumiere brothers, all the action took place within one static shot with few cuts. In this sense, shooting on the Ladybug is the same. But it also connects the medium once again with theatre. The camera becomes the audience and the actors perform their roles within a single space. In this case, you’re working on a 360 stage. [7]

The conceptualisation of the space when planning action within the 360 arena also presents problems with the design and construction of the work due to the fact that one cannot see it as it will appear, except in the 360 environment.


The best way to do it was by doing a floor plan - a bird’s eye shot to see how we use the space using the 360 camera - I don’t think we would have been able to do it without the plan.
There were certain scenes that worked in the film – for example the ventilator scene, that was a bit surprising, you have the main character who has escaped and ends up in this vent shaft and when you watch it as a single screen it is quite boring - but we were quite surprised watching it back in the 360 how interesting it was because he was going all the way round the 360 degree screen crawling in that space and that is the first time the audience actually get to just to see that main character. [8]

 
The process of production is not automated and no matter how inventive in its conception each work was the result of painstaking production processes. The degree of detailed work involved in production is perhaps exemplified by Udo Hudemaier’s piece City of Plymouth produced entirely using still images. Each image underwent a high degree of processing; particularly evident in the shifting of perspective as each window of a high rise office was seen square-on without perspective distortion while its setting flew up and down the buildings, an effect produced by photo manipulation one frame at a time. Other producers had different problems but each had to devise a strategy for maintaining alignment of individual images to create the synchronous whole image on projection. Perhaps two methods most used were to assemble the whole 10:1 ratio panoramic image in a programme such as Adobe’s “After Effects,” necessitating use of the flat one screen image to represent the enclosing panorama. Another method was to fragment the sections of panoramic images into five single screens, which then were augmented with other footage and effects – one screen at a time. In this latter method the sense of the wrap-around panoramic image or ‘suggestive image space’ of the work was more difficult to sense during the edit. [9] In all cases the only time the work was seen in its intended form was during the exhibition inside the dome.

Conclusion

The experience of preparing creative works for projection within the ICCI360 Arena has raised many subjects for further research. In this paper we have identified just a small number of these subjects and have outlined: 

The large numbers of international creative practitioners who are exploring the 360 degree format are limited to the presentation of their products as navigable spherical images presented within the frame of a computer monitor. Within the computer screen one notionally has complete access to 360, but the computer screen always frames this. Whereas the frame established by the screen in the dome seems to be unframed, it is in fact the dome itself, which provides a frame: the 10:1 aspect ratio of the projected image is a very narrow window. This could be construed as a possible inhibitor of an immersive experience. The 360 degree projection environment presents further possibilities and problematics for this work.

The significance of the size of viewing space and the position and size of the viewing screen relative to the audience is an important mediating consideration.  And further, the importance of viewing the 360 degree projection from the centre of the display for an optimal experience of the panoramic image necessitates a restriction in the size of audience. The scale of the immersive experience may be a factor for further exploration with regard to the 360 degree horizon and the height of screen: looking up to horizon or down upon it. [9] If the projected visual horizon is not at eye level the viewer’s ability to relate to landscape is mitigated. If the image horizon is above the viewer it is related to as a picture on the wall, but if lower than eye level, one is more likely to be immersed and encounter a feeling of presence: the viewer needs to look ‘at’ or ‘down on’ and not ‘up to’ the image.

Of the many issues in production for the 360 environment, we should identify the following as of particular concern for future exploration: In constructing action and narrative sequences that employ the full scope of the 360 degree screen one runs the risk of the audience missing parts of the action which might take place behind, where their attention is focused. Should producers allow time for the audience to change their focus and indications - perhaps using sound to suggest this? Other devices include tracking action around the 360 degree screen by isolation of certain key points of attention such as actors/figures moving or the de-focusing or blacking out of other parts of the action. Currently the editing process requiring single screens to be laid out in a line of five may lead to over use of the symmetrical structuring of screens and a focus based on screen three (which is in the centre of the edited image) and screens one and five becoming disjointed.  The degree to which one might employ a full 360 panoramic image in relation to images montage - constructed of 5 single screens - also affects the experience of the work. It does seem crucial to the development of this form, that editing facilities are developed that permit visualisation of the work during editing.

The gap between what is in front and behind and the audience’s focus of attention represents an opportunity to exploit and explore a conceptual space that might be described as an interpretive space, and this provides a creative opportunity for future development.

References and Notes: 
  1. Google Earth, www.google.com/earth and 360 Cities, www.360cities.net (accessed September 2011).
  2. 360 Video, http://www.360video.com/gallery (accessed September 2011).
  3. J. Baudrillard, Simulacra and Simulation (Michigan: University of Michigan, 1994), ISBN: 0472 06521 1.
  4. J. R. Matthias and E. N. Ryan, "Cortical Songs," NonClassical Recordings, London (2008) .
  5. D.Hilton & J.Matthias, Inside Out, the Cortical Songs experiment, University of Plymouth, UK. AVANCA/CINEMA International Conference of Cinema, Portugal. (2010).
  6. Point Grey, www.ptgrey.com/products/ladybug3/index.asp (accessed September 2011).
  7. J. Plant, Co-producer Panoptica. Email correspondence, November 2010.
  8. C. White, Co-producer Panoptica. Interview, November 2010
  9. O. Grau. Virtual Art – from illusion to Immersion, revised and expanded edition (Cambridge, MA: The MIT Press, 2003), 151.