The Mixed Reality Performance

In the Mixed Reality performance a poet appeared on a virtual sage and performed to an audience that were located in a remote physical theatre. The awareness and communication between the performer and the audience were entirely facilitated though a mixed reality boundary. This page describes the technical set up and the experience from the event.

Goal
Set up
Experience
Links

Goal

A mixed reality boundary was used to facilitate an interactive performance, where the artist was in a virtual world while the audience members were located together in a physical space. From a technological point of view one of the main objectives of this project was to investigate if a mixed reality boundary would give the performer sufficient awareness of the audience and whether it would support effective direct communication with the spectators. From the performers' perspective a reason to enter a virtual world is that they can entertain their audience remotely. However, this can be suitably achieved with conventional video conferencing technology. Much more interesting is the use of the virtual world to enhance the performance. The shapes, colours and sounds in the virtual world can be designed to complement the delivery by the artist, producing a unique experience for the audience. Additionally, more than one performer and/or selected participants can enter and share the virtual world. In our experimental performance a second actor entered the synthetic space for the ending of the show.

Setup

The event was staged in the University of Nottingham Reality Centre. About 20 spectators were seated in front of a 180-degree screen, which offered the view of the virtual world and the graphical representation of the performer. Three front projectors driven by a Silicon Graphics Onyx machine provided the display on the screen. A video camera placed at top middle of the screen captured the view of the audience. Figure 1 shows this arrangement. The artist, on the other hand, was located in a separate room. His view of the virtual world was presented though a tracked Head Mounted Display (HMD). The artist also had two additional Polhemus tracking devices strapped on his hands which provided the gesture input for the embodiment animation.

Figure 1: set up in the physical theatre

In the synthetic world 3 virtual cameras provided the view for the 3 physical projectors. The live video of the audience, on the other hand, was dynamically texture mapped on a display integrated in the virtual world (see figure 2). This view provided the visual awareness of the audience for the artist. Additionally an audio link between the performer and the spectators was established. The performer had headphones and a cordless microphone while in the audience room 4 speakers broadcast the audio and an ambient microphone attached to one of the centre chairs picked up the sound form the spectators.

Figure 2: overhead view of the layout of the virtual space

Experience

Our initial performance lasted for half an hour and involved the poet Dave "Stickman" Higgins. The stage and poet's avatar were designed by the artist Derek Richards who also joined in the end of the performance as a supplementary actor. The performance began with the virtual poet entering from the wings and improvising a poem while the audience watched. After five minutes the poet directly addressed the audience for the first time, requesting that they stand up and asking them a series of questions. After picking on several individuals in the audience, he then persuaded them all to rise and to clap and chant along with the poem. Figure 3 is taken from the audience space and shows the poet avatar on the virtual stage addressing an audience member.

Figure 3: view form the theatre

The event successfully demonstrated the concept of using a mixed reality boundary for a performance and there were noticeable advantages for the artist. He felt that the use of the graphical embodiment increased his ability and the ease of becoming a new persona. In his opinion the virtual embodiment also gave license to be more daring and controversial with the audience, liking the experience to that of being a ventriloquist and the embodiment functioning as a dummy.

However, a number of factors were revealed which hindered the delivery by the artist. Firstly not all of the properties of the boundary were made explicit. The positioning of the 3 virtual cameras was such that the entire width of the graphical scene was displayed on the screen in the audience room (see figure 2), so the performer was visible anywhere on the front side of the virtual set. But on reflection it seems that it would have been conceptually useful for the artist if the field of view offered through the boundary at the physical side was made more explicit in the synthetic space. For example, floor marking could have been provided which indicates the visible part of the virtual space.

The Mixed Reality Performance

Contents

Goal

Setup

Links

People

Enabling Systems

Publications