Scripting an immersive future for theatre-goers

“Imagine your coffee table turning into a stage upon which you can watch a current theatrical show play out – that is the sort of thing we are looking into.”

So says the Royal Shakespeare Company’s (RSC) director of digital development, Sarah Ellis, explaining one of the ways that the theatre group is currently exploring what it means to perform live, using emerging technologies such as virtual reality, augmented reality, and mixed reality.

The RSC is the creative lead in a two-year project that kicked off this year and sees 15 organisations collaborate in a bid to reimagine the theatre going experience using the latest in immersive technology. It is one of three consortia that are sharing £16m of government funding through the Audience of the Future programme, which is exploring the impact on tech on theatre, museums and gaming.

The RSC consortium includes tech giants such as Epic Games, Intel, Magic Leap; research partners and universities DeMontfort University, i2Media Research, NESTA University of Goldsmiths, London and University of Portsmouth; and arts organisations Manchester International Festival, Marshmallow Laser Feast, Punchdrunk and the Philharmonia.

“We are collaborating and have a series of R&D where each creative organisation leads on a technical challenge which we want to explore and see if that can come into the theatre-making tool kit,” explains Ellis. “By partnering with the technology companies, we are able to gain from their expertise and potentially work with their technologies in different ways.”

They plan to uncover future opportunities in real-time immersive performance, hoping to change the way audiences experience live performance by making it more immersive.

At first sight, the idea of the Stratford-upon-Avon based Royal Shakespeare Company leading on the project sounds rather unusual. It is, after all, chiefly concerned with performing the work of Britain’s greatest playwright – who lived and worked over 400 years ago.

“By partnering with the technology companies we are able to gain from their expertise and potentially work with their technologies in different ways.” Sarah Ellis, RSC

That said, using digital tools is arguably just a modern-day version of what would have been attempted in Shakespeare’s day, when all the latest technology available in the 17th century would have been used to create huge, one-off spectacles for theatre-goers.

The RSC has always pioneered bringing technology into the theatre, says Ellis. “In 2016 we worked with Intel and The Imaginarium Studios to develop a version of The Tempest that turned Ariel into a digital avatar who could appear to be flying around the stage.”

The Tempest production combined stage props – such as a giant, rotting ship hull that mirrored the look of Tudor ship the Mary Rose – with projections and the avatar to create a three-dimensional, “immersive” experience for viewers.

Read more: What’s next in the world of immersive?

This year, the RSC is also running an exhibition, Digital Diorama, which uses augmented reality to bring scenes William Shakespeare might have taken inspiration from in Stratford-upon-Avon to life. It has worked with Magic Leap on the project.

“It felt very natural to get involved with Magic Leap, who are working at the cutting edge of virtual, mixed and augmented technologies, and the Audiences of the Future project is a natural extension of that,” adds Ellis. “This time we are exploring the many stages available to us to perform and share Shakespeare’s plays with new audiences and communities.”

“We are exploring the many stages available to us to perform and share Shakespeare’s plays with new audiences and communities.” Sarah Ellis, RSC

Virtual costs
It’s not the only theatre group exploring the possibilities for VR, AR and MR. Earlier this year, The National Theatre produced Jordan Tannahill’s Draw Me Close, which used live performance, virtual reality and animation to create a memoir about the relationship between a mother and her son, in the wake of her terminal cancer diagnosis. Mixing theatrical storytelling with technology, the performance allowed the audience to take the part of a protagonist. This kind of immersive VR theatre allows audience members to interact inside of VR in real time with both performers and physical objects to create a sense of deep immersion and presence. 

This month, the National Theatre has also debuted a new 10-minute show, All Kinds of Limbo: From Calypso to Grime, which has “volumetrically captured” vocalist Nubiya Brandon and the NuShape Orchestra for an immersive virtual experience, presented in an area set up with VR headsets.

One barrier to all this, of course, is cost, both for creators and audiences: a full VR kit does not come cheap. The development period for immersive theatre also takes a long time, given the complexity of the technology.

Critics also argue that live theatre hardly needs the tech, given that live drama offers compelling virtual realities of its own. “Who needs VR goggles when you can visit Prospero’s island or Chekhov’s cherry orchard simply by buying a West End ticket?” asked The Financial Times recently.

But there are valid reasons why theatre groups are exploring the medium. One of them is to draw in younger audiences – using tech to attract millennials to mediums that they often eschew, such as classical music, theatre and dance performances. They also want to make performances available to a much wider range of audiences, who might be limited by geography or cost.

The RSC, for example, is thinking broadly about the tech that it plans to use for its immersive theatre. It’s not just about wearing headsets.

“The group is exploring virtual reality, augmented reality and mixed reality, using tech developed by Magic Leap, Epic and Intel,” explains Ellis. “We are extending this to other platforms and devices such as phones, tablets and desktops to see if we can connect those experiences together.”

Ellis adds: “The technologies we’ll be using to make this work are photogrammetry, motion-capture and volumetric capture and we’ll be seeing which ones bring the actor performances alive and gives us the most interesting ways to present the work to audiences.”

It’s also thinking broadly about where the performances might be seen, not just inside a traditional theatre. The idea is that audiences will no longer be bound by their location, which could take place in live performance environments or even in the home.

Ellis explains: “Real time immersive performance is essentially about creating the live, 3D nature of a theatrical production outside of the auditorium. There is something magical about seeing actors act in real time in front of you, while a stage also gives a sense of reality missing from the 2D nature of television or film. Our project aims to explore how the different kinds of extra-reality technology - virtual, augmented and mixed - could be used to bring audiences a unique, truly immersive performance.”

Read more: Immersive tech in film and TV