The cutting edge of VR production

While VR is yet to find true traction in the wider consumer market, the tools for making immersive content find themselves at a bit of a crossroads.

“We are now witnessing a growth in diversified content and approaches,” says Davide Bianca, Head of VR / AR at Unit 9.

Bianca says the VR landscape used to be fairly polarised, with pre-rendered 360 CG video at one end and conventionally captured video at the other.

The leading edge content now is a blend in some proportion of the two and that immersive content is “now evolving into a much more diverse and stratified ecosystem.”

The consequence is that the tools are evolving too. While Nokia might have shuttered its high-end OZO camera operation last year (development of the software at least has been picked up by a new start-up, Imeve, formed by key members of the OZO team) there are any number of high-end options available.

OZOs, of course, are very much still in use, as is the Lytro Immerge following Google’s acquisition of the company earlier this year. Bianca also adds the GoPro Odyssey, MooVR and NextVR to the list of high-end options, with the qualification that you have to be lucky to get your hands on the latter in particular. Indeed there are a growing number of decent options for shooting immersive at all levels, from the prosumer to the high end

“The filming technology has come along way,” says Zahra Rasool, the Editorial Lead for Al Jazeera’s immersive media studio Contrast.

“The cameras now are much better than the ones I used a few years ago. Every time we filmed in VR we had to manually stitch the footage together from every lens to create one shot, now most cameras effectively stitch the footage when you export it.

”The stereoscopic sound from the cameras has become much better too, meaning you don’t have to be capturing audio externally.

”Low light is better as well. There can always be improvements, but the cameras out there now are pretty good,” she concludes.

And, of course, if all else fails, there is a rich heritage of home-brew rigs to fall back on.

“The technology has improved, the expertise has improved, and we are now starting to get a new raft of creative talent coming into VR.” Richard Mills, Sky VR Studios Technical Director

“In any creative medium, if you can’t find the hardware to make your idea come alive the way it should be, the answer then is to build the hardware yourself,” says Arnaud Baernhoft, Digital Producer, Alchemy VR. “There are many productions where we think up a scene and then realise that it’s not something that has been done before, therefore we have to find a way to make it achievable by creating a custom rig. That is the best part about working with emerging tech.”

Camera technology is only a part of the modern immersive landscape, though, with CG content provided by realtime games engines an important part of the mix.

The Unreal Engine dominated the industry for an extended period, but recent years have seen Unity 3D gain in popularity, partly because of its powerful memory handling features. And where once projects were very much siloed and either live action or CG, that is changing.

“We are at a crossroads at the moment,” says Sky VR Studios Technical Director Richard Mills, “To be able to have full interactivity and then full immersion you have to have the ability to be able to influence and interact with your surroundings which is very difficult with 360 video.

”CGI plays very well in Unity but it doesn’t necessarily have the same realism. But, if you can combine photorealistic environments, an object and sometimes a character with an enhanced CGI background, then you get the best of both worlds; you have the interactivity and you have the realism as well.”

There are still plenty of challenges to be solved, however. Baernhoft, for instance, would like to see progress made on data transfer rates and stitching, with AI routines promising clean, ready-stitched output from the cameras also showing potential.

“Having tools to edit VR in VR is also a next step to look forward to which will greatly improve workflow,” he adds. “Unity has been making great progress on this in the background and we are waiting for them to release their new products for this.”

The IBC showfloor is going to be an interesting place to gauge the speed of progress in the area.

“Unity and Unreal are pretty good and you can do most things, but in the actual building of objects we’re still down to Maya, Cinema4D, and then an import process,” explains Mills. “ If all those traditional compositing and design tools can be better integrated with VR that would be useful.”

He also adds a desire to see faster, network storage that can be used on location and in the studio. But the main areas of progress in immersive content he sees as being definitely not confined to the technological.

“The technology has improved, the expertise has improved, and we are now starting to get a new raft of creative talent coming into it who are looking at a hybrid way of publishing these experiences with influences form the game world, influences from the drama world, and writing specifically for the medium rather than trying to shoehorn something in there.”

It’s a viewpoint echoed by Marc Schipper, a transmedia producer and CEO with a specialisation in immersive theatre, who sees natively-written content as having a far more gamelike dynamic than we are used to with linear content. And while he counts the release of the £200 Oculus Go as a significant moving of the goalposts in terms of consumer adoption, it’s the forthcoming digital layer that augmented reality will confer on the real world that he sees as being the genuine game-changer.

“I suspect Apple will be the company that does it,” he says. “I’m looking at what they’re doing and the patents they’re putting out, and I really think that’s the direction they’re pivoting.”

It’s a viewpoint that has been given extra currency recently too with the latest research note to investors from noted Apple analyst Ming-Chi Kuo, who points to Cupertino’s legacy of reinventing UIs with great success (think of the iPod click-wheel and the iPhone’s multitouch) and suggests it will offer an “AR experience created by the AR glass, which will likely be launched in 2020.”

There’s a lot of interesting development going on at the extreme high-end too, which Bianca categorises as Hollywood magic: on the performance capture side you have Disney Research Zurich’s time-based capture rig Medusa and Digital Domain’s high fidelity facial performance capture software Masquerade; while in terms of VR experiences the Positron Voyager full-motion chair is pleasingly Ready Player One and Oasis-esque.

“In any creative medium, if you can’t find the hardware to make your idea come alive the way it should be, the answer then is to build the hardware yourself.” Arnaud Baernhoft, Alchemy VR

Such developments point the way to full haptic feedback, stereoscopic sound, and high quality content that can truly trick your brain into thinking you are in the middle of an action sequence and in full motion, he says.

Time will tell whether this trickles down and rolls out into the mass market, but one of the things most notable about the cutting edge is that its position very much changes depending on your environment and your circumstance. As part of the My People, Our Stories initiative, Al-Jazeera’s Rasool trained a group of 10 South Sudanese journalists to successfully storyboard and capture 360 video. She talks about the cutting edge as being less about deploying the latest technology and more about giving a voice to those in danger of being marginalised by it.

“The unique thing about VR is that it can immerse you in a location the way other mediums cannot,” she says. “The problem is that most of the people with access to the technology at both ends are white men. The stories that are being told are very much from that perspective, reinforcing the stereotypes that are already present in our media industry. We need to be telling the stories in a better way. We need to be telling the stories of the developing world, of the communities of colour.”