Immersive workflows for CG animation production

While virtual production for live-action content and feature films is a growth area the animation sector is under-served despite increasing demand for quality content across a myriad of platforms and formats.  

One 2020 IBC Accelerator Challenge aims to evaluate the potential of combining extended reality (XR) tools with real-time content engines as a fresh approach to animation production.  

The Challenge will investigate four areas: XR creation tools, workflows, transmedia production and convergence. The aim is to deliver a proof-of-concept (PoC) real-time computer-generated production model capable of empowering animated content creators and storytellers to reach audiences on any platform. 

The team will explore the viability of XR spatial design techniques and identify XR-native tools that are conducive to creativity and can be integrated into established pipelines. The Accelerator will investigate how far the VR interface can be taken.  

A key objective will be to facilitate transmedia storytelling by showing how XR design techniques can enable creators to bring stories to life across disparate formats including short-form broadcast animated video, live social media video, short-form interactive AR stories, adventure games and 3D printed products. If time allows, the PoC will also be evaluated for the production of HMD VR experiences and motion comic formats.  

The Accelerator will also look for synergies between various production workflows to identify best practice for master asset creation and the subsequent generation of optimised versions for different formats. 

• Read more: IBC Accelerator: Usable VFX archive 

Project lead is Rafi Nizam, a creative consultant with experience across visual design and story development for flat-screen and immersive formats. He was previously VP of creative for immersive entertainment at NBCUniversal. He sees an operational need for a methodology to create in a smarter way. 

“Commercially, franchises need to engage audiences in more formats than ever before,” said Nizam. “Established production methods don’t scale and don’t cross-over.” 

“With the advent of VR tools and VR interface for creators, you end up with a much more expressive way of creating content, because you are not bound by a two-dimensional interface. You can now inhabit the space within which you are creating 3D content,” said Nizam.  

Core to the project will be real-time engines. The Accelerator brings together two leading real-time 3D development platforms: Unity Technologies and Unreal Engine from Epic Games.  

“We are at the precipice of a paradigm shift in the way we create animation because these engines can now deliver, and oftentimes surpass, the quality of animation created in the traditional way with the 2D screen and the offline rendering process,” said Nizam.  

“Exploring a story in three dimensions versus constrained by a camera frame is something that has only been accessible to sculptors,” said Ron Martin, creative and technical director, Biz-Dev, Film at Unity Technologies. “Digital sculpting creates a spatial awareness and a more explorative means of storytelling.”  

• Read more: IBC Accelerator: Delivering TV as objects 

The Accelerator will evaluate whether this new methodology can unlock parallel creative production processes by unifying asset creation. Parallel processing would allow different departments to iterate together rather than being bound by a factory line process.  

“As we explore the creative opportunities, we’ll be able to give feedback to the community about the tools themselves and what creators require,” said Martin.   

“This is the future of content creation,” said Ben Lumsden, Business Development Manager, Epic Games – Unreal Engine. “Until recently, our ability to share stories and express ideas has been limited by the confines of distribution platforms and entertainment mediums. Real-time technology unlocks a world of new creative possibilities, and allows for our collective imaginations to be set free.” 

“The real power of what we’re exploring here is a true sense of world building, the idea that you can craft a rich universe, complete with dynamic and meaningful characters, and experience them across an array of different formats—whether it’s linear content like traditional television or film, or an immersive experience, a live event, or even a visual effect on a LED wall,” continued Lumsden. 

Canadian company MasterpieceVR, has been developing tools for artists who want to create in 3D since the first VR headset was released.   

CTO Jeff Ross observed, “With the traditional tools, creating 3D assets currently requires a lot of technical training that isn’t really what many artists are looking to spend their time learning and practicing.  We want to remove this barrier particularly for concept and 2D artists by using VR to allow more natural and intuitive ways to concept and create assets.”  

“We believe that XR tools and real time engines are the future of content creation for animation, film, games, and XR experiences,” he continued. Ross believes that the accelerator will allow small studios to be much more competitive and be able to address more platforms with the same IP faster than larger, more siloed studios. MasterpieceVR stands ready to respond quickly to feedback and create special builds for the Accelerator team.  

Two animation houses are also bringing their skills and experience to bear. Blue-Zoo is a London-based animation studio with a 20-year track record. A multi-BAFTA Award winner, the studio was one of the first worldwide to switch to GPU rendering back in 2015.  

“We believe we should always be developing and exploring new technology starting from a problem-based point-of-view,” said Tom Box, co-Founder and managing director at the firm. “It’s all too common for companies to develop the technology then find a problem to retro-fit into. This has been seen again and again with VR & AR start-ups. We believe there are strong problem-based uses for these platforms in animation, which we are itching to explore.” 

• Read more: IBC Accelerator: AI-automated video shot listing 

Mladen Djukic, founder and director of Aeon Production is excited by the opportunity for his company based in Bosnia and Herzegovina to contribute to such an international project, collaborate with big industry players, prototype quickly and share the results with the wider community and potential partners.  

His company has been animating for 18 years and already uses Unreal. “The Epic Games engine has helped us tremendously in the past few years, and we hope other small companies will benefit from this technology as well.” 

Nizam concludes, “In addition to exploring the potential for commercial and operational transformation of animation production, I hope the Accelerator will shed light on how a converged workflow might make creators more sensitive to the unique narrative opportunities of different mediums. This could lead to new story development and origination models for animated content. Irrespective of workflow, the story comes first.”