Virtual Production takes us back to the old disciplines

SMPTE’s Rapid Industry Solutions On-Set Virtual Production initiative (RIS OSVP) launched in July 2021, and in November it was awarded a grant to the value of $150,000 from Epic Games. The grant and the 40+ companies/bodies involved massively extend the long SMPTE tradition of bringing the industry together to collaborate on creating standards for new and emerging technologies.

“A few areas make this effort different,” said Grubin. “It includes a cross section of companies and participants that have only been operating in their respective areas of expertise, but now have specific new applicability to the broadcast and scripted content process.

“SMPTE is also examining the changing needs of our members and the industry through RIS, as part of its evolution as an organisation,” she added.

RIS has opened the doors for many companies to bring their thought leadership.

“SMPTE realised that the scope of companies involved with content creation was expanding. The wide mix of participants – which also includes other professional organisations like MovieLabs, EBU and ETC@USC – has allowed all participants to learn about the work being done by these other teams. It allows RIS to share technology and workflow developments on a wider basis,” said Grubin.

Colour science concerns

What aspects of on-set VP are accelerating fastest, and are there elements that are difficult to establish goals for?

“On-Set VP relies heavily on software-based tools which iterate at incredible speed, which means that work on a traditional standards process may be outdated by the time it is completed,” said Grubin. “RIS is a vehicle to explore how to provide interoperability best practices and workflows, while working towards standards that help with data flow and colour science concerns.”

What are the exact interoperability specs the group is working to resolve?

“The Camera and Lens Metadata sub group has been extremely active developing a charter of the ‘must have’ and ‘nice to have’ metadata fields recorded and carried through the entire workflow pipeline,” said Grubin. “This group has recently expanded to camera tracking metadata, and will begin work shortly on colour calibration and control systems. Ultimately, the agreed metadata charter will become the recommended interoperable ‘target’. This can be used to build open source or proprietary transcode methods.

• Read more Advances in virtual production

“The RIS covers all use cases for in-camera visual effects, from narrative/scripted, broadcast and live events. The advisory group identified overall interoperability and education needs which apply to all of these areas, including media transport and equipment,” she added.

Subject matter experts

What pronouncements around best practice can the industry expect?

“The real take away will be that OSVP tools can be leveraged in many types of use cases, and our initiative goal is providing a pathway to allow for faster democratisation of these processes. For example, we know users need information at their fingertips when they have questions, so we are working on an interactive version of the RIS OSVP wall chart,” said Grubin.

The group has to provide pathways for education and training, for both entering employees and seasoned professionals.

“We are piloting a resource grant program which matches universities in need with advisory group members who can provide subject matter experts for lectures, stage visits, and equipment. RIS has provided a non-competitive space for thought leaders to share their expertise, consolidate the technical information, and output best practices, workflow examples, or refer applicable projects to the standards process,” said Grubin.

What about the carbon impacts of power consumption and rendering? “There is a lot of great research being done in this area in Europe and Asia. As more productions understand the best way to leverage the tools, there are definite potential avenues to improved environmental impact and significant cost savings,” said Grubin.

“VP is a great tool and the biggest challenge is helping the industry understand the ways of leveraging the technology in the most effective way. SMPTE’s goal is to help gather, curate and distribute the information to the widest worldwide audience as possible,” she added.

The final pixel in camera

We now see VP supervisors emerging, and potentially VP will become its own department with its own budget, and no need to steal form VFX budgets.

“We have some individual contributors in RIS, from the production and post side of the workflow. We also spoke to several VP pioneers as part of our original research,” said Grubin. “We do see the traditional budget line items for pre-production, production and postproduction collapsing.

“I do not see post budgets vanishing though. Instead, each discussion will need to evaluate what can be captured as the final pixel in camera – which still may need a clean-up in post – and what makes more sense to do as a traditional VFX process in post. The processes are beginning to happen simultaneously, which harkens back to the days of practical special effects,” she added.

Photorealism via photographic capture, or from within computer engines is another discussion.

“One is not better than the other,” said Grubin. “Part of the education process for the entire industry is helping people evaluate what scenarios work better with plates and what works better with environments created using graphics. There are many use cases where both disciplines are used together.

“One challenge in the industry is helping people speak the same language. What a traditional narrative team would call ‘rendering’ is just called ‘compute’ by the gaming industry,” she added. “There is great work being done by MovieLabs, VES and others to create a common vocabulary.”

Talent does not have to travel

Is it totally wrong to shoehorn VP into a project after it has commenced?

“That is 100% correct. Virtual Production is taking all of us back to the tried-and-true ways of planning for pre-production, production and post. Users can end up with a huge expense without the results they want if the planning process is skipped in any way,” said Grubin. “It goes back to the discipline around filmmaking when we did practical special effects. You only exploded a car once. You storyboarded, you understood where your cameras would be, and you envisioned what you wanted to achieve before setting foot on set.”

The advertising industry loves the notion of real-time engines, and it is another media sector benefiting from VP.

“They can lock down a set and have whatever location they want at whatever time for as long as they need. Talent does not have to travel to a distant location, because the spot can be effectively created on a Volume Stage,” said Grubin.

“Most car commercials are actually created with VFX and no actual car. The ability to fly from exterior to interior – which is impossible in a practical shoot – has now become possible,” she added.

RIS OSVP deliverables are on the way, and updates can be monitored via here.

“We have some amazing education events over the next few months, including (the free virtual) SMPTE+ on September 20. We also have a full day track at the SMPTE Media Technology Summit, on October 24,” said Grubin.

• Read more SMPTE: The power of smart contracts