Virtual production: learning the trade at the National Film and Television School

The use of CGI in film and television production has moved from being a long, slow process undertaken in the postproduction stage involving shooting in greenscreen studios and actors emoting to a ‘tennis ball on a stick’ to an interactive environment that allows performers, directors, and cinematographers to see and move through computer generated environments in real-time. Alan Thorn, a video games industry veteran runs the Virtual Production Course at the National Film and Television School (NFTS) in the UK. The course offers the industry’s first dedicated certificate in Virtual Production.

Virtual production (VP) merges real-time CGI with traditional filmmaking techniques using giant LED walls and systems originally designed for use in video games such as Unreal Engine, to create immersive environments that seamlessly blend the physical and digital worlds. As Thorn explains, the industry is in need of professionals who can navigate this complex, hybrid landscape.

New skills needed

There are many areas of reskilling needed to work in this new production environment. Thorn identifies some of them. “One is in Virtual Art departments and knowledge of the technical workflows involved in working effectively with 3D assets,” he says. “Because we work with 3D models, whether it’s of worlds or it’s of people, they all have to be configured in very particular ways if they’re going to animate and perform optimally. For example, an artist could construct an organic element like a tree, and yet, will that behave and operate optimally when working on the LED wall? It turns out it’s a pretty intricate, technical process to ensure that the assets are where they need to be.”

Read more AI in the virtual studio: innovations in replay, tracking and ad verification lead the way

Making the virtual and actual work together seamlessly is extremely important says Thorn. “I think another area where there’s a lot of thirst for knowledge is in lighting,” he says. “The impact of trying to align the lighting of the characters in the foreground with the virtual sets that we build, trying to match it to the lighting that we have available in the studio, and being able to calibrate all of that with the LED wall.”

Thorn emphasises the importance of generalist knowledge across the entire production process. “In the VP pipeline there’s a lot of variables that technically can go wrong,” he says. “That’s not because the software is at fault. It’s because we are all learning what VP means as we progress. The industry is fast-paced, it’s changing and we’re having to adapt to that and invent new solutions. Sometimes we’re trying out things for the very first time on a shoot and improvising. One of the skills that our course addresses is adaptability on set - having a general enough and a broad enough knowledge of how all these pieces fit together as part of a holistic process.”

Education and training

Thorn describes the NFTS’ VP course as a comprehensive blend of software-based and in-person training. “The course offers part-time engagement so that some of the time students attend in person and some of the time they attend online sessions,” he explains. The six-month program immerses students in both technical and creative aspects of virtual production, primarily using Unreal Engine, a powerful tool for creating digital worlds and characters. “By the end of the course, and particularly in time for the very final exercise, they have to make a short film,” he says. “By that time, they will have had a good survey of what the Unreal Engine can do.”

In-person training at NFTS involves using an LED wall and other physical tools within a TV studio to integrate digital and physical elements. Thorn emphasises the hands-on approach and the four elements of the learning: “One is software such as Unreal and the technical part of building virtual worlds,” he says. “The second part is the physical equipment in the studio: cameras, lighting, the LED wall. The third part is linking all of that up - linking the physical with the virtual. Then the fourth part, which I think is absolutely critical - the teamwork part. Our students don’t just work in isolation. They work together as part of a creative team. And in the final exercise, they actually make a short film, in which they create the assets, they write the script, they direct, and they edit, so they see the pipeline of what is involved in using virtual production to create a short film from concept to completion.”

According to Thorn, the course attracts a diverse group of students. “About 50% to 60% of the students will, in some way, already be working in film or television,” he says. “But it will often not be within a technical role. They might be a director; they might be a DOP. Their training may have been very much with traditional methodologies and with traditional tools. And that may be because they’re either working with an existing studio that wants people to retrain because of upcoming projects, or in some cases, it might be people who want to take their career in a completely different direction.”

Read more IBC2024 Accelerator Project: Evolution of the Control Room

There are two other types of students identified by Thorn. “There’s another group of students who will come from a different perspective,” he says. “They’ll often have a software background, maybe they’ve used 3D modelling tools and they feel pretty familiar with that. But they want to get more exposure to making films, working at a TV studio, working with cameras, getting access to equipment they might not otherwise be able to have access to, and actually putting together a film as part of a team and as part of the creative process. Then the third group, the smaller group, are people who are from completely different backgrounds altogether. That might include people from games who have no experience of filmmaking at all. It might include people who are simply just really curious and really interested in breaking into the industry.”

Thorn emphasises the way that easy access to tools like Unreal Engine is bringing a broader range of people to the world of production. “I think it’s exciting because we’re seeing an era in which never before has it been so easy, even from the comfort of your living room to download the Unreal Engine completely free of charge,” he says. “And with practice and dedication, you can put together amazing looking sets.”

Making the transition

Technological change is never easy, particularly in a high-pressure, skills-based environment like film and TV production. However, Thorn believes that the transition to VP has a built-in element to smooth that transition.

“I think that there’s always going to be an adjustment period,” he says. “But I think what really helps many people adjust is just accepting the philosophy that although much of the work in virtual production happens in real-time on a computer, you can begin to think of it as just an extension of the real world. ‘Oh, you want to move that coffee cup’ - you don’t need to spend ages re-rendering an entire sequence. You don’t need to go back to a render farm to change that. You can just dive into the 3D engine and move the coffee cup - and there you are. The only difference is that it was on a computer rather than being a physical object. I think just thinking of that virtual space as an extension of the physical and that everything that you can move around and do in the virtual set is what you might be able to do in a physical space. Want to move the light? Yeah, you can do that. Just move it, you don’t need to clear that. You don’t need to go back to a VFX specialist and say, ‘well how long is it going to take to readjust that?’ because it’s all live. So, I think one way to approach it is to help people see that it’s an extension of the physical space. And I think when people begin to think of it in that way, they adapt pretty quickly, and they see the real value that it brings.”

For broadcast technology professionals looking to stay ahead of the curve, understanding and mastering these virtual production skills is not just an option but a necessity. As Thorn aptly puts it: “I think it’s incredibly powerful because when we see how far real-time technologies have come, they’re still the worst they’re ever going to be because in the future, they’re going to be even better. The prospect of that is really exciting.”

Read more Volumetric capture: how to bridge the skills gap for broadcast professionals