How MovieLabs aims to empower the creative process with software-defined workflows

MovieLabs, the non-profit R&D industry group, and its member studios (Paramount Pictures, Sony Pictures Entertainment., Universal Pictures, Walt Disney Pictures and Television and Warner Bros. Entertainment) has published a third white paper in its 2030 Vision series, one that takes a fresh look at the production workflow ecosystem and proposes a new path to follow.  

It offers a vision of production workflows that can be more easily assembled, managed, and secured than those currently followed in film and TV production. 

IBC SHOWCASE Register for the MovieLabs and Hollywood Studio Technology Leadership 2030 Vision Update

Titled The Evolution of Production Workflows - Empowering Creative Processes with Software-Defined Workflows, the white paper is concerned with implementing the software-defined workflow (SDW) principles of MovieLabs’ 2030 Vision. The aim is to enable creative tasks by production personnel to be assisted by software-based collaboration and automation systems, leading to workflows that are both more nimble and more adaptive. 

According to Richard Berger, chief executive of MovieLabs, the white paper is intended to reach technical participants in the production workflow ecosystem: “That means cloud providers, application developers, production technologists, and creatives who have to make decisions on supporting film and television production technology across the industry.” 

Recognising similarities 
The paper observes that although studios and production companies assemble “highly individualised workflows and processes tailored to their preferences and to the nature of the production”, many of the crucial elements — the editing process for example — are shared by different workflows. They also share a lack of adaptability, in that once something changes, the workflow must be modified in case it interferes with the creative flow. 

‘Our goal is to present a vision for the future and make some of the necessary common pieces available to help us all realise that vision,’ Richard Berger 

By creating a common framework that views the parts of the production process as defined steps, the stated aim is to allow software to be developed that understands these workflows, is able to communicate information about them, and ultimately is able to automate them. 

It also looks forward to the development of interoperable tools that will support automation in areas, such as collaboration, compute and rendering orchestration, asset movement, cost calculation, personnel scheduling, and project dashboards.  

“The white paper builds on concepts described in our 2030 Vision about the benefits of software-defined workflows and describes the workflow issues we are trying to address,” says Berger. “Today’s workflows are largely customised for each new production, with a combination of manual and automated processes that are stitched together in the best way to mitigate risk. Our goal is to provide creatives with ‘plug and play’ functionality where workflow components can be dynamically arranged and linked automatically through software-mediated collaboration and automation.” 

Assigning order 
Before you run screaming at the thought of some AI assigning software to take your editing job and turning you into a drone, the paper makes it clear that the end goal is workflows that are flexible and extensible, that are composed of reusable components, and always meet the specific needs of the creatives. It further states: “the production process is complicated and does not lend itself to ad-hoc architectures”.  

However, it sees that future success lies in highly configurable workflows that can be continually adapted to support new creative needs of the production, implement new business requirements, or interact with new partnerships. “Production teams will design and directly manipulate workflows, and software will manage the processes of collaboration and orchestration” it states. 

“The most important beneficiaries are the production executives, who will be relieved of much of the technology grunt work associated with supporting custom workflows and allow them to focus more on their creative output,” says Berger. “Additionally, we believe this will benefit technology and service providers by making it easier for everyone to communicate and interoperate, lowering the cost to develop pipeline components (for example, software and hardware) by making each part of the ecosystem automatically interoperate with all of the others, without the need to write custom connectors for each combination of components.” 

Unambiguous definitions 
It’s down to the designer of the workflow to define the architecture and choose what constitutes these components. This can be accomplished by defining the tasks that are used to perform specific functions, what assets and associated information those tasks communicate, which participants (the people and organisations involved) are involved, and what the rules are within the process. The paper gives examples of rules that can be built into workflow automation, such as, “Raw image captured invokes proxy encoding service” and “director’s approval required at this point.” 

Computers need unambiguous definitions and structure, so the advice is to first formalise all of this into an extensible data model.  

In this case the model proposed is an ontology, where the designer would define the assets, tasks, and participants, then define the relationships between them within the context of the stages of the media creation process. Further, to allow computer systems to understand and thus automate the workflow, the ontology would require a controlled vocabulary.  

“Controlled vocabularies are more than a list,” says the paper. “They capture additional information about how terms relate. For example, a Blackmagic RAW Frame is more specific than a RawFrame. Controlled Vocabularies also capture synonyms, such as model or mesh, and homonyms/homographs, such as using model for a VFX construct or model for a person displaying clothes, including the context in which each term applies.” 

‘Production executives will be relieved of much of the technology grunt work associated with supporting custom workflow,’ Richard Berger 

With the MovieLabs Production Ontology formalised, the next stage is to combine this into pipelines. “Pipelines instantiate tasks, asset types, and the connections between tasks as tools, files, and APIs,” says the paper. “More formally, pipeline definitions constrain the abstraction defined in the ontology.” 

Several such pipelines are demonstrated in the paper within the context of a day on a film set, showing how pipelines form the subsets of software-defined workflows that can be used across actual projects. 

• Read more: MovieLabs publishes new whitepaper on production security 

Nascent vision 
This white paper should be taken as part of a wider series available on the MovieLabs site, gathered under the 2030 Vision banner. Some of the systems and mechanisms proposed to implement such workflows are already in place, however. 

“Our goal is to present a vision for the future and make some of the necessary common pieces available to help us all realise that vision,” says Berger. “It will require the development and commercialisation of many different components, some of which are available today and some of which merely need to be interconnected with other components in new ways in order to create end-to-end interoperable workflows. We expect to see many developers working independently and collaboratively to create and market the components of these future software-defined workflows. 

There has already been some reactions to the concept from with the film and TV industry. 

“Prior to the pandemic the industry was at a breaking point,” Berger observes. “Workflow inefficiency was bounding the amount of new content that we could create as an industry, and our original proposal for SDW was warmly welcomed as a potential solution.  The pandemic has obviously slowed production, but also pushed the entire industry to adapt and accept changes to established ways of working. So as the industry comes back, we’re looking to capitalise on the opportunity for innovation and rebuild our industry such that it can grow considerably beyond the limits we were hitting in 2019. 

“There are new projects under way right now that apply key parts of the software-defined workflows vision,” he adds. “We expect to see proof-of-concept efforts and case studies increasing as the industry delves deeper into practical implementations of the 2030 Vision.”