Looking ahead: The ongoing evolution of IMF

Netflix has been an avid user of the Interoperable Master Format (IMF) for years and has adopted IMF Application 2 Extended for delivery of all Netflix original productions.

Serving a global audience, Netflix is producing a volume of high-quality content that has been unprecedented in the media industry. This wouldn’t be possible without an extraordinary ecosystem of (post) production partners, from dubbing studios to media processing facilities all over the world.

Componentised media workflows and specifically IMF provide the scalability required to support Netflix’s growth, as they enable continuous delivery of media assets into Netflix’s cloud infrastructure from partners around the globe. Traditional media formats use wrappers to combine sound and picture essences into a single file. However, componentised formats like IMF store essence components separately and instead use additional mechanisms to assemble an output timeline on demand.

IMF allows for automated validation steps to be executed upon ingest of materials into Netflix’s asset delivery system. Content properties, like audio language or image resolution, are verified against the requested properties and deliveries are automatically rejected should the values not match up.

Additionally, Netflix has developed an open source tool called Photon that verifies general compliance of IMF packages to the corresponding SMPTE standards. Photon is integrated in Netflix’s delivery platform as well as the vast majority of available IMF mastering tools, to give operators the opportunity to validate packages before uploading them.

Being a componentised media format, IMF uses XML-based Composition Playlists (CPL) to reference assets and to assemble timelines. These CPLs can be very straightforward, however more complex CPLs can reference assets partially, arrange multiple assets on a timeline and replace assets – even partially – with others.

This powerful mechanism enables versioning of content using localised/censored picture inserts, while not requiring storage or rendering of redundant parts between versions. Creating localised texted versions of content is the obvious application for this feature. However, it can also be used to efficiently correct issues that were detected during automated or manual quality control.

The flexibility of IMF inevitably allows for a high level of complexity, which often needs to be restricted by delivery specifications that reflect actual business requirements. Think: PDF documents describing required image resolution, frame rate, sound field configuration, etc.

”IMF’s design can easily be extended to add or extend support for emerging technologies like HDR and immersive audio”

Mastering operators have to create IMF packages according to these delivery specifications, to avoid rejection of deliveries. This practice is prone to error as mastering tools don’t necessarily provide settings for all properties called out in the delivery specification. At the same time, a delivery specification might not cover certain parameters expected to be configured in a mastering tool. Such ambiguities are a recurring reason for redeliveries and therefore cost time and money.

Delivery targets
Netflix is actively contributing to an effort within the IMF User Group to address this issue with a ‘delivery schema’ for XML-based, machine-readable delivery specifications. This will eventually enable IMF mastering tools to read the required parameters directly from the XML document and set up processing accordingly.

Additionally, the delivery schema will allow ingest systems and QC tools to verify compliance to specific delivery specifications. This generic solution will allow delivery targets to be added/updated by simply loading a new/updated XML specification, without requiring software updates in mastering or QC tools.

Thanks to IMF’s design, it can easily be extended to add or extend support for emerging technologies like HDR and immersive audio. This is crucial, as Netflix is continuously striving to improve user experience and to preserve creative intent. Being an advocate for innovative technologies and open standards, Netflix supports the development of a new standard at SMPTE that will add native support for immersive audio to IMF.

Improved connectivity in many parts of the world is enabling use of cloud technology for post-production workflows that are traditionally data heavy. While current developments are mostly targeted at collaborative workflows for creative post production, there is an opportunity to advance distributed delivery workflows.

Work on Netflix titles is often distributed across multiple partner facilities in different parts of the world. A facility creating localised timed text assets, for example, usually works off a proxy without access to the original IMF package.

This prohibits the facility from creating supplemental IMF packages for delivery, which could enable things like metadata-driven assignment of specific versions upon ingest into the Netflix asset delivery system.

IMF-aware media asset management systems could potentially retrieve the information from base packages online for creation of supplemental packages. Cloud based mastering solutions could be given access to work straight off IMF packages that are available online. Exciting times for technology providers and media firms alike.