Interoperable Master Format for TV: the solution to versioning woes?

The issues that arose over time included a dislike of JPEG 2000 as the preferred codec, how to create a metadata schemer, transcoding issues, automation requirements, QC, and issues around subtitling and captioning. Jump forward four years and the technology issues are all in hand, but there are some doubts that many EBU members will need a broadcast IMF version.

The BBC was always going to be a big taker and be a helper, as proven by Andy Quested, Head of Technology for HD, UHD, 3D and quality control.

He said: “We have this as a DPP/NABA project for broadcasters, with two primary use cases: incoming, meaning buying content masters for further compliance processing, and outgoing, which is sales mastering.

“The EBU has a working model based on an ST2067 core and App 2/2E but using ProRes. We will also look at an identical App with H.264 and H.265 in the future,” he added.

“We are also working with the Digital Production Partnership (DPP) and the North American Broadcasters Association (NABA) to incorporate all the business requirements for the widest range of broadcasters.”

There are two aspects to finding out where we are with a new IMF, and these come via big enthusiast Bruce Devlin, Chief Media Scientist with Dalet and a governor in SMPTE UK, and Frans de Jong senior project manager at the EBU, who will be fronting a free workshop on IMF for broadcasters on May 30-31. This will offer knowledge on versioning, apps, mastering, codecs, archiving, air master creation, subtitles and QC.

Pint glass precision

To find out how different the TV version is to the IMF Hollywood original, we started with Devlin.

“In terms of workflow and underlying standard there is very little difference. We are using the same core standard,” he said. “There is likely to be a codec difference, but that’s not universal. Within IMF we have the basic core spec that tells you how to do IMF, how to make the different versions of your title, what metadata you need to be present, and it tells you how to label audio.

“If the total specification is a pint glass, about 7/8ths of it is full with all of that core spec. The little bit that you put on the top is what we call the application, and that allows you to choose a variety of codecs or one specific codec, and then restrain that codec(s) to various operating points.”

For the delivery of SD and HD content there is already the application App 2/2E. What can that do?

“That is likely to be used for a lot of TV delivery, and all of that is already standardised. We have been plug fest reviewing and testing that for about two years, and it is looking really good,” said Devlin.

“The TV IMF initiative is all about do we need to do anything different? Do we need to add extra metadata? Are there any scheme additions required? What broadcast metadata needs to be plugged into the IMF standard to make it just right for TV? The answer is yes we do: probably you need to put in some extra metadata to help the televisual description.

“So this has taken our glass to 15/16ths full and then it is also likely that another codec is going to be required. That could be ProRes. We have just finished in SMPTE doing the ProRes and MXF mapping (document RDD 44). That is done and dusted now and people are experimenting with ProRes and MXF to see it that is the right codec,” he added.

IMF: a way of thinking

That is looking quite promising we hear, so what else fills out the pint glass? And are broadcasters apart from the likes of the BBC unaware of what Broadcast IMF can do for them? Surely many will see it gifting their archives new monetisation promise.

“What we are talking about is how much more different stuff we need for TV IMF. There is still some learning that needs to be done and I know this because myself and Pierre Lemieux (of Sandflow, and Chair of the SMPTE technical committee TC35 PM, for media packaging and interchange) created the SMPTE IMF certified training course and that was booked out for the last couple of times we ran it and it looks like it will be booked out for the June version as well,” said Devlin.

“What we are finding is that even people who think they know it all find new and useful information,” he added.

”What IMF gives you is more than a data format or just a collection of files; it is a way of thinking” - Bruce Devlin

It is Devlin’s belief that IMF will do for media what the Agile and Scrum subset methodologies did for the coding world.

He said: “IMF gives you the tooling to be much more Agile and Scrum like and get out incremental versions. You remember these much more easily, and more importantly if you do not like that incremental version you can roll back to the previous one and start again much more easily than you can do today.

“We have not had to invent anything new. All the tools already existed in MXF and what we have done in IMF is to say do not use these features of MXF. What we have done in MXF is constrained it so we have only got the features that are perfect for doing the versioning type of workflow. And that is the big secret,” he added.

Automation

For broadcasters looking at IMF, automation is one of the most important elements. And they will want it to be as economic as possible.

“That is exactly it,” said Devlin. “One of the key things about IMF is that you have lots of identifiers inside it. You can identify a version of a composition, or a specific piece of media, or a specific segment on a timeline. You can even identify the two resources that go together and join up to make that segment,” he said.

“There are identifiers at every level of IMF, and once you have them you can then start to track things. And once you can track you can start to automate movement. You can look back in time to replicate a decision you made yesterday with a new piece of material automatically. We are not relying on file names; everything is done with identifiers.

The only worry in terms of vendors is the subtle opportunities for the simple versioning that originally dogged MXF.

“We are trying to make certain that does not happen,” said Devlin.

“We have staged five plug-fests within SMPTE and another was done independently at the ITT in Germany. We have been simulating the sort of packages that would be created and consumed by movie studios, TV facilities and post houses, and pushing those between vendors to try and get rid of those subtle versioning issues,” said Devlin.

“So far it has been working well. We have got to the point where for interchange IMF is working really well. The video is working and the audio is working and everything appears to be in sync,” he added. “The big thing we are testing now is the rendering of the subtitles – to make certain the time text subtitles are rendered consistently between different screens.”

Rolling and scrolling

A particular version of TTML (Timed Text Markup Language) called IMSC1 (Internet Media Subtitling and captioning protocol of TTML) is being used. Devlin explained its values. He said: “It is a relatively new spec, relatively constrained, and all the vendors are learning about its capabilities. We have been through about six months of testing and it is looking very reliable. IMSC1 allows you to offer subtitles/captions for any frame rate, and then render them,” said Devlin.

“It is intended to be the one format people use for mastering and authoring multiple versions, and that is handy because that is the role of the IMF standard. IMSC1 is an XML document that defines a number of display regions. Text that you want to display is rendered into those regions according to the timing model. The regions offer support for the rolling and scrolling functions,” he added.

“We have to encourage the authoring tool vendors and the display tools to strictly adhere to the rules of IMSC1 so that we can get good interoperability and reduce the cost of these accessible services.”

What green values does TV IMF offer? “There’s a whole bunch of transcoding and computing you no longer have to do downstream, and if IMF becomes pervasive you are going to save around 20-25% on the total amount of compute that you need to do globally in generating all of those versions,” said Devlin.

EBU Workshop

The EBU workshop promises to match well to the SMPTE IMF certified training course: however, de Jong feels that everything published and known may not be quite enough.

He said: “It could be that broadcasters are saying they would like to see another codec, or they are missing something in the applications 2/2E for example.

“Our main question has been are our members interested in using IMF. And the answer from many has been they don’t know what it is, or they are currently struggling finding a main in-house format, and maybe this is too early,” he added.

“There is a big miss-match between what the Hollywood type companies are looking at and what broadcasters in general are looking at” - Frans De Jong 

The main promise of IMF is versioning in a very flexible way, so it is a no brainer for anybody creating multiple versions of a programme.

To find out what broadcasters really want, de Jong will put up several use cases at the workshop.

“The main one is if you get an incoming IMF because you ordered something to broadcast, and then you have to change this into a broadcast format. Next is if you are delivering to someone else and it is typically the bigger enterprise. Third is when potentially IMF could be used for different versions; you have created a programme and want to archive for later re-use,” said de Jong.

“France Television has looked at using IMF for the distribution of content on different OTT platforms, and then doing last-minute ad insertion based on metadata in the IMF,” he added. “That could be something where there is potential for many broadcasters because different outputs in a national market are also different versions.”

He is expecting a lot of questions from archive owners. “They will say, I have an archive of so many different codecs, how does that fit with the incoming IMF, and can I backwards apply it?”

Frans de Jong points out that the higher-level operational patterns in MXF could have been used to already achieve some of what IMF offers.

“The lesson that we have learned is that you should be really careful, really religious about these new specs and not just jump on the bandwagon and say ‘it is great’ before we see that people really believe it is beneficial to their business,” he said.

“On the one hand we want to be open about it and spread the message of IMF, but we do not want to say the EBU thinks you must use this.

“We never said we necessarily need a different version for broadcasters. That is an on-going situation, and we are testing that with a different participant group,” he added. “But at the moment the jury is still out.”

Somewhat cagey prior to discovering where IMF stands with EBU members at the workshop, de Jong will have the support of Marquise Technologies, one of the leading IMF implementers in software, Bruce Devlin and Andy Quested.

He said: “The BBC is at the forefront of trading in many other countries. We have seen some small-scale stuff in other broadcasters, but it is a niche in that sense.

“But use cases might multiply. Something that can evolve from clearly starting on the film side now sees broadcasters clearly very interested, and that may lead to new apps. And yes, it could go wider: there might be another industry like gaming to take it on,” he added.

“Right now, a lot of people in TV revolve around the codec question: for some it is hard to understand why broadcast ties itself to codecs. The other issues are legacy thinking and looking at how the metadata works. But from everything we have seen until now, IMF is very flexible.”