Creating a codec for tomorrow’s content

Formed three years ago at IBC2015 mainly because of uncertainty over HEVC licensing, The Alliance for Open Media (AOM) has an awesome governing membership of Amazon, Apple, ARM, Cisco, Facebook, Google, IBM, Intel, Microsoft, Mozilla, Netflix, and Nvidia.

Its other 24 members include Adobe, Ateme, BBC R&D, Bitmovin, Hulu, and VideoLAN. Its newest member is the giant Chinese online site iQiyi, which has some 500 million active monthly users.

Its AV1 codec was conceived as an open-source, royalty-free technology for delivering online video content, which could offer 30% plus quality gains over VP9 and HEVC.

The AOM, backed by first AV1 codec implementations from Ateme, Bitmovin, Google, Mozilla, NGCodec, Socionext and Video LAN, made a huge practical impact on IBC, also gaining several major corporate plugs in the IBC conference sessions.

With so much to report in terms of AV1 impact it may have taken some by surprise when AOM board spokesman Matt Frost, Director of Product Management with Google, said: “Work has already started on AV2. We had to leave some tools on the table just because there was work coming in we did not have time to integrate in AV1. Those will go into the next generation codec.

“The way we talk about implementations is that the AV1 standard is actually not a piece of software,” he added. “It is a written description of how you decode the AV1 bit stream. The big thing about a royalty free and open source codec is that you know what the terms are from the beginning.”

AOM has the core ambition of expanding the network and is mindful of recent Cisco stats that indicate 80% of data on the web will be video content by 2020.

“If we applied AV1 with its 30% gains to all the video out there we would have 30% extra capacity in each network,” said Frost. “On the one hand faster networks should make for better consumer streaming experiences, but as soon as people have a faster network they watch more video. There will always be an increase in demand.”

Frost predicts that we will see devices with hardware AV1, so expect mobiles or TVs with a dedicated processor that plays back AV1 quality: many other use cases spring to mind.

“The community of AV1 supporters will be able to use AV1 everywhere that we want, because there are certain applications like very high resolution where you really need hardware to power it,” said Frost.

“On the one hand faster networks should make for better consumer streaming experiences, but as soon as people have a faster network they watch more video. There will always be an increase in demand.”

Better experiences
The other core values in AV1 cover several key issues, starting with latency.

“The codec is very low latency, and a current generation has to be because you want to enable things like video conferencing and VR experiences.

“In VR, where you move your head, and it takes a nanosecond for the video to move, people become nauseous. And this whole thing holds true for video conferencing,” said Frost. “You want immediacy, especially in apps like distance learning, and when people want to do online musical collaborations.”

Perhaps unwittingly the millions of users shared by Netflix, Facebook and YouTube already see the improved picture quality allied to AV1, so will this trigger wide consumer recognition?

“It is not reasonable to expect consumers to be so sophisticated that they are looking for an AV1 TV, but they will see and discuss YouTube and Netflix content looking better. Consumers are smart enough to know new technologies are powering better experiences,” said Frost.

“Take HDR,” he added. “The codec supports all the features necessary: 10-bit bit depth, support for different colour samplings, and the various standards as with an LG TV supporting Dolby Vision, or a Samsung TV supporting HDR10+.”

Higher frame rates (HFR) are a question for the consumer manufacturing community, but in summary, Frost said: “The codec is virtually limitless in terms of the resolution it can support and frame rate (60/120fps) it can support. There is an incredibly complex eco system out there, and there are tons of different new tools that are necessary. We conducted a very in-depth review process to make sure that we were developing a royalty free codec, and is paying off hugely now.”

”Consumers are smart enough to know new technologies are powering better experiences.”

A pleasant surprise
Collectively the AOM believes that AV1 has rolled out like the original members expected. It all started with a core technology project based on Google’s VP9 plus tool sets from Mozilla and Cisco.

“Much has changed since then, but unfortunately for the video world HEVC licensing confusion has not disappeared,” said Frost.

“We all hoped in AOM that everybody would get along and work well together, but we were and are a bunch of big companies who are rivals. The pleasant surprise is that it has worked as we planned, and people have continued to get along and develop the codec. There is now a unanimous enthusiasm for creating another codec, but it is tough to now how long that work will take.”

The most noteworthy new member is iQiyi, but AOM has been talking to another giant corporation in the Chinese online market, and many other big brands.

Frost said: “We continue to see in-bound interest from big companies coming to us and saying: ‘We think the time is right to start looking at AO Media and AV1, because we are not sure the path we have been following with MPEG is the one we want to continue with.”

“We are pushing forward on another front. It is easy at shows like IBC to be focussed on 8K, AI, 5G and all the next glossy trends, but there are lots of people around the world who are just coming online with 2G networks,” he added. “They are looking at mobile video screens that are tiny. We want to make sure that these millions of people have a great experience, but not at HD or 4K. It starts with 240P.”

Frost was speaking for Google but said: “Netflix, Facebook and all the AOM companies and many beyond are thinking out their entertainment and educational strategies for the next billion users who are coming online.

“Back in IBC land we do keep a strong eye on other technologies, and it is interesting to see all the marketing and hype around 5G, especially the underlying engineering work and deployment of expensive hardware behind that so you can increase network speeds,” he added.

Going into IBC and since, the hard work being done by the AV1 engineering team involved speeding up the decoder and encoder.

“This is so they can be used in all the places we expect to see video; you want to see it on your Android phone, play it on your PC, or watch it in your TV, and all of that requires work to make sure the processes that are powering those devices can play back the video,” said Frost.

“With Google, our first implementation is our AV1 decoder. Ateme is writing its own encoder, while Cisco is creating a video conferencing application. All these first implementations will seed the market.”