Codec wars: The battle between HEVC and AV1

Encoding and compression is vital to the economics of video production and delivery but rarely causes much of a storm.

That’s changing as the current market leader, HEVC, is challenged by a new generation of codecs potentially capable of delivering improved video playback at greater efficiency and crucially, at lower cost.

Video compression has been dominated for as long as anyone can remember by standards bodies Motion Picture Experts Group (MPEG) and ISO/IEC.

MPEG-2 was the de-facto format of digital TV signals before MPEG delivered a more efficient scheme, MPEG-4/ H.264 AVC, in 2004. MPEG-4 cut the bandwidth needed to deliver programming by half, and its successor H.265/HEVC (High Efficiency Video Coding) published in 2013 managed the same 50% saving trick again.

However, HEVC has not taken off in the same way previous standards have because MPEG was not alone in its development. The joint patent holders of the technology have demanded licence fees which many in the industry consider either too high or too vague to justify investment.

In particular, HEVC adoption has been low for use in streaming and is why some of the largest tech companies - Amazon, Netflix, Google, Intel, Cisco, Microsoft, Nvidia and AMD among them - formed the Alliance for Open Media with the aim to create a royalty-free alternative.

That alternative, called AV1, has now arrived and it is claimed to be up to 30% more efficient than HEVC. The code for the codec is on the point of being frozen and encoding vendors are poised to release the product based on it at NAB next month.

HEVC cost and complexity

The cost of HEVC is significant. Licences are required from three main licence pools MPEG LA, HEVC Advance, and Velos Media - which represents the interests of Ericsson, Panasonic, Qualcomm, Sharp, and Sony.

While MPEG LA only charges per decoding units capped at U$25m a year, HEVC Advance caps its annual unit charge at U$40m and demands royalties for streaming, capped at U$5m a year. This ‘tax’ on distribution hits streamers like YouTube and Netflix and can be charged on a title by title basis.

The third major pool, Velos Media, has not yet declared its royalty rate. To make matters more complex, there are multiple other HEVC IP owners (such as Nokia and Technicolor) who have not yet joined a pool or announced a royalty policy.

“The problem with HEVC is that you don’t know what you will have to pay,” says Martin Smole, Engineering Director Encoding, Bitmovin. “One things is certain is that it’s significantly more expensive than H.264. The uncertainty is [around HEVC] certainly blocking adoption.”

NAB will offer the first chance for encoding/transcoding vendors to demonstrate the capabilities of AV1, providing a first thorough look at the codec’s quality, and decoding and encoding requirements, including comparisons with HEVC.

Bitmovin, which will have AV1 product in Las Vegas, says the new codec reduces bandwidth demands by up to 30% while still retaining or improving picture quality.

“In our tests with AV1 the efficiency is up to 30% better especially for higher resolution and higher bitrates and much better than HEVC,” says Smole. “It is less efficient for smaller resolutions.”

Others are more circumspect. Thierry Fautier, VP of Video Strategy at Harmonic and President of the Ultra HD Forum (a UHD standards promotions body including Dolby, Comcast and Ericsson) says the claims made for AV1 have yet to be verified by independent sources.

“The AV1 reference encoder is [as of today] a hundred times slower than an HEVC one,” says Fautier. “That will likely improve in the hands of encoding vendors but I still expect the additional complexity of the encoder to be around ten times vs HEVC.”

In addition, it will take some time for AV1 hardware decoding capability to be integrated into devices like set top boxes and mobile phones. Those chips have to be manufactured and added into consumer product – a process likely to take at least two years. Consequently, AV1 hardware decoders will not see mass volume before 2020, Fautier predicts.

Fautier also advises that the final ‘free’ cost of AV1 “has yet to be proven in court” since there may be challenges to the code if companies feel their patents have been infringed.

AV1 software encoding, on the other hand, is ready to go. Mozilla has already provided a successful browser implementation of AV1 for Firefox Nightly (powered by Bitmovin). HEVC is not supported by Google in Chrome or by Mozilla.

Content Aware Encoding

Clouding the future of HEVC further are new techniques able to extract more mileage from the legacy MPEG4/H.264 AVC equipment.

Content Aware Encoding (CAE) which uses machine learning to compare content against known parameters for a given device and/or media player type, can boost the picture quality and reduce the distribution cost are proving to prolong the life of AVC.

AVS2.0, a codec created in China, is being investigated by the Ultra HD Forum for integration in its future specifications. This codec can be used for OTT and broadcast.

With an installed hardware encoder base of 2 billion, HEVC is not going anywhere soon. It is tuned for broadcast, whereas AV1 is tuned for OTT. What’s more HEVC is right now the best codec for handling 4K UHD and HDR video.

HEVC was also given a boost last autumn when Apple threw its weight behind the codec, agreeing to support it for the first time in Safari and Apple iOS products.

“Netflix has stated that as soon as AV1 is available, it will initially support AV1 in browsers and then on connected devices once hardware support is available,” says Fautier. “Netflix is currently paying HEVC royalties, and if it can reduce these costs by using a different codec, without impacting the QoE, that’s a win-win situation. Apple knows it needs to be ready for AV1, especially if it wants to support Netflix services in the future.”

Logically, both HEVC and AV1 – along with another Google-led royalty free codec option VP9 – will exist side by side for the short term but unless HEVC patent holders lower their cost and become more transparent on licence structure there would appear only one winner.

“One thing is clear: If HEVC patent licensors thought they would have a nice ride to the bank, they now know that is not going to be the case,” says Fautier.

“As an industry, we hope that the HEVC licensors come up with a reasonable and fair solution to resolving the patent licensing issues.”

Bitmovin believes AV1 will become the standard for the future world of content which relies on large resolution video, VR and AR applications.

Others aren’t so sure and believe an entirely new codec is needed to handle the massive data demands resulting from 5G and ultra-high bandwidth low latency streaming.

“For VR (6DoF) and 8K we need a new codec,” says Fautier. “AV1 is not designed for that.”

MPEG is already on the case. Together with teams at ISO and ITU it has formed a Joint Video Exploration Team (JVET) to study the need to include ‘omni-directional’ video coding technologies in a future video coding standard. It is basing its work on HEVC. Expect news of its progress at NAB.