2020: Crunch time for codecs

The industry can kiss goodbye to the familiar pattern of one dominant video codec per decade. As we move into 2020 there are a raft of developments giving media streamers, device manufacturers and chipset vendors a multitude of options to trade operational performance with price.

Key developments include two new standards, MPEG-5 EVC and MPEG LCEVC, reaching final draft in April and full standardisation by October this year.

VVC, another MPEG standard, is also on course for standardisation by year end.

Meanwhile, AV1, which is being promoted as an alternative to MPEG schemes, is having to battle a patent pool challenge for AV1 implementation - contrary to a key reason for setting AV1 up in the first place which was to be royalty free.

Let’s unpick each of these in turn.

Essential Video Coding – licence friendly baseline
EVC is intended to be an alternative to coding standards such as HEVC, AV1 or VVC and expressly designed to be royalty free in its basic implementation. It was initiated by MPEG as a countermeasure to the opaque licencing costs of HEVC by being offered in two profiles.

A baseline profile is meant to provide performance in line with AVC/H.264 and includes only technology that is at least 20 years old and therefore whose patent licences have expired.

A main profile to EVC offers superior compression efficiency with addition of a small number of tools and a cost per use. The same tools can be individually disabled.

“The thinking here is that if you have licensing issues, you have the option of just turning off that feature which could five you extra leverage in patent discussions,” says Bitmovin head of research Christian Timmerer.

Like other traditional codecs, EVC will require hardware implementation and therefore it is likely to take at least a couple of years for initial adoption to take hold in devices including TV sets and smartphones.

• Read more: Broadcast codecs and the challenges of the streaming era

Low Complexity Video Coding – codec agnostic
Despite being part of the same MPEG 5 family there is no connection between EVC and LCEVC. It’s a fundamentally different approach to EVC in which the solution relies on taking an existing codec available in hardware – which could be AVC, HEVC or, in future, VVC - and adds a software layer on top responsible for improving the performance of the hardware encoder.

The vendor driving this is V-Nova on whose technology, Perseus Plus, LCEVC is based.

“LCEVC is a codec-agnostic enhancement coding standard – that is, an add-on, not an alternative – which enhances any other video codec (including EVC) by simultaneously improving compression efficiency and reducing processing power consumption,” explains V-Nova chief execuite and co-founder Guido Meardi.

The approach will make is possible for companies to deploy hardware video codecs once and upgrade them with software, often a far cheaper alternative to ripping and replacing devices in the field.

For example, it can allow UHD to be used where only HD was possible before due to being able to share decoding between the ASIC and CPU.

“Specifically, due to its lightweight nature, LCEVC can be efficiently implemented in software, and also at the application level,” Meardi explains. “This is thanks to its use of standard graphics hardware which is accessible and power efficient at all levels in the software stack.

“For these reasons, LCEVC has great potential to be adopted much more rapidly than traditional standards that typically require many years for new hardware components to be designed and manufactured before they even begin to be distributed in new devices.”

For example, a service currently streaming a mix of H.264/AVC and H.265/HEVC to a range of devices could roll out LCEVC enhancement across all of its services to improve video quality and reduce operating costs without changing any equipment or infrastructure. The business cases involved in LCEVC tests typically include improvement of QoE, expansion of customer reach, reduction of churn, reduction of delivery costs and reduction of infrastructure costs, all at the same time.

“The intention is to publish licence costs for LCEVC as soon as possible after final publication of the standard,” says Meardi. “It is anticipated to be a transparent, accessible and clear licensing structure to enable quick and wide adoption by the market players.”

Versatile Video Codec on track
VVC is on track to be finalised as an international standard by the end of 2020. The next-generation (after HEVC) standards-based codec developed jointly by MPEG and the ITU targets 30% bitrate reduction over HEVC at the same perpetual quality. Trials, including by BBC R&D, confirm this. VVC capabilities are expected to support immersive content, resolutions from 4K to 16K and 360° videos particularly in complement to 5G networks.

Since VVC will require hardware implementations it will likely take at least a couple of years for initial adoption. It is also hard at this stage to predict the cost of the licence for VVC but everyone is keen to avoid a repeat of what happened with HEVC.

As Ken McCann, strategist at Zetacast, explains: “Although HEVC has been a great technical success the licence situation is problematic. It has a complex arrangement of three patent pools plus individual IP holders [which means] creating a business plans with accurate estimate of the total licencing cost requires multiple NDAs and commercial discussion.”

The Media Coding Industry Forum (MC-IF) was set up to figure out how to best licence VVC. Members include ATEME, Apple, Bcom, JVC, Mitsubishi, NTT, Nokia, InterDigital and Ericsson.

AV1 – not so royalty free?
Some of these some companies are part of a group pursuing IP licences for their part in developing technologies that contributed to the VP9 and AV1 codecs.

AV1 was established by the Alliance for Open Media to outflank HEVC by being at least as good as MPEG’s codec but royalty free.

It has succeeded in the first goal, although there are some claims that its operational processing is higher than HEVC, but is facing serious challenges to its royalty free promise.

Through patents administrator Sisvel (which is also a member of MC-IF), companies including Dolby, JVC, Ericsson, Orange, InterDigital, NTT, ETSI and SK Telekom have declared that their IP is foundational to AV1 (and VP9).

AOM has to date insisted that its members, which include all FAANG, Intel, Microsoft, Cisco, Samsung, IBM, Nvidia, Tencent and Mozilla, own all patents related to AV1 and are prepared to give that away for free.

• Read more: AV1: Codec wars erupt

Now Sisvel has published a list of additional patents – more than 650 for VP9 and 1,000 for AV1 - and expects to reach a total of around 1000 patents for VP9 and nearly 2000 for AV1.

That means that the AOM either needs to concede licence terms with Sisvel or contest Sisvel’s pool claims – a process which at best could delay implementation of AV1 and at worst lead to huge legal expense and the internecine wranglings which beset HEVC.

Companies already implementing the VP9 technology have been informed of the technology’s cost, while those considering adopting AV1 would know that royalties will apply, Sisvel state.

“Our goal with these pools is to reduce the friction in the market,” says Sisvel chief executive Mattia Fogliacco. “We believe pools create efficiency by enabling potential implementors to sign one agreement licensing multiple patent portfolios in one transaction at a reasonable and transparent cost.

For a licensee that is in full compliance with terms set by Sisvel the cost is €0.24 for consumer display devices (including smartphones, TVs, VR headsets and computers) and €0.08 for non-display AV1 devices (such as set-top boxes, games consoles, streaming media players).

“The pool is the least expensive and most efficient option to clear,” licensing option, Sisvel states.

This story will run and run.