New forces enter the codec arena

As if the codec space wasn’t crowded enough, MPEG is entertaining the prospect of not one but two more standards for video delivery within the next couple of years.

Its recent call for proposals for two new codec technologies can be seen an attempt by the standards body to force HEVC license holders, including its own division MPEG LA, to reduce the terms of their license demands.

“Earlier this year the industry realised that new codecs were coming and started to panic but the result is that the market is getting even more fragmented,” says Thierry Fautier, chair of the Ultra HD Forum and VP video strategy at Harmonic.

“HEVC works and has 2 billion installed devices but the problem is the licensing. MPEG may feel that the more pressure that can be put on HEVC, the more it will give in.”

Samsung’s codec

The first proposal is for a new video coding standard “to address combinations of technical and application requirements that may not be adequately met by existing standards”.

As MPEG points out, coding efficiency is not the only factor that determines industry choice of video coding technology for products and services. There’s the cost implication too. To which one might add, politics.

The focus of this new video coding standard is on use cases such as offline encoding for streaming video on demand and live over-the-top streaming. The aim is to provide a standardised solution which combines efficiency similar to that of HEVC “with a level of complexity suitable for real-time encoding and decoding and the timely availability of licensing terms.”

Essentially this is what the Alliance for Open Media (AOM) is already doing with its AV1 codec.

If you are a consumer electronics manufacturer like Apple being charged even 1 dollar per device you sell to use the HEVC codec then that’s at least $200 million per year off the bottom line. In reality the cost could be more than this, added to which is the uncertainty around the actual licensing costs which certain other HEVC license holders will apply.

But that’s not all. Apple is a member of AOM but noticeably absent is its big rival Samsung. No surprise then that Samsung is the chief and, to our knowledge, only backer of this new video coding standard under proposal at MPEG.

 V-Nova’s codec

The second codec being considered by MPEG is actually an existing technology which acts to enhance the deployment of hardware codecs processing MPEG4/H.264 AVC and HEVC.

It is calling on the industry to propose solutions for ‘low complexity video coding enhancements’ and further explains that the objective is to develop a codec which has a base stream decodable by a hardware decoder overlaid by a software layer which will improve the hardware’s compression efficiency.

The target once again is live and on demand video streaming.

The argument here is sound – instead of waiting for a new codec to come along what if it were possible today to add software which improves compression efficiency and which also saves on broadcasters having to rip and replace their existing (AVC, HEVC) codec investments?

Turns out this is entirely possible since the initiative is designed around the Perseus Plus technology devised by V-Nova.

“MPEG wants to standardise what V-Nova does,” says Fautier. “So far their success was limited, one reason [being] is this is not [yet] a standard based solution.”

Any other vendor can submit a proposal for MPEG evaluation but it’s likely that V-Nova’s will win the day since it’s already in the market and MPEG’s specs fit the Perseus design.

“AVC and HEVC are massively supported by various platforms and hardware solutions/assistance is provided on many devices,” says Christian Timmerer, who speaks for MPEG.

“However, it has been shown that software solutions can enhance quality significantly which can provide a better quality of experience for the end user at lower (or equal) bitrate but with some additional processing efforts to be done on the terminal side.”

V-Nova itself claims that at or below HD resolutions, Perseus improves H.264 performance by more than 40% and at UHD resolutions improves HEVC performances by 70%.

Although its solution would not be royalty free, it will argue that its approach addresses the large installed base of AVC today such that broadcasters don’t have to change their decoder to HEVC or expose themselves to HEVC license terms.

“We welcome the efforts made by both MPEG and AOM to develop codecs, however, we are also conscious of the fact that lead times for new codecs are long and replacement cycles of devices in the field often even longer,” says Fabio Murra, SVP product & marketing, V-Nova.

“As such, this approach represents a game-changing opportunity, now and in the future, to accelerate the deployment of more advanced compression between the longer cycles of codec evolution.”

MPEG’s call for low complexity video coding enhancements is not only intended to enhance any existing video codec but could be used to provide an upgrade to any future codec.

“There is rapidly growing interest in enhancing compression efficiency, and doing so without necessarily requiring new hardware is something the industry and our customers find compelling,” states Murra.

Content Aware Encoding

It is an approach, though, that is competitive with Content Aware Encoding (CAE) for OTT, a technique that also does not need to change the decoder, and is supported by Apple’s operating system for mobile iOS11.

“We believe the most reliable approach is to focus on improving the codec implementation” - Mark Donnigan, Beamr

In contrast to encoding using ABR (Adaptive Bit Rate) where each resolution such as HD or SD is matched to a given bit rate, CAE can make more efficient use of bandwidth, particularly in live streaming scenarios.

The leaders in this field are Harmonic (which markets EyeQ content-aware encoding software), Brightcove (which includes CAE in its online video platform) and Beamr (which deploys a patented content-adaptive bitrate solution, CABR, over HEVC).

“Beamr can reach high quality video at additional savings of as much as 40% over a comparable HEVC encode using our CABR mode, which in my mind obfuscates the need for an ‘enhancement’ layer,” says Mark Donnigan, vice president, marketing, Beamr.

The Isreali and Palo Alto-based outfit has achieved greater than a 50% improvement in the bitrate efficiency of its HEVC codec from first launch five years ago. Since its tech is integrated with the rate-control of the encoder it means that, in theory, CABR could be used with a dual-layer approach.

However, Donnigan maintains that since CABR often leads to bitrate reductions of an additional 30-40%, “a user adopting CABR won’t likely need the added implementation complexity of a dual-layer solution to meet their objective for high quality at lower bitrate.”

He says, “The question that must be asked is whether the benefits of the additional complexity such as the MPEG committee is calling for, are worth it? We believe the most reliable approach is to focus on improving the codec implementation, which when combined with content-adaptive technology such as CABR, will yield a benefit for all devices with HEVC decoders today - no updates required.”

He argues that the importance of playback compatibility is often overlooked by every new codec or technology enhancement.

“As a striking example, this is the situation right now with AV1, where there are encoding vendors offering solutions for creating AV1 compatible files, but - outside of a very narrow combination of a desktop computer and a beta browser - the files cannot be played back where consumers want to consume the content such as on a mobile device, TV, game console or media player.

“The fact is, with a well-designed implementation of an existing codec such as HEVC, [new codecs] are not required,” he says.

Donnigan believes that MPEG’s proposal for low complexity video coding enhancements is “an academic initiative” that will not have commercial appeal because the complexities of developing and implementing it outweigh the codec efficiency that he says Beamr delivers today.

“And we deliver efficiency completely inside the standard of HEVC with no changes required to the decoder or player application.”

Whatever happens with the Samsung or the V-Nova promoted proposals, it will be many months before either gains standard status as the proposals move within MPEG. It’s not a sluggish organisation – far from it. It’s got multiple standards efforts on the go all the time for which the process includes working drafts in multiple iterations until a final draft gets the vote of national bodies. According to Timmerer, that could take two years (or more) depending on the number of contributions.

By then in 2020 MPEG’s main next gen codec initiative will be about due. This is VVC (Versatile Video Codec) which could well trump every other codec out there.

What’s more there’s an independent body to police the whole situation and prevent the mess which the industry has got itself into with the lack of clarity around HEVC licensing.

The Media Coding Industry Forum (MC-IF), launched at IBC2018, and has companies like Canon, MediaKind, Sony, Nokia as well as Apple on board. HEVC Advance, one of the HEVC patent holding pools is also a member.

Speaking without his MPEG hat, Timmerer admits, “We are entering a situation with multiple codecs and this might be a problem in terms of market fragmentation but usually the market will make its decision one way or another.”

Referring to AV1 and other codec upstarts, Fautier says “VVC will wipe out all these monkey codecs, assuming it can be licensed to everyone’s satisfaction. Development has to go hand in hand with common sense.”