Tech Paper authors discuss entries at IBC SHOWCASE

The authors of IBC 2020 Technical Papers have taken part in SHOWCASE sessions discussing their entries.

The Tech Paper sessions, which are available on-demand as part of IBC SHOWCASE, offer insight into the problems and solutions outlined by the latest batch of technical papers.

This year’s Technical Papers cover a vast array of subjects ranging from drones to sustainability. The papers are available to read at IBC365.

The Technical Papers are an important element of IBC, providing an opportunity for forward-thinking technologists and companies to unveil their ideas and research to media industry leaders hungry for new technology concepts, their possible uses and practical applications.

Entries came from sectors of the media, entertainment and technology industry, with all submissions rigorously reviewed by a panel of experts.

Dr Paul Entwistle, chair of IBC’s Technical Papers Committee, said the most popular topics this year included virtual and augmented reality, including 3D conferencing, 3D capture and the necessary delivery frameworks.

“We are also beginning to see how this technology can move from niche to mainstream as compelling experiences are blended with traditional television,” he said.

“Video compression, a cornerstone technology of our industry continues to improve and surprise. And the application of artificial intelligence and machine learning goes from strength to strength with papers spanning production assistance - including a synthetic news reader - to picture enhancement.”

Last year, the IBC2019 Best Conference Paper prize was scooped by Nokia Technologies and Innogiant for its research into real-time decoding and immersive augmented reality experiences for mobile devices.

Find out more about the technical papers sessions below.

 

The Drones are Swarming

Today it is hard to find a TV programme which does not include stunning aerial shots from drones. Recently, however, signal processing experts have become excited at the potential of using multiple-drone swarms to capture even more stunning 3D objects on the scale of cathedrals or even whole landscapes. ‘Photogrammetry’ on this scale poses complex problems of: flight planning, autonomous drone control, positional sensing (via GPS, barometer, accelerometer and compass), scene capture, surface modelling and data management - but fortunately, a new range of tools aims make the whole process comfortingly practical. Our exploration of the latest research in drone technology includes development of a simulation environment for scene capture and a multi-drone control dashboard for the production director.

Evolving Trends in New Media Concepts

It was once the case that media technologists and media content creators rarely met. Today their combined innovative talents are sparking many experimental forms of media, as well as new tools with which to produce them. One such form is personalisable media, where users can shape their own version of a programme from the single, rich, multiple-representational format which is broadcast. An example of this is object-based media which may allow choices of multiple screens or alternative narratives.

Other novel concepts employ AI to generate metadata or to create productions which would be impossibly time-consuming for mere humans. An example is the ability to select and edit audio-visual content submitted by viewers. Think, for example, of the community applause items broadcast during the COVID pandemic; selected videos from the public could be included in professional broadcasts within minutes of the event, despite the huge volumes received. Think also, of an entirely automated news summary; always up-to-the-minute and fronted by a virtual presenter. In this session we bring together work from Europe and South America to celebrate the latest in innovative media concepts and to assess their potential impacts on workflow.

The Versatility of Volumetric Video

It is not an exaggeration to say that volumetric video processing is fast-heralding a revolution in media technology through its versatility. What began largely as a form of immersive entertainment has grown into a huge number of applications, including: gaming, sport analytics, cultural heritage, education, medicine, animation, e-commerce, industrial visualisation and personal communication.

In this fascinating glimpse of the state-of-the-art in volumetric capture and processing we shall examine three active areas. First, new work which unveils the idea of animatable volumetric content and then shows how it may be captured, animated, rendered and streamed using hybrid geometry-video methods. The substantial volumetric processing involved is off-loaded to a cloud-based server. Second, we examine the precision requirements of a many-hundred-camera array for the capturing of sporting events, either for free-viewpoint spectating or for analytical purposes. And third, we explore the very latest international work on the challenging topic of person-to-person communication in virtual space – 3D holoconferencing – with a look at the different social applications that it will make possible.

Extended Reality Media Experiences

Committed to keeping media industry professionals informed about new developments, IBC always seeks to present updates by leading contributors to international standards research. Within this session is the very latest of these updates; a thorough look at recent developments in MPEG-I relating to extended reality XR (a general term embracing virtual and augmented reality systems). Here, our world-level expert will describe the latest in: immersive video streaming systems and interfaces, the concept of a Scene Description and how this Description will allow us to compose a 3D scene from 3D objects.

By way of demonstrating the power of XR, the session continues with a presentation by NHK showing how they are able to enhance their 8k UHDTV viewing experience with simultaneous 6-degrees-of-freedom volumetric video. This video is delivered via the domestic internet and viewed via a personal mobile device. NHK describes this as ‘volumetric tele-existence which brings the performers and viewers into the content world’. Theirs is a fascinating and detailed story of the processes involved in producing, streaming and time-synchronising the two media. This culminates in a prototype through which we can watch as our favourite actor seamlessly steps through the TV screen and into our living room!

Super-resolution: filling in the missing detail with machine learning

In this session we look at two state of the art machine learning based video enhancement systems. The paper from SK Telekom reports on a tool that performs up-conversion, frame interpolation and aspect ratio adjustment, common tasks for any content provider. Whilst the paper from the BBC demonstrates a machine learnt up-conversion filter, offering a demonstrably useful improvement in modern video codec design and asks, can you trust your algorithm? Both a serious and a fascinating question when you consider the risk of bias as your algorithm goes about “improving” skin tones and “enhancing” facial features. There are also two excellent supporting papers from RheinMain University and Shanghai Jiao Tong University, which between them discuss machine learnt inverse tone mapping and gamut extension (of particular relevance in HDR) as well as up-conversion.

Video coding: Can new codecs VVC and EVC actually deliver?

Before the dust has settled on HEVC, new codec VVC has reached final draft showing a 40% coding improvement. In this session we present two papers - the first an exemplary paper from Interdigital provides not only a description of VVC’s new technical tools, but also a performance analysis of their individual coding gains. The second, from Ateme asks, so can we now deliver 8K? and considers both the traditionally developed codec VVC, as well as EVC, the codec designed to ease licensing issues. In addition to the presented papers, this session champions two supporting papers – an adaptive quantizer from Comcast which considers the Human Visual System when targeting mobile devices with HDR, and a further 8K delivery study from B-com, comparing traditional codecs against machine learnt up-conversion.

Sustainability in public service media – an opportunity to show leadership?

In this presentation, the EBU looks beyond energy efficiency and the consumptive expansion it tends to drive and investigates the broader means to sustainable behaviour. For many commercial organisations, the idea of driving a reduction in consumption or moderating technological advancement (beyond the need for cost reduction / profit maximisation) is at best alien, if not contrary to their purpose. However, for public service media, with their unique role and public service mandate it could be an opportunity to showcase sustainability leadership – and drive the conversation onto how to measure and present this benefit, as they pursue their goal to inform, educate and entertain all of us.

Trust: protecting your news brand from manipulated content in our social media world

In this presentation a consortia of companies come together to present their solution to the problem of fake news - a multifaceted issue from a prank edit to political manipulation, virally spread on social media. Focused on establishing provenance, the authors not only outline a technical ecosystem, but talk to the complexity of user education and awareness. Can this be done at scale? quickly? accessible to all content providers small as well as large? and protect end-user privacy? Learn from the experts as they seek to tackle the fake news challenge that becomes increasingly important each and every day.