IBC2019 technical papers: Immersive

Some 56 matches during the 2018 FIFA World Cup were live streamed to connected television with 4K UHD HDR with immersive audio.

This technical paper presents the experiences, challenges, techniques and results of the immersive audio live streaming obtained by Globosat, in partnership with Samsung Brazil.

The agreement between the two parties involved saw the live streaming of 4K UHD HDR signal produced by Globosat with immersive audio mixing and another feed with only the 4K UHD SDR 180 degrees central camera signal with international audio.

The 4K UHD HDR live production at Globosat consisted of audio mixing for immersive audio and SPORTV (sports channel) talents (announcer and sports analyst), branding and commercial insertions. The opening of the live streaming occurred 15 minutes before the start of the game and closed 5 minutes after the end of the game.

The technical paper by i2 media research ltd, c/o Psychology Department, Goldsmiths University of London introduces the Immersive Audience Experience Evaluation Toolkit, an online questionnaire designed to measure audience perceptions of impact and value of different immersive experiences.

Initially created and developed in 2017, the Toolkit has been applied to various immersive media formats and contents such as virtual reality (VR), augmented reality (AR), mixed reality (MR), games and screen-based media. With examples of insights generated for creators and funders of immersive experiences, this paper summarises the need for, development and refinement of the Toolkit.

Drawing from audience evaluations with 460 people using 10 different immersive contents, the added value of the Toolkit to the creative industry of further exploring aggregated and comparative ratings is also discussed.

While, significant advances in content production technologies, video compression, rendering and displaying are enabling the creation of a new generation of immersive environments able to exhibit media with an unprecedented level of quality and offering a much stronger sense of reality.

The emerging immersive environments are by design very heterogeneous, including among others: large screen visualisation systems with flat and curved screens, dome projection and cylindrical and cubic caves with large field-of-view (FoV) capabilities.

This technical paper presents a set of tools designed to simplify the distribution, playback and adaptation of high-quality immersive content for new immersive environments. The tools are based on a high-performance HEVC codec that allows to encode, decode and play ultra-high-resolution video in 8K and 16K resolutions using standard computing systems.

In addition, a flexible video rendering engine allows adapting the content to different output display environments. The new tools have been validated in two advanced immersive installations and using new immersive content created with different techniques (8K camera, 3D CGI, and point clouds) that have been produced in the context of the Immersify European Project.

Omnidirectional MediA Format (OMAF) is a media standard for 360° media content developed by the Moving Picture Experts Group (MPEG). More complex and tailored multimedia services are needed for advanced media processing and delivery such as virtual reality content stitching, packaging, and adaptive streaming.

To achieve the desired result, these complex workflows require many advanced functionalities to work together on the media content. In order to address the needs of advanced services, MPEG is also developing a new standard called Network-based Media Processing (NBMP), a standard that aims at increased media processing efficiency, faster and lower-cost deployment of interoperable media processing functions and the ability to provide large scale deployment by leveraging the public, private or hybrid cloud services.

This technical paper covers both OMAF and NBMP standards. Additionally, end-to-end design and proof of concept is provided to enable an immersive virtual reality experience to the end-users.