UHD, immersive audio, 5G and virtualised OBs: all the broadcasting innovation at the Beijing 2022 Winter Games

The Winter Olympics features a UHD HDR production and immersive 5.1.4 audio; live virtual reality; 70+ hours of 8K and an experimental virtualised OB van as part of a record 6,000+ hours of content produced for the event.

For all that, it is the use of 5G in Beijing which may prove most game-changing for wider live production.

Host broadcaster OBS is promising, as it did with the Tokyo Games, the most immersive televised Winter Olympics yet. Many of the same tech advances deployed in Japan pop up again.

Here’s everything you need to know.

UHD

Tokyo 2020 set a new benchmark for an Olympic broadcast in being a full UHD HDR production. This is being doubled down in Beijing with a single UHD HDR workflow from which HD SDR will be derived. OBS claim this to be a ‘full native UHD’ set up but admits that – as in Tokyo – it will also rely on several specialty cameras “that at this time can only operate in HD 1080p SDR.”

“Virtualisation will redefine broadcast production requirements and workflows and simplify them,” Sotiris Salamouris, OBS

In fact, the number of HD contribution feeds (41) it is creating outweighs that of UHD (31) with presumably HD to UHD conversion of numerous cameras taking place to get to a master UHD format. It will prepare 43 HD feeds for international distribution compared to 36 UHD.

The HDR to SDR conversion includes a set of look-up tables composed in-house to enable better interoperability between the two co-existing HD SDR and UHD HDR workflows. The UHD production will adhere to SMPTE 2036-1 at 50 Hz with HDR in Hybrid-Log Gamma (HLG).

“Although our production workflow is greatly simplified by having a single workflow model, achieving consistency across all HDR and SDR sources remains quite a complex undertaking on a job of this scale,” says Isidoro Moreno OBS’ head of engineering.

Immersive audio

This will be first Winter Olympics captured through a 5.1.4 audio configuration, ideally giving viewers a more realistic audio experience. To the 5.1 surround sound mix is added an overhead sound layer, from four hanging ceiling microphones with adjustable heights. In total, OBS will be using more than 1,600 mics (40 different models). Two audio QC rooms installed inside the IBC (international broadcast centre) will guarantee quality consistency across all sports. The 5.1.4 audio configuration will be provided for both HD and UHD.

“In Tokyo, this immersive audio set-up helped mitigate the absence of spectators,” says OBS CEO Yiannis Exarchos. “However, in Beijing, there will be limited spectators in the venues and their presence will certainly enhance our coverage.”

Digital transformation

Underpinning all the technical capability of OBS’ move to remote and virtual workflows is the switch from traditional broadcast hardware to one that’s fully IP-based. This change was completed a couple of years ago. Equally important is a broadcast-specific cloud-based platform that OBS created with Alibaba. Content delivery, signal processing, post-production and many other elements are now based in the cloud.

Excharcos says, “For the first time, OBS will distribute the multilateral signals in HD and UHD via the cloud, and it will also be the first time that rights holders will have the ability to edit the content available on our online distribution platform [Content+] remotely. It means a more efficient way of working and addresses broadcaster’s huge demand for content to share on their digital platforms, without having to multiply their resources.”

Virtualised outside broadcast test

An important project for Beijing 2022 is a virtualised OB van which has a control room based on virtualised “cloud-ready” technologies. “We are trying to replicate the way an OB functions with reducing the physical broadcast footprint to the minimum,” explains Exarchos. “This pilot programme will be based at the curling venue. Depending on the results, we could use such a set-up at future Games. Not only does it offer greater flexibility and scalability, but the OB operations could be performed in a much more sustainable way than having a huge number of trucks coming from all around the world.”

The first stage of this project prioritises functionality and interoperability, as well as ingesting and processing of 1080p50 SDR video feeds from 18 cameras used for the coverage of one of the sheets at curling. Four additional native IP cameras, dedicated to the virtualised OB van project, will be connected to the network stack, eliminating the need for camera control units.

“Virtualisation will redefine broadcast production requirements and workflows and simplify them,” says CTO Sotiris Salamouris. “It will also reduce the set-up time to the minimum, given that the systems can be configured remotely before moving to the host city. In future, one could imagine having more flexible production crews, not necessarily based in the venue compounds, but perhaps operating from their own country premises, eliminating the need for dedicated full production crews on-site and avoiding the long-term rental of worldwide broadcast equipment for the live production of the Games.”

Nonetheless 15 conventional OB vans plus one flyaway system and nine ‘field datacentre production units’ will be deployed in Beijing.

Primetime for 5G

OBS is able to expand its traditional camera positions at these Games because of more widespread 5G coverage.

“5G has now reached a level of sophistication where it can be applied to production in earnest,” says Mario Reis, director of telecommunications. “Thanks to the full-scale implementation of a 5G network across all Beijing 2022 Olympic venues, we can now use its capabilities in our live coverage. It provides our production teams with greater flexibility than having to tether and plug in wired cameras, and the added mobility of camera positions will help OBS capture the action from unique angles.”

Where, in Tokyo, 5G-connected cameras were only used for ENG coverage at the Ceremonies, for Beijing 2022 production teams will deliver live signals over 5G from more than 30 live and near-live cameras, including those fitted on snowmobiles at crosscountry skiing and those at the start and finish areas at alpine skiing. 5G-connected cameras will also be used as part of the virtualised OB van project to capture the action from curling.

Intel has helped select the 5G transmission and receive technology (encoders and modems) that goes with those cameras. China Unicom has established the telecom networks required to send the 5G signals back to the IBC, as well as tweaking the established network to allow OBS to do all this in real-time with low latency transmission from cameras to production units.

“5G has now reached a level of sophistication where it can be applied to production in earnest,” Mario Reis, OBS

“5G has great capacity to support low latency and high bandwidth live broadcast transmissions over a public infrastructure,” says Salamouris. “This is certainly a key enabler for field production, especially considering the limitations of the legacy broadcast solutions that rely on dedicated radio frequencies that become scarcer and scarcer.

“However, there are several challenges for this approach to take off, since it requires certain network configurations that the carriers should adopt to secure the necessary availability levels that broadcast requires. On that front, China Unicom, with Intel, was really instrumental helping us engineer a solution appropriate for our quite demanding needs.”

Going remote

For Beijing 2022, the majority of the broadcasters will be using remote production to run all or part of their production outside of China. Driven in part by the necessities of the pandemic, but also the shift to remote production workflows, the size of broadcast teams being sent out to the Games has fallen dramatically - nearly 40 percent fewer broadcast personnel on-site in Beijing compared to PyeongChang 2018. For the first time, more than 20 broadcasters will receive feeds in real-time at their centralised production house back home through the cloud.

• Read more OBS boss hails early cloud adoption

“The fact that broadcasters have shown significant interest for this new service is indicative of the growing integration of cloud-based workflows and how broadcasters are offloading more of their traditional video infrastructure to the cloud,” says Exarchos.

The OBS Video Server is now fully hosted in the cloud to provide “a more efficient and scalable system, while reducing on-site hardware and all costs associated.” This is also first time that OBS’ Multi-channel Distribution Service will be distributed via cloud as well as satellite.

All in all, this means that for the first time in Olympic broadcasting, the distribution of live signals over the cloud will be of equal volume as through standard delivery models.

8K

China Media Group (CMG) and NHK (rights holders for China and Japan, respectively) will collaborate to produce 8K coverage from select venues as well as from the Bird’s Nest Stadium. NHK is producing the figure skating events; CMG takes charge of the Ceremonies, freestyle skiing/ snowboard big air and speed skating in 8K. A feature of the Main Media Centre (MMC) is a 15 x 8 metre 7,680 x 4,320 pixel display for showing off the results.

“Early Olympics trials of 8K were simplistic,” says Salamouris, “in that we were only using a few cameras, and only one type of production unit. Now the whole 8K production has really matured and the technology around it is moving quickly so that it will eventually become an option for more and more broadcasters.”

8K VR

Beijing 2022 marks the first ever multi-sport event to be covered in 8K VR. 70-80 hours of the format will be produced from sports selected on the ability to place cameras close to the athletes.

Up to six 180-degree monoscopic cameras and one 360-degree camera will be in action. Viewers will be able to choose camera perspectives of live streams or watch a produced stream on VR headsets Oculus Quest/Quest 2 and Pico, and also on mobile. VR VOD content will be available. Broadcasters are further invited to use the 8K VR feeds as presentation backdrops.

5G plays into the evolution of VR at the Games. High downstream bandwidth from 5G will “free” consumption of content to be happening anywhere, and not just where Wi-Fi is available.

“A few years ago, it would have been hard to discern faces and other details while using VR, but now with 8K resolution the experience is far more lifelike,” Salamouris says. “However, that increased resolution can only be available to the user if they have the network to support it. With 5G you have the necessary quality to transmit to final users, be it on a 5G enabled VR device.”

AI / ML

AI tools will be used in figure skating and ice hockey to speed highlight package creation. “We are moving closer to being able to integrate AI technology as part of our toolsets and using it for our video tagging workflow,” says Salamouris. “If this content isn’t properly tagged, then it is very difficult to work with. For a long time, we’ve employed students to tag our live content. While we wouldn’t replace those students with an algorithm, AI would allow us to tag far more content and offer ultimate flexibility and expandability compared to the capabilities of human-only loggers.”

OBS in-house technology, which it began developing ahead of Tokyo, is called Automatic Media Description. “We train the system to automatically search for specific content/video sequences, and once indexed, stitch this content together to produce quick highlights packages which are made available to OBS producers,” he explains.

Although some athlete tagging is currently achieved by OBS loggers, it is practically impossible to tag all athletes in all available video frames, says Salamouris, but this is often what rights holders require.

“After Beijing 2022, we will start experimenting with automatic switching, which would mean using AI in live broadcast operations,” he says.

Data visualisation

Specifically for Curling, data gathered from an overhead camera will visualise stone trajectories, contact points, and distance between stones for on-screen analysis. In other disciplines, real time ‘Jump’ data such as speed, height, length, duration and angles of skis is collected from motion sensors and processed by computer vision analysis.

2D image tracking

After being introduced at Tokyo, 2D image tracking (also referred to as athlete ‘pinning’) technology crops up again in Beijing, for the biathlon and crosscountry skiing events.

A ‘patch’ (a square) is defined on selected video frames to identify each of the athletes. The computer then creates a ‘label’ that is attached to each of the identified athletes which is maintained even as the image changes. This captured data is then made available to a graphics rendering platform for on-screen presentation, displaying the exact location of the athletes.

• Read more Inside Discovery’s Winter Olympics broadcast plans

Additional data captured using more traditional GPS positioning can be combined with the ‘labels’ to identify athletes, their speed, distance to finish or relative position to the leader.

Live speed measurement

Speed measurement in the coverage of alpine skiing events was introduced as part of the TV graphics at Sochi 2014. Until now, the speed was measured from a very specific position and limited to only snapshots, or just a few seconds of data. This was due to the limited sensor coverage. For Beijing, OBS will deploy a multitude of antennae with increased reception capacity that allow for the capture of more data throughout a much larger portion of the downhill course. The use of such technology will also guarantee an overall better speed measurement in terms of accuracy and frequency of updates.

Multi-Cam Replays

Arrays of 4K high speed cameras will be deployed at ten venues including at Figure Skating, Ice Hockey, Freestyle Skiing/Snowboard Halfpipe and Short Track Speed Skating. Another system will be near the end of the take-off ramp of the space-age looking ski jumping facility to capture the first seconds of the skier’s flight.

These replays can be paused at different points in motion, an effect “similar to action scenes in the The Matrix,” says OBS. A rig with 120 4K cameras will be used at the Ice Hockey venue.

The rigs are remotely operated by a single operator who can freeze the action, manipulate the replay from side to side around the athlete, as well as zoom in. Since the system simply stitches together these feeds and does not have to virtually create filler frames, no rendering is required, allowing clips to be ready in under five seconds.

For curling and speed skating, OBS will process multi-camera replays in the cloud, the first time this has been attempted live. All frames captured by the array of cameras installed at these venues will be sent to an edge server and reconstructed in the Alibaba Cloud to generate the replay clips. Those will be up-converted to 4K in the cloud before being sent back to the production unit in the venue compound.

“In some cases, you need multiple cameras that are placed around an object and fly the cameras around, but you also have the volumetric technology that Intel has developed, which records and recreates a model of a solid object,” informs Salamouris. “In the first case, the flight camera pattern is more fixed, whereas in the second case, the producer has much greater flexibility in terms of the flight camera trace that you can build around it.”

While the volumetric approach is much more flexible and richer in terms of produced results, it comes with significant computational complexity that also leads to much higher turnaround times, compared to the simpler ‘stitching-based’ method. In Beijing, OBS will deploy both technologies.

Gender equality

Beijing 2022 will feature the closest ratio of female to male participation at a Winter Games, with female athletes making up about 45 percent of those competing. New mixed gender events include snowboard cross mixed team, ski jumping mixed team event, and freestyle skiing mixed team aerials. More gender balanced coverage is on editorial minds at OBS; the term ‘women’ will be used across all sports, rather than ‘ladies.’

“Sport media has traditionally been male dominated and we will continue to keep up our efforts to open new opportunities for females to work in sports broadcasting so that eventually we have a 50-50 split between women and men,” Exarchos says. “We are not there yet, but we are on the right path and would encourage our fellow broadcasters to follow our example.”