IBC2021 Accelerators: 5G LBXR

This Accelerator project breaks new ground, and has been designed to examine how 5G’s low latency and provision of edge computing in particular can amplify important aspects of LBXR’s (Location-Based eXtended Realities) immersive interactivity, technical feasibility, inclusion, and accessibility.

Project lead is Xiangyu Lian, chief engineer of the Telenet Innovation Center based in Brussels.

I wouldn’t say we have everything we need for this use case, but for a technical trial we are on a good track,” he says, arguing that 5G in Europe is rolling out more slowly than anticipated due to a range of issues including Covid and security concerns involving Chinese vendors.

This has led to an extended period of test and evaluation, which is perhaps no bad thing as there are important considerations still to be decided with several aspects of 5G deployment which the Accelerator in part will address.

“For an example, from an operator’s side we can either way we own the edge computing units, or we say we don’t need to fully own them, we can share them but just need to create a local breakout. That way the traffic doesn’t need to go the full loop, you have a breakout point that directs you to the nearest edge computing resource, and that can be shared with different operators, or be accessed via cloud infrastructure. There are quite a lot of options and the exact solutions that will be used depends on market, preferences, budget and more.”

Games without frontiers

There are a range of different workflow scenarios that the LBXR Accelerator is exploring with relation to 5G.

It is looking at testing and validate 5G’s connectivity towards XR equipment via different 5G devices, building a streaming solution from cloud infrastructure to XR equipment for different XR content, and providing access to 5G networks and cloud infrastructures from user case locations to multiple locations across Europe, as well as other potential locations around the world.

Xiangyu points to the games market as an ideal test bed for 5G use cases in general, due to its requirements for low latency, high bandwidth, and serious computing power, as well as an audience that is willing to pay for premium access, assuring rapid ROI.

Read more IBC 2021 Accelerators: 5G and remote production in live sport

Hado represents precisely the sort of new use case that has been brought about by technological confluence and that will find itself accelerated by 5G. Billed as the world’s first physical esport and growing rapidly worldwide, its players compete on a physical court while using head-mounted displays and armband sensors to both see and wield energy balls and shields. Currently it operates in a closed arena on a 5Ghz network, but the Accelerator is looking at implementing a remote play POC.

Remote play experiments where the two teams compete in different arenas have been tried before, but while all the AR elements from the other team can be seen, the players can’t.

“We will have two teams in two different locations, and the players and spectators will be able to see a single court created from the data with avatars representing the players,” Jim Sephton, Hado

“What we are doing through the Accelerator is solving that,” says Jim Sephton, ex UK team captain and in charge of Augmented Reality sports development with Hado UK.

“We’re working with Noitom to scan the players in realtime and we will be merging that data with the Hado gameplay data for the output to viewers. So, we will have two teams in two different locations, and the players and spectators will be able to see a single court created from the data with avatars representing the players.”

• Read more Comment: Introducing the 2021 IBC Accelerators

There are technical hurdles to this. While neither Hado or Noitom create large datasets, perfect synchronisation with no data dropout is vital before the two streams are brought together for real-time rendering. 5G’s low latency is therefore a key enabler, while edge computing is also going to be vital for the rendering part of the process.

As Roch Nakajima, President of Noitom International says: “You want to use 5G for the best reason, you don’t want to waste it. If I can render locally then it’s going to be much easier than rendering and pushing material through a finite pipeline.”

Workflow choices

Integration is also an issue, as the pipelines for providing the realtime avatars to the players’ and the spectators’ devices are different. “There is no universal language yet for all the different parts of the workflow to speak to each other,” explains Nakajima. “But because you’re eventually typically trying to render something in Unreal or Unity, ultimately you have to speak one of those languages. The issue we currently have with the POC is that Hado is Unity-based and Weavr is Unreal-based.”

Weavr is the technology platform put together by a consortium including leading esports company ESL, amongst others, that is being used for the spectator views. “We need to be able to feed data to both platforms so that the Hado players can see each other, but all this needs to be coordinated,” says Nakajima. “There is more data, different languages that are being spoken, plugins that need to connect things, and more.”

Nevertheless, understanding the workflows necessary for scaling to large audiences via B2B and B2C consumer models is an important part of the POC. As Sephton points out, the potential for innovative interactive and immersive viewing is immense; viewers will not only be able to scrub backwards and forwards in the action, but will also be able to rotate the view around 360 degrees, zoom themselves in for a POV perspective of individual plays, and more.

Porting that technology to other sports is an obvious long-term ambition, but not an easy one in turn. It works in this use case because Hado already provides the full telemetry data. Providing that for a conventional sport, even for one with a comparatively constrained playing surface such as tennis, would be decidedly non-trivial, particularly as scanning the ball’s movements in realtime would be extremely challenging. An issue for a future Accelerator perhaps.