IBC2023 Accelerator Project: 5G Mocap for Performance Arts and Live 3D Animation

No comments

Two previous Accelerator Award powerhouse winning teams have combined forces to harness the powers of 5G indoors, this time focusing on two unique, creative workstreams that will both power live, cutting-edge motion capture and low-latency connectivity into creative output, focusing on the performing arts, and 3D animation into children’s hospitals.

The project is led by Champions including Vodafone Group, Curing Kids Cancer, Outernet Global, Production Park, University of Strathclyde, Neutral Wireless, Goldsmiths University, with D&B Solutions, Noitom, AMD as the project’s Participants.

The first Performing Arts workstream will use the latest in FPGA (Field Programmable Gate Array) innovations and will aim to demonstrate Ultra Low Latency 5G private networks to support the creation of new, immersive audience experiences for those at a live (or, as-live) performance/venue, and those engaging remotely in another performance/venue.

The second workstream for Live 3D Animation will aim to demonstrate a sustainable, accessible and inclusive way to entertain children who need it most and show how IP holders can bring animated worlds to hospital wards with sick children anywhere in the world, using a system that does not require costly render farms, rely on outsourcing production or cause studio burnout. It will also showcase how new formats of ‘live’ kid’s content can be created with meaningful, two-way interaction between the children and the animated characters.

Muki Kulhan, IBC’s Innovation Co-Lead said: “Motion capture has grown from a niche filmmaking tool into one of the most in-demand and necessary production workflows across media and entertainment formats, including live dance performance, and interactive 3D animation, both of which feature in this very human-centred IBC Accelerator.”

“This Challenge has merged two previous Project of the Year Accelerator Award winners who are truly pioneering private 5G innovation and creative Mocap IP, this time exploring how 5G’s low latencies can effect and sync the movements of professional dancers in global locations, and also how this speed via 3D Animation can impact and interact with sick children in cancer hospitals, a truly inspiring, tech for good initiative”, she concluded.

Project Objectives

Objectives Workstream 1: 5G & Live Performing Arts

Experiment in building new “smoke and mirror” low latency audio and video ‘illusions’ that can be piped from ‘anywhere’ to ‘everywhere’ over appropriate back haul, using terrestrial or celestial Public Internet
Innovate and explore new ways to interact with a ‘Live’ performance with Motion Capture and VR projection techniques
Speed of Sound and Digital Video Compression don’t mix that well, every frame of video is 16 feet of sound travelled, 3 frames best for 4K UHD so audio and vision in sync at 50 feet, requires large screens sizes via 4K UHD laser projection
Highly mobile, quick to rig in overcoming cost prohibitive cable runs (easing H&S requirements) enabling new creative freedoms in being able to open up venues that can’t be cabled
Be clean as possible, following on from 2022’s ‘low’ carbon approach (via 5G in the Middle of Nowhere project)

Objectives Workstream 2: 5G & Live Animation

Utilise AniSign and motion capture to perform the animated character in real-time
Use 5G to enable performers and audiences to connect anywhere
Present on a high resolution (8K?) display to heighten the immersion
Motion-track the display so it can be “placed” in the animated world
A live performance system to trigger changes in the animated world in response to the child’s emotions
Disguise the screen to dial up the magic

Catch the 5G Motion Capture For Live Performance And Animation project, showcasing its PoC and findings at IBC2023, on Friday 15 September, from 11:15-12:15 at the Innovation Stage.