CAPA II Accelerator project: Revolutionizing live broadcast for Formula E

No comments

The IBC Accelerator special incubator project Connect and Produce Anywhere, Phase II (CAPA II) had an early run out at the recent Formula E London ePrix. John Maxwell Hobbs reports.

With onsite involvement from the BBC, Channel 4, VizRT, LAMA, TSL, Techex, and remote participation from Google Cloud, Norsk, Vodafone, and others, the CAPA II project team demonstrated how live coverage of a major sporting event could be covered using only software-based production tools.

CAPA-Formula E - 1

Connect and Produce Anywhere: Formula E

Remote workflows and flexibility

Describing the CAPA II setup for the London ePrix, Ian Wagdin, Senior Technology Transfer Manager from the BBC, one of the project Champions, said: “We’ve got a Google Cloud Cluster running locally on commodity HP and AMD hardware, and that’s running a whole bunch of software. It’s part of our aims and ambitions to generally think about software-based production running on edge infrastructure rather than traditional hardware-based production,” he said. “So, although we’ve got some control surfaces which are quite specialised, all of the applications are running on software. The advantage of edge infrastructure is that we can take a cloud cluster out to a location and run high throughput processing locally, which means we don’t require the same bandwidth as sending all the camera feeds back to a centralised control centre, and we can also reduce the latency.”

Ian Wagdin headshot

Ian Wagdin, BBC

He described some of the benefits of using this approach: “Here, we’re plugged into a local connection, but equally, that could be over a basic internet connection, and that means we can choose to operate remotely or from a broadcast centre anywhere in the world. The underlying principle really comes back to that dynamic media factory type of work, where if you deploy an IoT-based infrastructure stack, you can rely on the IoT side of things to deal with what it’s really good at, like processing, and connectivity, and security. And then we simply run our media applications over the top of that. And we run exactly the same way as any other business, rather than having to build specialised hardware-based trucks.”

Cost-effective solutions for lower-tier sports

The project also aims to make media production more accessible and affordable for lower-tier sports and smaller broadcasting operations. Wagdin highlighted this objective: “What we’re trying to do is understand what this means for lower-cost OBs which is really a massive driver.” By focusing on reducing costs and automating processes, the project seeks to democratise high-quality media production. Automation not only reduces the need for extensive manual intervention, but also enables broadcasters to handle a larger volume of events efficiently.

Key participants and contributions

Channel Four: The broadcaster played a crucial role in the project management and innovation aspects of the Formula E project. Tia Shenton, an apprentice Broadcast Engineer at Channel Four who participated in the event, explained the reasoning behind the broadcaster’s involvement: “Channel Four does not produce content directly but aims to leverage new technologies to support their creative partners,” she said. “We are interested in using these advancements to produce content in cost-effective ways, particularly for projects with limited budgets.”

VizRT: Liam Hayter, Product Manager at VizRT, described the company’s pivotal role in deploying the TriCaster Vector, the software-only version of its vision mixing product within Google’s distributed cloud. “It’s managing all the different types of feeds - not just NDI,” he said. “We’ve got 2110, SDI conversion and SRT coming in. We have Dante and NDI audio being thrown into the mix. So, we’ve shown the flexibility of the software with that. At the same time, we’ve been providing our FreePlay instant replay system virtualised for the first time in the Google Cloud.”

BBC: The BBC contributed significantly through its Connectivity Architecture department. Ivan Hassan, Lead Architect Innovation, discussed the BBC’s focus on managing the connectivity that supports the operations within the project. He also emphasised the benefits of collaborative projects such as this one. “This project brings together a number of key areas the BBC are looking into,” he said. “This creates an agnostic environment that enables the BBC to get the best value for the licence fee, and, of course, it means the whole industry comes together and we can move the industry forward, as a whole.”

TSL Products: Describing TSL Products’ experience in providing broadcast monitoring and control tools for the project, Control Solutions Architect Maurice Snell said: “We are definitely learning. We have done cloud deployments before, but they’ve all been AWS, so there are a number of technical hurdles to get into Google’s cloud environment. And it’s good that we’re learning here rather than for the customer desperate to go on air. We’re growing our protocol library. We’re growing our deployment options to add Google, so that’s good for us.”

Lama: Audio production software company Lama has taken on a major challenge with the Formula E project. Ewan Cameron, the Co-Founder of the company, highlighted the excitement and potential of the new tools provided by Lama. “We’ve been extremely gagging to do this,” he said. “Because we’ve heard about these wonderful tools and we want to go out there and start playing with them.” Cameron further elaborated on the benefits of cloud-based and local audio processing, adding: “Normally this will be done by mixing physically on site, and where it’s actually being processed, but also where you can actually mix it from as well. You can literally load the same projects on your own on the laptop itself as on the VM (virtual machine) as on Amazon or Azure or wherever you want to move.”

Jeroen Dieks, Chief Product Officer and Founder of Lama, described the setup: “We’re taking in Dante, and then through Lama Connect, our virtual patchbay, we’re converting it to NDI and then sending the NDI to the cluster,” he said. “And on the cluster, we’re running a VM that also runs Lama Connect and AutoMix, our mixing software. We can actually switch between what mix we want to listen to, so it’s quite a complex workflow.”

In addition to audio mixing, Lama introduced a groundbreaking decentralised intercom solution. Dieks explained the innovation: “We are now in the middle of developing an intercom solution that’s basically decentralised so that every node has its own audio matrix and also its own monitor section.” This development enhanced the flexibility and functionality of the communication systems, allowing for more efficient and adaptable operations across the project.

Formula E operations

Wagdin highlighted how Formula E’s pre-existing IP-based setup facilitated the project’s integration. “Formula E’s use of an IP model to manage its broadcast operations made it easier to provide the necessary feeds for experimentation and development within the project,” he said. “This existing infrastructure enabled a smoother transition to software-based systems.”

CAPA-Formula E - 2

Connect and Produce Anywhere: Formula E

Furthermore, Formula E’s remote operational model served as an exemplary case for other sports and broadcasts aiming to adopt similar approaches. By leveraging Formula E’s sophisticated yet flexible IP-based system, the project aimed to develop scalable solutions that could democratise high-quality broadcasting for various events.

Challenges and solutions

Interoperability: One of the main challenges faced during the project was ensuring interoperability between different systems and components. Traditional broadcast equipment often relies on proprietary standards, making integration with new technologies difficult. The project addressed this issue by focusing on open APIs and collaborative development, allowing for smoother integration and more flexible workflows.

Deployment models: The project also highlighted the need to refine deployment models for software-based systems. As Wagdin pointed out: “Different deployment models come with their own set of challenges, and finding the right mix of virtual machines and containers is crucial. The ongoing experimentation and development in this area will be key to optimising performance and efficiency in future projects.”

Scalability: This is another critical aspect of live broadcast production. The project demonstrated how cloud and edge computing, along with containerised applications, can provide the scalability needed to handle varying demands. This approach allows broadcasters to scale their operations up or down as needed, ensuring optimal performance and cost-effectiveness.

Real-time processing: Real-time processing is essential for live sports broadcasting, where even slight delays can impact the viewing experience. The project addressed this challenge by leveraging high-throughput processing locally at the edge, reducing the need for high-bandwidth connections and minimising latency. This approach ensures that live feeds are processed and delivered with minimal delay, enhancing the overall broadcast quality.

Innovative solutions and technologies

Software-based production: The shift towards software-based production is a cornerstone of this project. Traditional hardware-based methods are being replaced by more flexible and scalable software solutions. This transition allows for lower latency, higher flexibility, and the ability to produce content remotely, which is particularly beneficial for dynamic and geographically dispersed events like Formula E races.

Cloud and edge computing - The use of cloud and edge computing is a significant advancement in live broadcast production. By deploying software within a distributed cloud on the edge, the project was able to handle various input feeds and formats efficiently. This approach not only reduces latency but also provides the scalability needed to manage the high demands of live sports broadcasting.

Containerised applications: The use of containerised applications, as discussed by Wagdin, is another innovative solution implemented in this project. “Containers provide a more efficient way to run software, allowing for easy deployment and scalability,” he said. “This approach is particularly useful in live broadcast production, where different components can be swapped or scaled as needed without significant disruptions.”

Learning and adaptation

One of the key benefits of this collaborative effort was the ability to learn and adapt in real time. As Wagdin noted: “The project provided valuable insights into deploying new technologies in a live environment, identifying challenges, and developing solutions. This learning process is crucial for the ongoing development and refinement of software-based production systems.”

Future of live broadcast production: The Formula E project represents a significant step forward in the evolution of live broadcast production. This successful implementation of software-based systems, cloud and edge computing, IP-based infrastructure, and containerised applications showcases the potential of these technologies to revolutionise the industry.