Artificial role of AI in sports production

Such is the buzz surrounding anything to do with Artificial Intelligence or Machine Learning at the moment that it’s no surprise to find it firmly slotted into the folder marked ‘zeitgeist’ when it comes to live sports production. Perhaps even more so than remote or ‘at-home’ production, it is being billed as delivering the sort of savings that can help square the circle of increasing viewer expectation coupled with rising rights costs and over-pressured budgets. The problem is, however, that hype seems to be running a good deal of distance ahead of delivery now. AI is being used in live sports production, but the use cases are somewhat constrained now and, while there are any number or future areas that it could influence and develop, 2019 is not necessarily the year that they will be appearing.

One high-profile broadcaster who asked not to be named offered the following.

“There are a wide range of potential uses, but we’re not currently making use of AI in our productions at the moment. We’re keeping a close eye on developments within the field and are constantly undertaking trials of AI and other technologies to see how our roadmap might evolve to include it in the future. It’s not there yet, however, though it shows promise in areas like automated clip creation or camera tracking.

• Read more: AI: Moving into the mainstream

“It’s worth pointing out too that there is a big difference between the clever algorithms we see in operation in things like player tracking and score prediction, however, and what could be genuinely classed as machine learning…”

Such obfuscation about what AI is and isn’t is, of course, not just applicable to sport. Artificial intelligence and machine learning have become interchangeable terms, in that what most people think of as AI should be called ML and a lot of what is being marketed as AI is simply a well-written algorithm given a hyperbolic marketing spin.

Dr Paul Hawkins, who famously developed the Hawk-Eye system, did his original PhD in artificial intelligence and offers a succinct ‘No, not really’ when asked if he can think of any use cases involving AI in live sports production. As CEO of Hawk-Eye Innovations though, now a part of Sony, he does see areas where its ability to extract meaningful conclusions out of large data sets intersects with the business of sport and player tracking.

“AI is probably more useful in high value areas such as evaluating the vale of players and assets” Dr Paul Hawkins, Hawk-Eye Innovations

“It’s probably more useful in the high value areas such as clubs and franchises evaluating the value of their players, assessing the value of their own assets and deciding which ones to buy,” he says. “There’s a lot of work going into that, but needless to say that those companies are playing those cards quite close to their chest.”

His contention is that although sports is a huge industry there is a comparatively small amount of money in the sports production sub-sector. As a result, investment in future AI and ML will be funneled more into the truly major industries where it can have both a profound impact and generate profound profits, such as medical research and autonomous vehicles. 

Data collection
Nevertheless, when information sifting power is offered as a service, as it increasingly is by companies such as AWS and others, it provides a route for broadcast industry vendors to tap into the R&D efforts that are targeting those other industries. As Carlo de Marchis, chief product & marketing officer at Deltatre puts it: “The data side of the chain has been the first candidate for some AI/ML treatment and we are now moving into applying it to the video side of it, to either use video as a data source or direct video manipulation.”

“Any AI/ML system you put in place has to be taught by humans – that is the key element of a successful implementation” Carlo de Marchis, Deltatre 

Certainly, in terms of data there are several projects underway and several services being offered. Much of these falls under the heading of automation, with the coupling of AI-powered services such as facial recognition to metadata logging and clips generation exemplifying the sort of service that is on offer. The idea is that a system is trained to recognise the faces of players in, for example, a football match and tags the footage. It is also trained to log certain events, though some degree of human intervention can be called for here; identifying a goal or a corner is much easier, for example, than a dive in the penalty area.

• Read more: How vendors are adopting AI

“Any AI/ML system you put in place has to be taught by humans, that process is the key element of a successful implementation,” comments de Marchis.

This is one of the areas at the moment where AI’s promise is justified. At Wimbledon 2018, an IBM Watson powered system clipped over 75,000 individual points from nigh on 1000 hours of footage for the AELTC, compressing highlights turnaround time from 45 minutes to 5 as it did. At IBC2018, Timeline TV gave a presentation about a MAM system it had set up for ITV Sport based on Tedial technology, which features 200,000 assets and integrates with its EVS system. An AI layer has been developed by Tedial that will sit on top of the MAM and automate some of its processes, and that should be going live in the near future with some impressive projected impacts. Tedial’s Jérôme Wauthoz, for example, estimates that a 15-20-minute job of creating five different highlights packages can be done in a couple of seconds with five mouse clicks.

NASCAR is another one of the sports organisations that has taken advantage of this sort of service, using Amazon Rekognition to tag its 18-petabyte video archive with metadata referring to drivers, cars, lap times and more. And there is a fair bit more than can be done if sports organisations desire it, Amazon Rekognition being able to undertake sentiment analysis based on facial expressions, for example, and the IBM system at Wimbledon also analysing crowd noise to rank reactions to the on-court play.

These systems tend to be considered okay for outputting to channels such as social media, where volume of content and cost-effective comprehensiveness of coverage are the main considerations. But there is less confidence as you move up the ladder to Tier 1 broadcasters and primetime highlights packages.

Automated video
The challenge in the future will be to take that capability into a live production environment and marry it up to, for example, super slo-mo in a way that is both more efficient than current director/skilled operator communications and more cost-effective. 

For now, the next front opened by the proponents of AI in live production is on the video side of the equation. Several companies have developed AI-led systems that are designed to automate video capture of sporting events, typically by flying virtual cameras around larger pictures captured in, say, either native 8K or a stitched together panorama. Others meanwhile are looking at training self-steering cameras to follow the on-field action, which hold out the prospect of being installed discreetly with no seat-kill and dovetail with the move towards remote production in the industry.

“It can simplify production; automation and enable new workflows,” says de Marchis. “It can enhance production, with data interpretation, prediction, and pattern recognition. And it can personalize production; learn users’ behaviours and predict users’ needs.”

Where AI in the real world of live sports production is in 2019 is exemplified by the Timeline TV system or the Wimbledon project, which will hopefully show further evolution during this year’s Championships. But more work is being done in the field all the time, more broadcasters are conducting tests, and, as costs decrease, in the next few years the substance will very probably catch up with the hype.