AI in streaming: investment set to soar as new applications solidify

If you still harbour any doubts about the revelatory potential of AI in broadcast and streaming, then a reading of the 2024 Broadcast Transformation Report from Haivision – which is itself a leading provider of low-latency video streaming and video encoding solutions – will assuage them. Drawing on interviews with 814 participants from across the full range of broadcast and media services and technologies, the document is laden with statistics that point to a steep and dramatic adoption path for AI-based applications.

In the current context, AI is a lower priority for broadcasters than enabling remote production or transitioning to IP; but that won’t be the case for long. Indeed, 60% of participants predicted that AI will be the tech with the biggest impact on the industry in the next five years, with 49% planning to use – or already using – AI in their workflows. Given that this eclipses the anticipated impact for technologies as seismic as 5G and 4K/UHD, it’s evident how ‘big’ AI is expected to be in professional media.

Moreover, advantages with relevance to streaming applications also figure prominently in the survey of anticipated benefits resulting from AI implementation. These include efficiency and productivity gains through automation (cited by 41% of respondents), picture and bitrate optimisation (31%), and the provision of personalised viewing experiences (13%).

It’s no surprise, then, that a round of interviews with streaming technology specialists revealed both insight and excitement around a host of AI-based applications that look set to herald significant improvements and efficiencies.

Personalise and repurpose

Joe Claypool is VP customer engineering at MediaKind, an encoding, decoding and stream processing specialist. He describes the pace of AI’s impact on media to date as “astonishing”, noting that it is already “transforming processes like audio generation, closed captioning and automated highlight creation”, as well as boosting operational efficiency through the use of NLP (natural language processing) to obtain real-time status updates about complex systems and navigate user interfaces intuitively.

MediaKind’s approach has been to “steadily integrate” AI and machine learning (ML) across its operations and services, with early advancements including the deployment of ML for up-conversion of various formats and resolutions. More recently, says Claypool, “MediaKind has introduced AI-based sponsor logo detection, a valuable tool for accurately tracking and quantifying on-screen exposure, which has streamlined billing and maximised sponsorship value for media customers, especially in the sports space. These AI-driven analytics mean ad strategies are optimised with precise viewer data. Additionally, AI-driven optimisation in encoder bitrate management has further strengthened the company’s reputation for delivering best-in-class video quality at reduced data costs.”

At IBC2024, MediaKind introduced an AI pipeline service that allows customers to harness and automate workflows using leading AI models. The company also announced a strategic partnership with ThinkAnalytics, allowing customers to benefit from enhanced personalised content discovery, GenAI features and targeted advertising offerings.

Looking ahead, Claypool identifies three key areas – personalisation, automation and content repurposing – as “pivotal” AI-driven opportunities in streaming services. On the personalisation front, he points to a recent project with the NBA to help the league’s European expansion, ensuring that local fans see more of favourite players such as Luka Dončić in Slovenia and Nikola Jokić in Serbia.

In terms of automation, MediaKind recently announced the introduction of MK.IO Flow Control and AI-Assisted Automation, an API that streamlines content workflows and automatically handles routing, configuration and processing. Then, for content repurposing, Claypool says the company is using AI to “breathe new life into legacy content, upscaling SD to HD, and opening up fresh revenue streams by bringing older media experiences up to modern, high-quality standards.”

It is also anticipated that sports – in keeping with its fundamental importance to the overall streaming landscape – will remain a specific target of attention. “We’ve had a lot of success with sports organisations and we continue to look at how we can reshape and streamline how sports content is created, delivered and consumed,” remarks Claypool. “MediaKind adaptive workflow tools allow sports organisations to bypass the investment of time and money in building or custom integrating this type of solution. [As a result] they can focus on outcomes and identify the right questions to ask and the right AI tools for their goals.”

Streaming efficiency

One of the companies underlining the Haivision report’s assertion that ‘picture and bitrate optimisation’ is set to be a critical benefit of AI implementation is Media Excel, which chose IBC 2024 to showcase its new AI-based DIVA (Dynamic Intelligent Video Adaptive) encoding technology. DIVA performs real-time analysis of incoming video streams, dynamically adjusting encoding settings and optimising codec performance. According to the company, the technology achieved ‘unparalleled’ video quality while delivering a ‘20% or better’ reduction in bandwidth usage for streamed HEVC-encoded video content.

“Our approach to leveraging AI through cutting-edge deep learning models is very pragmatic,” says Narayanan Rajan, CEO of Media Excel. “We aim to deliver the best visual quality while significantly reducing the cost of streaming video at scale. This becomes even more critical as more content owners are streaming directly to consumers, and cloud egress and CDN [content delivery network] distribution costs start to dominate TCO [total cost of ownership].”

In a white paper accompanying the new technology, Media Excel drills down into its use of AI-based deep learning, which begins with the analysis of various content in order to understand the characteristics of the videos and model them. Through this process, the company aimed to create a reference (labelled) dataset that closely resembles exemplary solutions, making it possible to find optimal encoding parameter values for videos with a host of different attributes. The combination of this dataset and deep learning techniques yielded a model capable of finding the best possible encoding settings based on the actual characteristics of the content.

“Our whole question here is, ‘how do you maximise video quality using AI around a target bitrate?’ So it’s not pure VBR [variable bitrate], but you are kind of ensuring that the bits go into the right place while enabling everything downstream to work really well, too. You are being smart about allocating the bits. But then there is also [this whole dimension] of being quality-focused around a particular target bitrate, to which end we’ve built models for that based on the training of a number of different pieces of content.”

At present, Rajan suggests that the benefits of AI-based streaming optimisation in general are most readily accrued when an organisation is operating at a not insignificant scale. “But we also think that [with increased efficiency] things will look a bit different 12 to 18 months from now and the landscape of who can benefit from it will shift to the left on the graph so it includes large-scale and small-scale.”

He is also excited about the possibility of exploring improvements in areas such as adaptive de-noising, image restoration, HDR tone map conversion, and a host of tasks “on the pre-processing side. Again, it will be driven by an emphasis on being very pragmatic about the value we’re trying to create for our customers so they can drive value in their own businesses.”

In fact, this last comment could arguably be applied to the entire current narrative of AI in streaming. Across the board, automation is bringing discreet but distinct improvements in efficiency, personalisation and localisation, but without adding to companies’ existing workloads. Of course, it’s early days, and it’s very likely that with the passage of time we will see more radical use cases and applications emerge, including around content repurposing. But for now, there are already plenty of innovative solutions for end-users to explore and be inspired by.