AI and ML: driving content automation

In the Content Everywhere industry, the deployment of AI and ML varies considerably depending on the company and its product or business model. The use of the techniques to handle content is on the rise, and as Bart Lozia, CEO of Better Software Group (BSG), says, “we’ve barely scratched the surface of AI’s capabilities and already there is an enormous change in how we deliver content to viewers. Solutions become more sophisticated, which means that we can utilise them for our and the users’ benefit”.

“With the advent of ChatGPT,” says Meghna Krishna, chief revenue officer at Magnifi, “generative AI has become a common topic these days, but most people haven’t considered its full potential in areas like video production, where numerous brands are adopting AI-based solutions to enhance their workflows, from brainstorming to editing”.

New tools of the trade

So who is doing what with AI and ML in the Content Everywhere industry, and how will the use of this technology continue to evolve?

For Ai-Media, as its name suggests, AI plays a key role in ensuring the accuracy of captioning, transcription, translation and audio description services, for example. Bill McLaughlin, chief product officer at Ai-Media, notes that the growth of automatic transcription and translation technologies “has transformed our work more thoroughly than almost any other specialty in media production”.

“Compared to a decade ago when little professionally distributed content was being subtitled automatically, the amount of accessibility subtitling occurring has grown exponentially and the production price per hour is down 90% in many cases,” he says. “Where expert manual editing is still performed, it still begins with AI-generated text as a guide, and there is a faster production turnaround time and lower cost than fully manual efforts. We are also still making a lot of advances in areas beyond just language models, like AI/ML technologies for positioning and styling subtitles on the screen or producing generative text descriptions of scenes and sound effects.”

At the same time, he cites handling of quality control as the biggest challenge the company faces in increasing adoption of AI and no-touch workflows with customers.

“Human content editors or translators aren’t perfect either, but there is more risk for automated systems to miss the mark by a wide margin. It can be easy to spend nearly as much on human oversight or review of AI as it would be to have the human editor do the entire job. So Ai-Media has had to become much more experienced in fields like statistical quality measurement to guide our approach and produce an outcome for our customers that’s optimised for both cost and quality,” McLaughlin says.

Meanwhile Rick Young, SVP and head of global products at LTN, says his company experiments with AI “to drive the various production solutions we deliver across a broad variety of sports specifically for LTN Arc, our high volume, scalable production versioning service.”

In many cases, Young says, “we gain significant efficiencies by enabling a single operator to actively create two, four or more versions of live games.”

Looking ahead, LTN expects many events will be able to run in a near hands-free manner with operators assigned to take action only on an exception basis. This will involve on-screen identification and translation of graphics, live action events, in-game audio or dynamically identifying scene changes and other data sources.

“While there is much discussion about AI risks associated with the automated creation of misinformation and its potential to be used for ill purposes, we focus on the ways that AI and automation can drive content creation to serve broader audiences with tailored content that meets the needs of a specific region, demographic or platform with relevant live programming of all types,” Young adds.

Some companies such as Red Bee Media have been using AI for a number of years. According to Richard Kydd, chief technology officer, “Red Bee Media Access Services were already successfully using AI speech recognition to produce live captioning 20 years ago”.

Today, Red Bee Media says it uses automatic speech recognition (ASR) throughout its subtitling production workflows, both for fully automated subtitling and to improve workflow efficiency.

“I believe that accurate automatic speaker recognition is the latest frontier in fully automatic captioning. Our fully automatic captioning for live captioning is maturing rapidly because the speed of accurate delivery is so critical compared to offline captioning,” Kydd says.

He adds: “The latest developments in large language models hold out the hope of greater accuracy improvements as these models represent a deeper context. The LLM (such as ChatGPT) breakthroughs are also promising in various adjacent areas, such as higher-quality machine translation, quality assessments, editing for reading speeds, and script extraction. Furthermore, opportunities for greater automation in audio description seem to be advancing fast, and sign language avatar technology will likely progress more in the near future, although the use of avatars will only remain appropriate for certain scenarios.”

Venugopal Iyengar, COO, Digital at Planetcast International, says his company’s use of AI/ML “has moved incredibly quickly from lab experiment to deployment, so that AI is now top of mind when we’re working on a new project. A recent example is the Indian Premier League cricket (IPL), the world’s second most expensive sports rights, for which we are now using AI to automate the creation of instant highlights.”

Iyengar says AI is making a real difference to Planetcast’s Contido suite of content supply chain management solutions

“We have been using AI-backed tech checks and baseline work like segment/break recognition to increase throughputs. We also use AI in areas of compliance (like language recognition). The AI-led features that the Contido team is presently working on include the use of the Whisper speech recognition engine to auto-generate subtitles and to aid search. The team is also building a custom NER (named entity recognition) model using AI, to improve the accuracy of AI-generated metadata and asset tagging,” he explains.

In other examples, Krishna from Magnifi says her company’s AI video editor “is trained on extensive data and possesses the ability to identify crucial moments and footage in a sports match using meta-tags. By specifying tags like ‘goal’, ‘yellow card’ and ‘Cristiano Ronaldo’, the editor can automatically generate clips of all his goals and plays from a specific game in under 45 seconds.”

Other features from Magnifi include auto-flipping, which Krishna says allows brands to customise videos according to the dimensions of various social media channels.

Yang Cai, CEO and president at VisualOn, says his company launched a bandwidth-saving solution, called Optimizer, with an ML-based algorithm that “efficiently, automatically, and optimally configures the encoder to achieve the best results based on the input contents”.

“We are still in the early stage of applying AI and ML to video streaming, especially live video streaming,” Cai adds. “Classifying, recognising objects, and recognising faces are tasks that AI and ML should manage in the future, which can help solve problems, from protecting privacy to improving the quality of user experience.”

Where next?

So what is the future for AI/ML in Content Everywhere, and what impact will it have on workflows, jobs and more? Furthermore, how will companies address concerns about ethics and ensuring that AI algorithms are not biased, for example?

“We have found that if you think AI might be useful, it almost always is,” comments Iyengar. “And better still, you can find out very quickly if AI is up to the challenge. We’re lucky to have a development team of over 200 software engineers, who are all excited by the challenges and opportunities AI can deliver. So far, our experience tells us that you shouldn’t expect a human alone or a 100% AI solution to be perfect, but that a human-supervised AI usually provides the most reliable solution.”

Krishna says it’s now possible for smaller teams to produce videos as fast as traditional broadcasters, across many channels at the same time and with the same or higher levels of quality and creativity.

“However, this doesn’t imply a massive loss of jobs in the industry,” she says. “On the contrary, AI tools serve as valuable assistants to video production teams, freeing up their time to focus on the creative aspects rather than the technical and routine tasks that can be easily automated. AI also generates new job opportunities, demanding skills that are essential in an AI-driven environment.”

Frank Miller, chief technology officer at Varnish Software, observes that ML and AI have great potential for the content production and delivery ecosystem, but warns that a clear understanding of the capabilities and challenges with each of these tools is needed for the correct application.

“It is key that all of these opportunities have the caveat that we must understand the algorithms and methods in order to ensure that they produce results of value. For example, the heart of ChatGPT is deep learning in large language models, which is similar to a human brain with multi-layered neural networks that must be trained and curated. A small perturbation in training can result in biases that drive strange behaviour,” he says.

Like any technology, Miller adds, “selection of the right tools, awareness of the capabilities and limitations of the tools, and proper application is key to value. The value these tools bring in efficiency in encoding, automation of workflows, prediction for proactive decisions and real-time anomaly detection will be key to protecting the customer experience…as long as we have a clearly defined policy framework around each ML/AI model that we place in the workplace.”

Tzvi Gerstl, chief technology officer at Synamedia, says AI will continue to transform media production workflows, saving time and resources, with new features and more automation of tasks across content curation, video editing, transcription and subtitles, and metadata tagging.

“While AI is known for its ability to analyse large video data sets quickly, I believe its role in content creation will have the greatest impact on the industry, whether enhancing video quality, adding visual effects, and auto-generating content using synthetic media,” he says.

Gerstl concedes that although AI raises concerns for some, “I believe we are at an inflection point thanks to the new opportunities AI is bringing to the TV and video space”.

Red Bee Media’s Kydd says he has no concerns related to the technical aspects and “thoughtful application” of ML and AI.

“However, I am wary that any technology can be overtaken by over-enthusiastic expectations and unrealistic hype becoming fashionable rather than functional,” he says. “I do see a rush to AI at the moment, which reminds me of past bubbles where it can become hard to sort the genuinely ground breaking and valuable from the novel and entertaining that has no actual application.”

According to Kydd, “in my experience, we should focus on the value of the result and how that value can become material for our customers instead of relying on labelling something as AI/ML/LLM and hoping to gain success”.