Unifying AI applications for broadcasters

Global broadcaster applications of artificial intelligence (AI) offer a wide range of use cases from achieving efficiency in editorial, automating workflows and tagging metadata.

Speaking to IBC365, researchers from Al Jazeera, Globo TV and iSize Technologies spoke of the diverse applications of AI currently being employed to monitor regulatory compliance, the application of AI to enhance media workflows, aid editors and producers in image selection, video encoding and delivery over IP as well as overlaying metadata for quality assurance in the ever converging ecosystem.

Debunking the assumption that AI will take away human job functions, Al Jazeera Media Network head of media and emerging platforms Grant Totten explains that AI is “massively important” for organisational adoption to save costs with an advantage to job shuffle responsibilities and enhance the editorial and creative storytelling.

He says: “AI changes our regulation control check and enables us to spend more time telling the story better and giving greater impact to our audience on the news.”

Creating bespoke neural-network based encoders and formats as well as developing neural networks to enhance current video encoding standards is a focus for iSize Technology technical director Yiannis Andreopoulos.

• Read more: New adventures in AR

The challenge for Andreopoulos is finding how to enable deep learning for advanced video encoding without breaking standards for encoding and delivery formats.

He explains: “To achieve our results, we exploit the fact that content can be pre-processed and compacted before being compressed with a standard encoder and focus our AI-based pre-processing and compaction on optimising perceptual quality for the smallest number of encoded bits.

“This deviates from the current practice of focusing purely on signal-to-noise ratio, which has poor correlation to human visual perception.”

“The term AI is a loaded word, it is happening today with the likes of Microsoft, Amazon and Google creating toolsets and enabling shifting skillsets across staff.” Grant Totten

Put simply: “We present novel techniques and systems to significantly enhance video encoding and delivery over IP networks based on deep neural networks,” Andreopoulos says.

Its AI-based pre-processing and compaction provides a smaller number of pixels to be encoded by the standards-based encoder and will be demonstrated in the IBC2019 Future Zone, located next to hall 8 at the RAI Auditorium.

Benefits for broadcasters include advanced video, AR, VR technologies to be delivered at scale and increased volumes and resolutions without clogging the network infrastructure or overtake the energy consumption of the encoding and decoding devices.

He adds: “Increased intelligence will be in the way devices understand human perception and human preferences, but also in continuously being refined by data-driven learning rather than hand-crafting algorithms.”

Infiltrating artificial intelligence
Totten expects that AI will soon become “common place, just like cloud and electricity,” but it is early days, he acknowledges: “The term AI is a loaded word, it is happening today with the likes of Microsoft, Amazon and Google creating toolsets and enabling shifting skillsets across staff.”

Al Jazeera has taken advantage of the benefits of AI, albeit it “late to the game”. Totten adds: “We had a data science team dealing with marketing to our audience and the analytics, but we started to realise a lot of potential within our extensive media archive.”

Working to a 24 hour news cycle its editors and producers have limited time to share impactful stories with its audience, however since using AI to filter and find archive content the possibility to enrich its metadata across the content assets has enabled “effective business practices and the ability to move fast against the competition.”

• Read more: Winning audience engagement with artificial intelligence 

Pointing to a couple of use cases, he adds: “The presenters focussing on the topics and IT working together to understand how our news is showing both side of the story.

“Detecting bias and regulatory compliance with Ofcom is a central focus, where regulators stipulate propaganda, we need to be central to show both sides of the story, which is why AI tools are so effective.”

Standards that must be identified include ensuring equal coverage is provided to both sides of a story, that groups in a news story are referred to according to internationally acceptable norms, that claims in a story are fact-checked, that visual maps of geographical locations are used effectively to highlight the country or region being discussed.

For example, Ofcom’s regulation of certain types of imagery and language have not been used or something as simple as identifying mistakes in names displayed on our visual graphics are accurate to the person speaking in an interview.

Totten explains: “Computer vision and natural language processing (NLP) techniques such as speech-to-text, facial recognition, OCR, named entity recognition that help us convert our media assets into vast stores of data to be mined in the form of powerful search engine and analytics engine.”

TV Globo research Edmundo Hoyle explains the benefits of AI which help to regulate moderation of content before the image becomes a thumbnail.

He says: “We use AI to avoid publishing images which contain nudity, weapons, or violence, and we also use it to limit the degree of violence in thumbnails.

“The AI algorithms are always improving their accuracy and detecting new kinds of behaviours. I think that in a few years AI algorithms will be able to detect almost all of our feelings.”

The application Hoyle will present at IBC2019 is our first step in automating the process of selecting images that are used as thumbnails on Globoplay, which is Globo TV’s digital platform.

He adds: “The main challenge was to replicate the rules that editors use to select images that are used as thumbnails and to resolve this we used techniques from NLP, deep learning, and computer vision.”

Augmenting and applying AI
In five years’ time, Hoyle expects that the quantity of data generated from AI applications will be “more clever and more accurate,” as well as generating new data and AI- focussed developments.

He adds: “I think that AI is changing the whole industry every day with the arts and many other industries will benefit from the innovation of AI.

“We are already utilising common AI techniques to increase our productivity, reduce costs and create novel ways to generate content.

“We have only a few ways in which we use AI, but this number will dramatically increase in the future.”

Andreopoulos echoed this expecting AI to augment existing capabilities and standards rather than replace them.

He says: “We will see existing systems and devices like mobile phones and other consumer electronics becoming increasingly more intelligent based on software updates and increased use of cloud-based data analysis.”

Al Jazeera’s Totten points to the research and development of AI, predicting “more interesting applications realised in the next two quarters with media enrichment of archives.”