No comments

ABSTRACT

This paper describes an experimental system that can create good quality subtitle files for video clips derived from broadcast content. The system is designed to run automatically without the need for human verification.

The approach utilises existing metadata sources, an off-air broadcast archive and an archive of original subtitle files along with audio fingerprinting and speech-to-text technology to identify the source programme. It then locates the position of the video clip, verifies the match between the video clip and the subtitles and create a new subtitle file.

This paper also reports on the results of the work using a large corpus of over 7,000 video clips and further, smaller sets of clips from different television genres, and explores where improvements might be made. It also looks at the limitations of the current approach discussing alternative methods for providing subtitles for video clips.

INTRODUCTION

In the UK the BBC provides subtitles for 100% of its television programmes on all of its main television channels. Subtitles are also provided on all these same programmes when on the BBC’s video on demand service, iPlayer. However, the situation is very different for video clips that the BBC provides on its websites.

At present, a small proportion of clips are subtitled manually, but the majority do not have subtitles. As many websites become more reliant on video content the need to provide subtitles for these clips is increasing.

Our understanding of the use of subtitles is also improving. Audience surveys have indicated that around 10% of our adult TV audience use subtitles daily and around 6% use them most of the time (1) but we have no data on television viewing with subtitles by children. However, with iPlayer the BBC can begin to record accurate data on the use of subtitles on a per programme basis. Currently, verified data is only available for iOS devices, but early indications reveal high levels of subtitle use.

A sample from a week in March 2016 indicated that overall subtitle use is around 18%, while usage levels on tablets is higher at over 20%. However, the most interesting figures are for the BBC’s children’s services where subtitle usage is around 30% and content classified as ‘Learning’ where use is around 35%.

Further work is on going to understand these patterns of subtitle use for on-demand content and to look across other platforms. However, it is clear that for our on-line audience, subtitles are an important part of their viewing experience and this helps to motivate investigatory work to try and extend subtitle availability on-line.

BACKGROUND

The BBC has many thousands of video clips on its websites and the number is growing every day. Until now, finding subtitles for video clips has been a manual process, either by retrieving subtitles from the original programme or by authoring new ones. However, most video clips provided on the BBC’s website are either derived from, or closely related to a broadcast programme.

Where a clip is taken directly from a broadcast television programme, subtitles should exist to cover the duration of the clip. If it is possible to locate the broadcast programme and the associated subtitle file and then to identify the timing of the clip, then a new subtitle file can be created for the clip.

At NAB last year we described our initial work, focusing on providing subtitles for clips on the BBC News website (2). Because most news programming is broadcast live, the work included a user interface to enable manual correction and retiming of the subtitles.

Towards the end of last year we were asked to look at video clips from general programmes, where the subtitles are mostly pre-prepared and so of high quality. The request was to provide a subtitle search that could be triggered by a video clip being uploaded though the BBC’s iBroadcast publication interface and to provide subtitles without the need for an additional user interface. Also required was a batch processing version which could find subtitles for clips already published.

In both cases the search needed to be automatic and the quality of the subtitles had to be verified without the need for human oversight.