No comments

ABSTRACT

This paper provides insights of an advanced architecture for authoring and consumption of second screen content aiming to develop innovative functionalities for enjoying multimedia content by connecting it to other related content, augmenting it with virtual information of interest, and allowing navigation of the 3D reconstruction of the scene.

Innovative technology and the underlying architecture for efficient production of second screen applications is described, which includes novel front-end authoring tools as well as back-end enabling technologies such as visual search, media structure analysis and 3D A/V reconstruction to support new editorial workflows.

A strong commitment on standardisation by the project’s partners ensures the future-proof utility of the results.

INTRODUCTION

Current second screen applications let user enjoying broadcast programmes access related information on other – typically internet-connected – devices. Although several services are already available in the consumer domain the problem of ensuring sustainability of such applications in the media production domain and for generic programme formats still remains vastly unsolved.

The most relevant challenges to be tackled in this area are: 1) making production costs related to enriched content affordable for broadcasters; 2) second screen presentation must be ergonomic and functional, and links should include rich and exciting content (e.g., 3D models) to ensure a differentiating factor for the quality of user experience; 3) authors and editors must be supported by efficient authoring tools, which must also be easy to integrate in existing workflows.

To address these challenges BRIDGET, an FP7 European Commission’s collaborative project, has developed the namesake notion of a bridget: a link from the programme being watched to (combinations of) external interactive media elements such as web pages, images, audio clips, different types of video (2D, multi-view, 3D free viewpoint) and synthetic 3D models.

Bridget are links which represent some inherent semantic relationship between content items. As such, they can be products of an editorial decision, taken by someone inspecting (manually or automatically) content items, or they can be objects of a workflow which involving different roles taking care of finding, organizing and crafting the data that constitute them.

In order to facilitate the bridget creation, the project develops: a) advanced visual search tools that help locate semantically related images and video segments in archives; b) advanced media structure analysis and content annotation tools that help identify candidate content segments to be enriched and offer quick and efficient navigation through content; c) advanced A/V 3D reconstruction tools to provide new content types to enrich the main content; d) efficient authoring tools integrating the above technologies in the production workflow.

RELATED WORK

Second screen applications for TV shows have a long history, and there are many existing applications which are relevant in the context of this work.

Amazon’s X-Ray service, Google Play Movies & TV are examples of general purpose applications that provide additional content related to a reference main content e.g. in-depth information about actors playing in a movie which does not necessarily need to be watched at the same time.

Another class of applications are the live sync applications, i.e. applications that provide additional content on a time-synchronous way by exploiting some audio-based or event-based synchronisation mean.

These normally provide a more detailed and interactive experience than general purpose apps, e.g. for episodes of fiction series. Examples are the AMC’s Story Sync and Team Coco.

An application based on a similar model to BRIDGET’s is the SyFy Sync application, which uses audio-based synchronisation to allow second screen experience independently of the actual broadcast time.

Despite the liveliness of the developments in the reference area of BRIDGET, there is still a lack of systems and approaches capable of making second screen application sustainable in production, which is exactly where the novelty of BRIDGET lies.

This novelty resides in the capability released for broadcasters and media companies to efficiently and rapidly produce such applications for a wide range and variety of programmes, including archived ones, to retain editorial control over the additional content, and to exploit the vast amount of relevant content (e.g., outtake) which remains available downstream of a production for a further innovative usage.

This is ensured by the employment of a flexible architecture integrating a set of cutting edge backend technologies and an authoring tool specifically designed for the purpose. These features are explained in the next two sections.