Amazon reveals how Prime Video is incorporating genAI

The company is using Amazon Bedrock, a service from Amazon Web Services (AWS) for building generative AI applications, to create personalised content recommendations, spoiler-free recaps, AI-powered sports highlights, and to boost accessibility. The tool has also been used for enhanced asset management to streamline operations.

Amazon shared five specific ways Prime Video is now using generative AI from AWS:

1. Personalised content recommendations

Amazon Bedrock is helping power personalised content recommendations directly within the “Movies” and “TV Shows” landing pages of Prime Video. Viewers will see “movies we think you’ll like” and “TV shows we think you’ll like” collections on each page that are curated based on their interests and viewing history.

2. X-Ray Recaps

The X-Ray Recaps feature on Prime Video creates brief summaries of full seasons of TV shows, single episodes, and even pieces of episodes. They are created by a combination of Amazon Bedrock managed foundation models and custom AI models, trained using Amazon SageMaker. Amazon Bedrock Guardrails are applied to verify that summaries remain spoiler-free.

3. Prime Insights

The most recent seasons of NFL Thursday Night Football (TNF) and NASCAR events on Prime Video saw the debut of several new Prime Insights, which highlight key sports performance angles and storylines. The AI-powered tools were built through a collaboration of Prime Sports producers, engineers, on-air analysts, AI and Computer Vision experts with an AWS team using Amazon Bedrock. Prime Insights illustrate hidden aspects of the game and forecast key moments before they happen. For example, the “Defensive Vulnerability” feature on TNF is powered by a proprietary machine learning model. It employs thousands of data points to analyse defensive and offensive formations and highlight where the offence will—or should—attempt to attack.

4. Accessibility

Dialogue Boost analyses the original audio in a movie or series and uses AI to intelligently identify points where dialogue may be hard to hear above background music and effects. Then, the feature isolates speech patterns and enhances audio to make the dialogue clearer. For Dialogue Boost, Prime Video uses a variety of AWS services including AWS Batch with Amazon Elastic Container Registry (Amazon ECR), Amazon Elastic Container Service (Amazon ECS), AWS Fargate, Amazon Simple Storage Service (Amazon S3), Amazon DynamoDB, and Amazon CloudWatch, among others. Initially launched in English, Dialogue Boost now supports six additional languages including French, Italian, German, Spanish, Portuguese and Hindi.

5. Enhanced video understanding

Prime Video is using the Media2Cloud on AWS guidance, which delivers media analysis at frame, shot, scene, and audio levels. The guidance helps to enrich assets’ metadata, such as celebrity, text on screen, moderation, mood detection, and transcription. Media2Cloud enables faster, more accurate, video understanding for enhanced content management, search capabilities, and audience engagement. It is powered by Amazon Bedrock, Amazon Nova, Amazon Rekognition, and Amazon Transcribe. Metadata from media assets is automatically fed to AWS Partner, Iconik, media asset management (MAM), helping Prime Video to enrich hundreds of thousands of assets and improve discoverability in its marketing archive.