The Set Top Box revolution

The transformation include broadband access, which is now widespread in developed countries, often exceeding the capacity of broadcast networks; new delivery technologies, such as object-based video, allow for innovative forms of narrative content consumption; and use cases not normally associated with the industry players, such as gaming or smart home control, are converging with their core business.

To meet this opportunity (and raise to the new challenges), future set-top boxes (STB) will require increasing computer power to handle the wide variety of use cases and experiences that are outlined in this article. This means that more compute performance, greater artificial intelligence (AI) capabilities, and enhanced security protections are all now a must in the next generation of STBs, with this also feeding into the next generation of digital televisions (DTVs) too.

Improving user experience

Firstly, it is worth exploring the growing availability of content and OTT service providers. The list of service providers is no longer limited to the global top three or four. An increasingly long list of specialized or regional vendors are also appealing to a broadening user base. These services, and the platforms carrying them, compete for the user’s attention in order to drive consumption, which is key for monetization through subscription or advertising.

Competition for attention is driving better user experiences based on compelling, high resolution user interfaces and content recommendation engines. 4K UIs are possible already, and web technologies can be used to create scalable, smooth interfaces. These and other technologies all require enhanced compute capabilities on STBs.

Object-based content

Linear content is still by far the dominant format in today’s media, but the availability of IP as primary delivery channel and advances in content production have brought significant advances in object-based media.

This means that, to a certain extent, each user may receive an individual version or a program, from conventional choices, such as subtitles or audio tracks, to innovation around branched narratives (e.g. ‘choose your own adventure’ storylines), camera angles in sports and personalized advertising. The combination of various objects into a final stream can be done in the cloud, in the device, or via a combination of the two. Object-based media standards are still emerging and in the coming years we are likely to see new creative uses of this technology.

AI super resolution

AI is rapidly becoming ubiquitous in media services, and not just during production (for example, automated content classification and advanced vision features in sport events), but also during content consumption. In STBs, it is being used for picture quality enhancement, with AI super resolution emerging as an important technology in the picture quality process on the devices. This uses a combination of AI and traditional video processing techniques to achieve a high-quality picture when upconverting video sources from lower resolution to 4K.

For the higher-tier STBs, this could go all the way up to 8K. While 4K STB devices currently dominate the current market, it is only a matter of time before more 8K STBs enter and are shipped more widely globally. Due to the increased resolution and screen size, traditional scaling methods on STB processors may not be sufficient to achieve the desired quality. This is one area where Arm is seeing a lot of innovation, as STB OEMs reassess the IP required to deliver high-quality pictures using newer AI methods.

Smart camera support

Working from home during the COVID-19 pandemic has changed consumer behaviours, normalising the use at home of videoconferencing applications, which used to belong in the office. In the home, these applications fit quite naturally in the living room.

These experiences will be further enriched with the use of AI technologies and addition of new smart cameras. Smart cameras use immersive visuals that recognise hand and body gestures, enhance or blurs out the background, provide intelligent focus with face tracking and even enable augmented reality (AR) filters.

Other screens – such as those for smartphones, tablets and laptops – have had built-in cameras for a long time, but users will need reassurances that their camera-enabled TVs have appropriate levels of security and privacy-preserving technology. Arm is actively working in this area, as we explore later in the paragraph on greater security demands.

Gaming and fitness applications

Alongside video call applications, gaming has seen a significant increase in popularity during the past months (as anyone who has tried to buy a latest generation console will attest).

Besides conventional gaming consoles, various gaming services are being delivered through STBs, either natively in the device or via cloud gaming providers like Google Stadia and Tencent Games for the China market. Cloud gaming will require STBs to enable video and command streaming and low latency features. Meanwhile, native gaming platforms need STB devices with much improved Central Processing Units (CPUs) and Graphics Processing Units (GPUs) for better gaming experiences, but have the benefit of not requiring a fast, reliable broadband connection. Recent advances in mobile graphics also make high performance gaming more accessible than ever. 

Another trend we have observed during the COVID-19 pandemic is the increasing number of people exercising from home. A new wave of smart fitness applications are riding this trend, benefitting from the availability of smart cameras on STB devices. To make these applications work, AI algorithms for body position tracking will be required, alongside an always-on voice subsystem for voice command and digital assistant features.

Always on Voice UI

To seamlessly enjoy these new experiences on STB devices, always-on voice control will be a particularly important feature. Voice UI, widely available already on today’s STBs, enables more complex applications and experiences, such as the fitness applications described above, as well as basic voice commands for an improved user experience (UX). The use of dedicated Neural Processing Units (NPUs) in STB devices could be important, advancing AI functionality to minimize power consumption in standby mode and providing a responsive experience.

Greater security demands

With all these new experiences and features coming to STB devices, there is likely to be a growing emergence of new threats and attacks. Security protections for all the content now arriving via content and OTT service providers will be especially important, with content becoming a huge differentiator in the market for streaming services.

Moreover, with video being operated on by more processing engines in the chipset, the big challenge is ensuring that all the content is kept secure across the entire platform. This means protecting the content, operating system, applications and assets from attacks. In addition, due to new applications on STBs, there is now far more user data, with personal information and assets for social media accounts, payment and video calling all stored on devices. There needs to be more advanced security requirements to protect user data, with STB manufacturers looking at more sophisticated security protections across devices. This is likely to be similar to how we protect user data on today’s smartphones.

An evolving market

It is an exciting time for STB devices, with far more experiences and features coming to these devices. However, this will require greater processor power, advanced AI capabilities and enhanced security protections. Whatever the tier of device – from premium to entry-level – there will need to be more compute and security capabilities across the CPU, GPU and NPU in the device chipset. Arm is providing the IP and solutions needed to enable these advanced capabilities and new use cases across STB devices.

The STB revolution is leading to a new wave of innovation entering the market. Everyone in the value chain – but especially the operators and OEMs – needs to be prepared to take advantage of the amazing opportunities on STBs that will fundamentally transform the user’s home entertainment experience.

• Learn more about Arm’s involvement in the Smart Home market here.

By Pablo Fraile, Director, Home Segment, Arm