Virtual Reality and Dash

No comments

ABSTRACT

Virtual Reality (VR) has lately gained significant attention primarily driven by the recent market availability of consumer devices, such as mobile phone-based Head Mounted Displays (HMDs). Apart from classic gaming applications, the delivery of 360° video is considered as another major use and is expected to be ubiquitous in the near future.

However, the delivery and decoding of high-resolution 360° videos in desirable quality is a challenging task due to network limitations and constraints on available end device decoding and processing. In this paper, we focus on aspects of 360° video streaming and provide an overview and discussion of possible solutions as well as considerations for future VR video streaming applications.

This paper mainly focuses on the status of the first standardisation activities to support interoperable 360° video streaming. More specifically, MPEG’s ongoing work on Omnidirectional MediA Format (OMAF) is introduced – aiming at harmonisation of VR video platforms and applications.

The paper also discusses the integration in MPEG Dynamic Adaptive Streaming over HTTP (MPEG-DASH), which is considered a cornerstone of 360° video streaming services with OMAF content. In the context of the general OMAF service architecture, three distinct delivery approaches and considerations for content protection are discussed.

INTRODUCTION

In recent years, there has been a lot of activity around Virtual Reality (VR) as evidenced by large industry engagement. Many important players in the computer industry have demonstrated support for VR and introduced Head Mounted Displays (HMDs), such as Oculus Rift, HTC Vive, Samsung GearVR, Sony PlayStation VR, and Google Daydream.

Expecting that the increasing popularity of consumer VR HMDs will lead to an increased demand for VR content, various companies have also started to develop omnidirectional cameras to allow capturing of 360° video content. There are many low-cost consumer solutions, such as Ricoh Theta, Samsung Gear 360, and LG 360 Cam, as well as more expensive professional 360° cameras, such as Nokia OZO, GoPro Omni, and Fraunhofer OmniCam360, already available on the market.

At the same time, major multimedia streaming platforms, such as YouTube and Facebook, have already launched support for 360° video streaming for VR devices and there have even been several successful live game broadcastings of professional American sports leagues or live event streaming in VR. Motivated by the industry interest on 360° video delivery for VR, several industry forums and standardisation bodies have started work.

In February 2016, MPEG launched an activity on Omnidirectional MediA Format (OMAF) that aims at standardising the storage and delivery format for 360° audio-visual content by the end of 2017 in order to avoid market fragmentation. In early 2017, the Virtual Reality Industry Forum (VR-IF) was established to support high quality interoperable VR experiences. Also, the 3rd Generation Partnership Project (3GPP) has an ongoing study item on VR that could possibly lead to a normative work item starting in the course of 2017. Moreover, the W3C WebVR Community Group is specifying APIs for accessing VR devices on the web.

This paper provides an overview of the key concepts of OMAF in Section 2. Section 3 describes three different streaming approaches, which are under consideration for inclusion in OMAF specification and discusses their advantages and disadvantages.