This paper describes a novel scene-adaptive imaging technology designed to enhance the image quality of wide-angle immersive videos such as 360-degree videos. It addresses the challenge of balancing resolution, frame rate, and dynamic range due to sensor limitations by dynamically adjusting shooting conditions within a single frame on the basis local subject characteristics.
Abstract
We present a novel scene-adaptive imaging technology designed to enhance the image quality of wide-angle immersive videos such as 360-degree videos. It addresses the challenge of balancing resolution, frame rate, and dynamic range due to sensor limitations by dynamically adjusting shooting conditions within a single frame on the basis local subject characteristics. This involves capturing still subjects at high resolution and moving subjects at increased frame rates, adjusting exposure time according to subject brightness while maintaining pixel readout rate. To validate this approach, we developed a block-wise-controlled image sensor prototype with 1.1 million pixels that enables flexible control of shooting conditions individually for 272 separated blocks. Real-time scene analysis and a feedback control system were also developed. Experimental results demonstrate that the proposed method improves subjective image quality compared with conventional imaging that captures the entire frame under a single shooting condition, even at the same data rate.
Introduction
The global demand for highly immersive video content, such as 360-degree videos and dome screen videos, is escalating. Accompanying this demand is an increasing need for cameras capable of capturing wide viewing angles effectively (e.g. panoramic cameras and omnidirectional multi-cameras). Wide-angle videos typically feature subjects exhibiting diverse textures, movements, and brightness on a single screen, requiring image sensors to meet rigorous performance quality, including not only resolution and frame rates exceeding ultra-high definition television levels (see [1]) but also excelling in dynamic range for incident light. However, developing an image sensor that fulfils all these requirements simultaneously is challenging. Traditional image sensors, such as Complementary Metal Oxide Semiconductor (CMOS) image sensors operating under constant shooting conditions across the entire pixel array, are limited by a trade-off between resolution, frame rate, and the noise performance related to dynamic range (El Desouki et al [2] and Kawahito [3]). Moreover, higher pixel readout rates lead to increased data transfer streams and higher power consumption in image sensors.
No comments yet