Enabling Multiview- and Light Field-Video for Veridical Visual Experiences

Abstract

With the advent of UHDTV and the inclusion of High Dynamic Range, High Frame Rate and Extended Color Gamut 2D-imagery is able to push technical parameters up to the limits of the human visual sense. Consequently, developments in sensor technology can be used to capture information beyond 2D-imagery. In this paper we introduce multiview- and light field-video as an option to capture (at least parts of) the plenoptic function and therewith drive veridical visual experiences. Our contribution is on tools for capturing and encoding so called 5D light fields. We have built a multi-camera array producing up to 6 GigaRays/s and a real-time hierarchical H.264 MVC encoder that enables encoding the light fields in form of a legacy compliant video stream.