The LiveRay project will create the first integrated light-field camera & light-field display system that enables the direct capturing of 3D light fields with a purpose-built camera, and visualize it in real time on a glasses-free light-field 3D display. Light-field is an emerging concept for representing rich 3D visual information that is able to capture real world phenomena with unprecedented image quality. A light field describes visual information as a set of light rays that pass a (sensor / display) surface, but apart from capturing the position and color of each light ray, the direction of each light ray is also recorded.

As a start, the partners will identify the mapping between light field representations on the capture and display side, finding the optimal configuration of resolution, field of view, angular resolution, bit depths, depth range, and other parameters. We will then implement light-field conversion algorithms to map the captured data onto the display's light rays, bridging any identified gaps with sampling, filtering and interpolation techniques where necessary.

The interfacing between the two devices and the accompanying software libraries will be matched, specifying electrical connections, protocols and compression techniques, considering the high data rate required for both the camera and the display side, as well as distributed processing needs on the rendering side. Distributed processing will also be introduced on the capture side to achieve better performing depth computation, and all-in-focus rendering.

Combining light-fields captured by a number of synchronized light-field cameras into a single light-field is currently not possible. Fast algorithms will be developed that can run on distributed processing systems, to enable real-time multiple light-field capture, stitching and display.

The outcome of the stitching process will require novel rendering techniques on the display side, as the data captured in this way will form a light-field with a non-planar plane of reference.

Software to support the demonstration of the envisaged use cases will be developed, as well as sample datasets will be recoded to support marketing and dissemination activities.