Computational imaging, relighting and depth sensing using flexible thin-film sensors

Imaging has transformed from the use of photosensitive films to computational methods, which allow for several novel imaging systems. For instance, coded aperture with new reconstruction approaches actualize flat camera designs. New materials have enabled the design of scalable, flexible, as well as transparent thin-film sensors. Enhanced single-pixel cameras are spatial-temporal illumination and compressive sensing enabled. Computational reconstruction approaches have been applied using a wide range of raw data from various sensors, and have been noted to allow for advanced image processing, for instance, light fields refocusing.

In a recent paper published in Optics Express, Professor Oliver Bimber and Dr. Alexander Koppelhuber a post-doctoral fellow at Johannes Kepler University Linz in Austria presented a model and reconstruction approaches for computational imaging using flexible optical sensors. The model and the methods could also be applied for depth reconstruction as well as relighting. The proposed sensors consisted of a luminescent concentrator. A luminescent concentrator is a flexible transparent foil which is doped with fluorescent dyes.

The authors adopted a luminescent concentrator foil prototype which was fixed on a transparent support cylinder. They cut aperture structure at the concentrator edge and the center hole into the foil. Polymethylmethacrylat multi-modal optical fibers were employed to direct the integrated light signal from every aperture to photodetectors at scan cameras for direct measurements. The team implemented a spatial light modulator to project the speckle patterns.

Light of predetermined spectral sub-band scattered from the cylinder onto the luminescent concentrator was absorbed and consequently emitted at larger wavelengths by the fluorescent dye particles in the concentrator foil. The light was then moved to the concentrator edges by total internal reflection. They were able to record the transported light via an elementary aperture cut into the concentrator material, within a small integration angle at varying positions and directions along the edges of the foil.

Via a hole in the concentrator foil, the researchers illuminated the cylinder with a series of speckle patterns from a micromirror device spatial modulator. The authors, from light back-scattering measurements, were able to reconstruct the cylinder images as observed from the modulator for various lighting conditions. The outcomes of the study paved way for computational relighting as well as depth reconstruction from shading.

The developed apertures had an advantage over the single pixel detectors in the sense that, as opposed to the latter that collect light via a narrow aperture, the former received light over a wider solid angle as well as a large area over the concentrator foil. This yielded high signal-to-noise ratio, which did not demand sensitive photodetectors.

The authors managed to raise the signal-to-noise ratio of photo sensors through the implementation of the luminescent concentrators. The use of luminescent concentrators comes with some benefits; light collection can be multiplexed over a number of overlapping integration areas to enable reconstruction of a number of lighting conditions while performing a few parallel measurements. In the future, the team will focus on improving signal-to-noise ratio as well as reconstruction quality.