Thin On-Sensor Nanophotonic Array Cameras

We propose a thin nanophotonic imager that employs a learned array of metalenses to capture a scene in-the-wild. We devise an array of lenses mounted directly on top of the sensor cover glass (top center). Each lens in our array (illustrated on the left) is a flat (700 nm thick) metasurface area of nano-antennas which we design to focus light across the visible spectrum. The peripheral elements capture images at slanted field angles, making it possible to capture a wide field of view of 100◦, more than twice as large as the most similar design from Tseng et al. [2021] shown on the right side. The proposed computational camera is capable of recovering images outside of lab conditions under broadband illumination. We illustrate here the matching physical image size as measured on the sensor, with the gray area on right illustrating same sensor size.

Replacing Compound Optics with Metalens Array

Today’s commodity camera systems rely on compound optics to map light originating from the scene to positions on the sensor where it gets recorded as an image. To record images without optical aberrations, i.e., deviations from Gauss’ linear model of optics, typical lens systems introduce increasingly complex stacks of optical elements which are responsible for the height of existing commodity cameras. In this work, we investigate flat nanophotonic computational cameras as an alternative that employs an array of skewed lenslets and a learned reconstruction approach.

Probabilistic Image Reconstruction

We propose a deconvolution method that relies on the physics-based forward model (PSFs illustrated on the left) along with a learned probabilistic prior (diffusion model on the bottom right). The proposed method is an unrolled optimization method that alternates between inverse filtering steps using the calibrated PSFs, diffusion steps that sample from the natural image manifold with conditioning on the current iteration, and a merging step that combines the image estimates from all sub-apertures (green). The unrolled optimization method is trained in an end-to-end fashion with paired training samples captured with a co-axial reference camera (bottom and left).

Experimental Evaluation on Broadband Scenes

Experimental evaluation of the proposed thin nanophotonic camera for broadband indoor & outdoor scenes. The proposed nanophotonic array optic with the probabilistic deconvolution method reconstructs the underlying latent image robustly in broadband environments.

Optical Performance Analysis

We analyze the optical performance of the proposed nanophotonic array lens via its theoretical modulation transfer function (MTF), i.e., the ability of the array lens to transfer contrast at a given spatial frequency (resolution) from the object to the imaging sensor. The analysis validates that the proposed metalens exhibits a improved MTF under different angle of incidence (AoI) compared to the existing designs, approaching the efficiency of diffraction limit for normal incidence of light and achieving higher MTF values for larger incidence angles.