Neural Nano-Optics for High-quality Thin Lens Imaging

Video Summary of Neural Nano-Optics. We propose a computational imaging method for end-to-end learning of ultra-thin meta-surface lenses and the reconstruction of captured measurements. Although the size of a rice grain, neural nano-optics almost achieve the same quality as bulky compound lenses.

Salt Grain-sized Cameras

The ultracompact camera we propose uses metasurface optics at the size of a coarse salt grain and can produce crisp, full-color images on par with a conventional compound camera lens 500,000 times larger in volume.

Comparisons against State-of-the-art

Neural nano-optics outperform existing state-of-the-art metasurface lens designs in broadband and it is the first meta-optic imager that achieves high-quality, wide field-of-view color imaging.

Comparisons against Conventional Cameras

Neural nano-optics can recover images with quality comparable to conventional cameras with bulky refractive compound lenses.

Metasurface Proxy Model

We learn a nano-optic by using an efficient differentiable proxy function that maps phase values to spatially varying PSFs. This allows our approach to be more than three orders of magnitude more efficient than full-wave simulation methods such as FTDT.

Feature-based Deconvolution

To recover images from the measured data, we propose a feature-based deconvolution method that employs a differentiable inverse filter together with neural networks for feature extraction and refinement. This combination allows us to learn effective features that facilitate accurate and physics-based deconvolution utilizing the knowledge of the PSF to promote generalizability.

Learning Meta-Optics in an End-to-end Fashion

With the metasurface proxy and neural deconvolution model in hand, comprising a fully differentiable imaging pipeline, we design the proposed nano-camera in an end-to-end fashion. The figure below illustrates the learning method and a corresponding optimization run.

Diffractive Lens Achromatic Capacity (DLAC)

Our optimized meta-optic design meets several criteria that were not met by previous methods. The nano-camera allows us to capture full-color images over a wide field-of-view. Furthermore, our aperture is the largest demonstrated for meta-optics at 500 microns, allowing for increased light collection. Our optic also exhibits a low f-number, allowing for the optic to be placed extremely close to the camera sensor. No previous metasurface has demonstrated imaging with this combination of large aperture, large field-of-view, small f-number, and large fractional bandwidth. Previously, achieving any one of these metrics came at the cost of some other, for example, one could achieve a low f-number by sacrificing the aperture size. To formally quantify our design specs, we propose a new metric called the Diffractive Lens Achromatic Capacity.

Meta-optic PSFs

We visualize the simulated PSFs for each design. Our optimization produces a meta-optic whose spatial PSFs are compact across all wavelengths and are slowly varying across field angles. Other designs exhibit PSFs with greater blur, especially for the red wavelength, and with rapid variation across field angles. The PSF corresponding to the cubic design contains large tails that cause the overall size of the PSF to be much larger than that of all other designs. As such, we only display the center crop of cubic design PSFs from Colburn et al.

Siemens Resolution Chart

Siemens Resolution Star captures compared to heuristic designs by Colburn et al. and a refractive compound lens. Computational reconstruction with our neural nano-optic imager recovers spatial resolution in broadband almost matching the performance of a refractive compound lens. See Supplemental Material for USAF 1951 chart comparisons.