Beating Spectral Bandwidth Limits for Large Aperture Broadband Nano-optics

  • Johannes E. Fröch

  • Praneeth Chakravarthula

  • Jipeng Sun

  • Ethan Tseng

  • Shane Colburn

  • Alan Zhan

  • Forrest Miller

  • Anna Wirth-Singh

  • Quentin A. A. Tanguy

  • Zheyi Han

  • Karl F. Böhringer

  • Felix Heide

  • Arka Majumdar

Nature Communications

Flat optics have been proposed as an attractive approach for the implementation of new imaging and sensing modalities to replace and augment refractive optics. However, chromatic aberrations impose fundamental limitations on diffractive flat optics. As such, true broadband high-quality imaging has thus far been out of reach for fast f-numbers, large aperture, flat optics.

In this work, we overcome intrinsic spectral bandwidth limitations, achieving broadband imaging in the visible wavelength range with a flat meta-optic, co-designed with computational reconstruction. We achieve on-par imaging performance with a single refractive lens, where we achieve an average MTF contrast after computation of > 30% in a range of 0 − 70 lp/mm, in comparison with an MTF contrast of ~ 50%, for a refractive lens in the same range (after computation). Moreover, we demonstrate improved imaging capabilities using a meta-optic at field angles of 10° and larger. The designed meta-optic is suitable for ambient imaging, and video rate captures with short exposure times in the ms range, which we directly exhibit by capturing dynamic scenes. We validate on-par performance with compound imagers with a learned reconstruction method using probabilistic diffusion models.

Overall, this work shows how broadband imaging in the visible is achievable with an ultra-flat optical element and large aperture (1 cm), and thus challenges the well-trenched belief in the community that broadband imaging is impossible using a large-aperture meta-optics.

Johannes E. Fröch, Praneeth Chakravarthula, Jipeng Sun, Ethan Tseng, Shane Colburn, Alan Zhan, Forrest Miller, Anna Wirth-Singh, Quentin A. A. Tanguy, Zheyi Han, Karl F. Böhringer, Felix Heide, Arka Majumdar

Beating Spectral Bandwidth Limits for Large Aperture Broadband Nano-optics

Nature Communications

Paper
Supplement
Bibtex

Extended Depth of Focus for Full-Color Imaging

By constraining the propagation to only the on-axis field, we compress the design space of required scatterers from ~ 109 to ~ 104. The broadband capability is implemented in a two-step optimization process. First, a meta-optic is designed to exhibit maximal focal intensity for a dense sampling of wavelengths. This yields extended focus profiles along the optical axis – as plotted for three different wavelengths within the optimization set. In the second step, we use this phase profile as the starting point for an end-to-end computational imaging optimization approach.

Left image: Schematic description of the design challenge to achieve broadband focusing. By constraining the propagation to only the on-axis field, we compress the design space of required scatterers from ~ 109 to ~ 104. Right image: Schematic description of the two-step optimization process. A meta-optic is first designed to exhibit maximal focal intensity for a dense sampling of wavelengths and then optimized by an end-to-end image reconstruction loss.

1 cm-Aperture Metalens Prototype

We integrated the 1-cm aperture flat nano-optic directly with a commercial camera using a 3D-printed aperture. Without the aid of any relay optic, we can directly assess the imaging performance from a system-level perspective. The large aperture size of the presented broadband meta-optic enables imaging at high framerates. The square scatterer geometry and material platform (SiN on quartz) are suitable for mass manufacturing in foundries using nanoimprint lithography, which is cost-effective and scalable.

Images from left to right: Integrated 1cm metalens camera prototype; Optical microscope image of the center of the meta-optic;  Scanning electron microscope (SEM) images of the scatterer in a oblique view of 30°.

Experimental Capture and Reconstruction

Our metalens camera prototype enables ambient imaging at millisecond-scale exposure times. Scene details are reconstructed from these measurements via a probabilistic diffusion-based algorithm. The raw video input (left) and its corresponding reconstructed output (right) are displayed for direct comparison.

Probabilistic Diffusion-based Reconstruction

The same image scenes are captured with a meta-optical camera and a compound lens camera. For spatially varying aberration correction, and better noise reduction, we designed a probabilistic diffusion-based neural network. To ensure the viability of the specific architecture for imaging under arbitrary conditions (ambient conditions and scene independence), we evaluate the method with a co-axial paired image capture system.

Experimental Evaluation

We directly compare ground truth captures (obtained via the compound lens imager), images reconstructed with a physics-based inverse filter, and the learned backend. We note that the presented images were not part of the training set but were captured for verification of the approach. The learned probabilistic diffusion model outputs images that yield lower haze, better color accuracy, produce more vivid hues, better noise reduction, account for spatial variance, and non-uniform sensor response. As such, the broadband meta-optic with the learned backend produces images that are almost on-par with compound lens camera imagers.

Comparison of images taken with a compound refractive lens, the proposed broadband meta-optics, and a physics-based inverse filter, and with a learned backend. Zoomed in details of the highlighted boxes in the images are directly compared next to each other.

Metalens vs. Refractive Lenses

An optical setup was developed to measure the point spread function (PSF) and contrast values of various optical elements under different angles of incidence (0°, 5°, 10°, and 15°) and wavelengths. A plano-convex lens was used to compare PSFs at these angles. The system also evaluated modulation transfer function (MTF) enhancement for line pair contrast before and after deconvolution across different optical designs, including a hyperboloid metalens, a refractive lens, and an end-to-end broadband meta-optic. Results show improved contrast after deconvolution and highlight the broadband design’s performance advantages.

Top row left: Schematic of the optical setup to measure the point spread function (PSF) for various optical elements under different angles of incidence (aoi). Top row center and right: Comparison of measured PSFs for a plano-convex lens and the broadband end-to-end meta-optic for aoi of 0°, 5°,10°, and 15°. Scale bars in the images correspond to 40 µm for aoi of 0° and 5°, and 200 µm for aoi of 10° and 15°. Bottom row left: Schematic of the optical setup to measure the contrast values for various optical elements for different wavelengths. Bottom row center: Examples of enhancement in MTF for line pair contrast before (dashed lines) and after (full lines) deconvolution for the hyperboloid (red diamond), refractive (blue square), and end-to-end design (green star). The error bars represent one standard deviation of the average intensity. Bottom row right: Comparison of the obtained contrast values for line pairs are shown before (left column) and after deconvolution (middle column), as well as the relative difference (right column). From the top down, the plots correspond to the refractive lens, the hyperboloid metalens, and the end-to-end broadband meta-optic.

Experimental Comparison

In the first row, a set of images was projected on an OLED screen, captured with various optical elements, and then reconstructed. To evaluate the design in the absence of learned priors, we employ Wiener filtering as a physics-informed inverse filtering method which relies on the PSF for reconstruction. From left to right, columns correspond to (a) the ground truth, and the computationally reconstructed images with (b) refractive lens, (c) hyperboloid metalens, and (d) end-to-end designed meta-optic. We additionally provide direct image comparison with magnified details of (e) a ground truth image for (f) a refractive lens and (g) the end-to-end meta-optic.

Related Publications

[1] Ethan Tseng, Shane Colburn, James Whitehead, Luocheng Huang, Seung-Hwan Baek, Arka Majumdar, Felix Heide. Neural Nano-Optics for High-quality Thin Lens Imaging. Nature Communications, 2021

[2] Praneeth Chakravarthula, Jipeng Sun, Xiao Li, Chenyang Lei, Gene Chou, Mario Bijelic, Johannes Froech, Arka Majumdar, Felix Heide. Thin On-Sensor Nanophotonic Array Cameras. Transactions on Graphics 2023