Dispersed Structured Light for Hyperspectral 3D Imaging
- Suhyun Shin
-
Seokjun Choi
- Felix Heide
- Seung-Hwan Baek
CVPR 2024
We introduce dispersed structured light, a low-cost high-quality hyperspectral 3D imaging method. By placing a diffraction grating film on a conventional camera-projector setup, we disperse structured-light patterns. Analyzing the images captured under the dispersed structured light enables accurate hyperspectral 3D reconstruction. (a) Capture configuration, (b) estimated hyperspectral image in sRGB, (c) comparison with spectroradiometer measurements, (d) estimated depth map, (e) estimated hyperspectral image.
Dispersed Structured Light for Hyperspectral 3D Imaging
Suhyun Shin, Seokjun Choi, Felix Heide, Seung-Hwan Baek
CVPR 2024
Imaging Setup
We devise our imaging system with the goals of compactness and affordability. We combine a conventional trichromatic camera with a trichromatic projector and a diffraction-grating film mounted in front. This configuration makes light from the projector undergo dispersion. As such, patterns emitted from the projector are spatially dispersed depending on wavelength. Our prototype consists of an RGB projector equipped with a diffraction grating film, and an RGB camera.
The left image shows an example projector pattern and its corresponding captured image, exhibiting clear first-order diffraction. For zero-order diffraction, light does not experience wavelength-dependent changes, and the projected squares retain their original shapes. For first-order diffractions, each square undergoes wavelength-dependent shift.
Hyperspectral 3D Imaging with Dispersed Structured Light
White Scanline Pattern
We use the scanline structured light patterns that scan through the whole column. The first-order diffracted light from the i-th scanline pattern produces a narrow-band illumination spread across multiple columns. Scanning columns from left to right enables illuminating each scene point with every narrow-band light, facilitating high-quality hyperspectral reconstruction.
Dispersed Structured Light
We combine a conventional trichromatic camera (FLIR GS3-U3-32S4C-C) with a trichromatic projector (LG PH30N) and a diffraction-grating film (Edmund #54-509) mounted in front. This configuration makes light from the projector undergo dispersion, and, as such, patterns emitted from the projector are spatially dispersed depending on wavelength.
Reconstructed Result
This figure shows the spectral accuracy of our DSL measured on a ColorChecker and metameric fake and real leaves. For smooth spectral curves of color patches, DSL successfully reconstructs the hyperspectral intensity. Moreover, DSL enables telling clear difference between fake and real leaves. We obtained the ground-truth intensity using a spectroradiometer.
Experimental Scene Reconstructions
The following scene consists of a colored board, a painting of a plant, and some drawing tools. We show the reconstructed hyperspectral image in sRGB, the reconstructed depth maps, the comparison with spectrometer measurements for 3 points and detailed hyperspectral image from 430 nm to 660 nm at 5 nm intervals. Our DSL method accurately reconstructs the high-frequency hyperspectral spectral curves across each of the nine bandpass filters, underscoring the advanced capabilities of our DSL method in handling high-frequency spectral reconstruction.
White Scanline Pattern
Dispersed Structured Light
Recovered Scene
Additional Reconstructions
The following scene consists of a colored board, and real and fake plants. We show the reconstructed hyperspectral image in sRGB, the reconstructed depth, the metamerism of real plant and fake plant learned by spectroradiometer, and the detailed hyperspectral image for 5 nm interval.
White Scanline Pattern
Dispersed Structured Light
Recovered Scene
The following scene inlcudes a toy bear, colored board, and various other items with diverse reflectance.
White Scanline Pattern
Dispersed Structured Light
Recovered Scene
Related Publications
[1] Seung-Hwan Baek and Felix Heide. Polarimetric spatio-temporal light transport probing. ACM ToG, SIGGRAPH 2021
[2] Seung-Hwan Baek and Felix Heide, “Polka Lines: Learning Structured Illumination and Reconstruction for Active Stereo,” CVPR 2021