Neural Single-Shot GHz FMCW Correlation Imaging

Single-chirp GHz FMCW Correlation Imaging

Correlation Time-of-flight (ToF) methods excel in active depth sensing [2], using intensity-modulated signal interference to infer depth from phase shifts. High-frequency ToF systems need phase unwrapping algorithms for absolute depth due to phase shifts exceeding 2π. Single-frequency unwrapping recovers relative depth but requires scene assumptions. Multi-frequency unwrapping uses lower frequencies to unwrap high-frequency phases, enabling absolute depth reconstruction. We propose an depth estimation approach that integrates frequency modulated continuous wave (FMCW) operation with all-optical correlation ToF, enabling accurate absolute depth reconstruction in the GHz range from a single-chirp measurement, eliminating the need for multiple frequency measurements and effectively halving the capture time.

Our method begins with FMCW capture hardware emitting frequency-modulated signals into the scene. Reflected signals are captured, and theFMCWcorrelation signals are optically computed in the time domain. Phase shifts are then extracted before using FFT converting these signals to the frequency domain. Next, pixel-wise frequency decoding is applied to the frequency domain signals to determine absolute depths for each pixel. The final step involves a test-time optimization, which further refines the depth output by integrating the estimated depths with phase information.

All-Optical FMCW ToF Measurement Prototype

Building on Baek et al. 2022 [1], we implemented the proposed method with the all-optical FMCW prototype system shown below. (a) shows a schematic illustration of our set up and the photograph of the experimental setup is shown in (b), with light paths marked in green. The illumination module starts with a polarizing beam-splitter (PBS) converting a 3mW, 532 nm light beam into vertically polarized light. This light is then modulated by a half wave plate (HWP), a quarter wave plate (QWP), and an Electro-Optic Modulator (EOM), in sequence. After reflecting off a mirror, the sequence is repeated in reverse. An RF generator inputs a frequency chirped sinusoidal voltage (7.15 GHz center frequency, 20 MHz bandwidth) to the EOMs. The modulated signal travels to the scene and back to the detection module, which mirrors the illumination setup with synchronized HWPs, QWPs, and EOMs, and a mirror. The returned light is demodulated, mixed with a reference signal, and passed through a 1 MHz lowpass filter to extract the correlation signal.

Assessment

Using the above prototype, we validate the proposed system on real-world scenes containing target objects with different materials and complex geometric structures. In each case, we gathered 100×100 pixel measurements using our experimental prototype, assisted by a galvo scanning system. And we additionally compared to the single-frequency phase unwrapping method PUMA [3] using the same 7.15GHz measurement. Given that PUMA [3] exclusively reconstructs the relative wrap count, we modified its output to correspond with the depth range of the test scenes.

Related Work

[1] Seung-Hwan Baek, Noah Walsh, Ilya Chugunov, Zheng Shi, Felix Heide. Centimeter-Wave Free-Space Neural Time-of-Flight Imaging. SIGGRAPH 2022
[2] Ilya Chugunov, Seung-Hwan Baek, Qiang Fu, Wolfgang Heidrich, Felix Heide. Mask-ToF: Learning Microlens Masks for Flying Pixel Correction in Time-of-Flight Imaging. CVPR 2021
[3] J. M. Bioucas-Dias and G. Valadao, “Phase unwrapping via graph cuts,” IEEE Trans. on Image Process. 16(3), 698–709 (2007).