PUBLICATIONS

Spatially Varying Nanophotonic Neural Networks

Kaixuan Wei, Xiao Li, Johannes Froech, Praneeth Chakravarthula, James Whitehead, Ethan Tseng, Arka Majumdar, Felix Heide

Science Advances

Neural Light Spheres for Implicit Image Stitching and View Synthesis

Ilya Chugunov, Amogh Joshi, Kiran Murthy, Francois Bleibel, Felix Heide

SIGGRAPH Asia 2024

Learned Multi-aperture Color-coded Optics for Snapshot Hyperspectral Imaging

Zheng Shi, Xiong Dun, Haoyu Wei, Shiyu Dong, Zhanshan Wang, Xinbin Cheng, Felix Heide, Yifan (Evan) Peng

SIGGRAPH Asia 2024

SAMFusion: Sensor-Adaptive Multimodal Fusion for 3D Object Detection in Adverse Weather

Edoardo Palladin, Roland Dietze, Praveen Narayanan, Mario Bijelic, Felix Heide

ECCV 2024

Radar Fields: Frequency-Space Neural Scene Representations for FMCW Radar

David Borts, Erich Liang, Tim Brödermann, Andrea Ramazzina, Stefanie Walz, Edoardo Palladin, Jipeng Sun, David Bruggemann, Christos Sakaridis, Luc Van Gool, Mario Bijelic, Felix Heide

SIGGRAPH 2024

Split-Aperture 2-in-1 Computational Cameras

Zheng Shi, Ilya Chugunov, Mario Bijelic, Geoffroi Côté, Jiwoon Yeom, Qiang Fu, Hadi Amata, Wolfgang Heidrich, Felix Heide

SIGGRAPH 2024

Neural Single-Shot GHz FMCW Correlation Imaging

Cindy (Hsin) Pan, Noah Walsh, Yuxuan Zhang, Zheng Shi, Felix Heide

Optics Express 2024 (Spotlight on Optics)

Gated Fields: Learning Scene Reconstruction from Gated Videos

Andrea Ramazzina, Stefanie Walz, Pragyan Dahal, Mario Bijelic, Felix Heide

CVPR 2024

Neural Spline Fields for Burst Image Fusion and Layer Separation

Ilya Chugunov, David Shustin, Ruyu Yang, Chenyang Lei, Felix Heide

CVPR 2024

Flow-Guided Online Stereo Rectification for Wide Baseline Stereo

Anush Kumar, Fahim Mannan, Omid Hosseini Jafari, Shile Li, Felix Heide

CVPR 2024

Cross-spectral Gated-RGB Stereo Depth Estimation

Samuel Brucker, Stefanie Walz, Mario Bijelic, Felix Heide

CVPR 2024 (Highlight)

Polarization Wavefront Lidar: Learning Large Scene Reconstruction from Polarized Wavefronts

Dominik Scheuble, Chenyang Lei, Seung-Hwan Baek, Mario Bijelic, Felix Heide

CVPR 2024

Neural Exposure Fusion for High-Dynamic Range Object Detection

Emmanuel Onzon, Maximilian Bömer, Fahim Mannan, Felix Heide

CVPR 2024

Dispersed Structured Light for Hyperspectral 3D Imaging

Suhyun Shin, Seokjun Choi, Felix Heide, Seung-Hwan Baek

CVPR 2024

Spectral and Polarization Vision: Spectro-Polarimetric Real-World Dataset

Yujin Jeon, Eunsue Choi, Youngchan Kim, Khalid Omer, Felix Heide, Seung-Hwan Baek

CVPR 2024 (Highlight)

Neural Étendue Expander for Ultra-Wide-Angle High-Fidelity Holographic Display

Ethan Tseng, Grace Kuo, Seung-Hwan Baek, Nathan Matsuda, Andrew Maimone, Florian Schiffers, Praneeth Chakravarthula, Qiang Fu, Wolfgang Heidrich, Douglas Lanman, Felix Heide

Nature Communications

Inverse Neural Rendering
for Explainable Multi-Object Tracking

Julian Ost*, Tanushree Banerjee*, Mario Bijelic, Felix Heide (* indicates equal contribution)

arXiv Preprint

Thin On-Sensor Nanophotonic Array Cameras

Praneeth Chakravarthula, Jipeng Sun, Xiao Li, Chenyang Lei, Gene Chou,
Mario Bijelic, Johannes Froech, Arka Majumdar, Felix Heide

SIGGRAPH Asia 2023

Kissing to Find a Match:
Efficient Low-Rank Permutation Representation

Hannah Dröge, Zorah Lähner, Yuval Bahat, Onofre Martorell, Felix Heide, Michael Möller

NeurIPS 2023

ScatterNeRF: Seeing Through Fog with Physically-Based
Inverse Neural Rendering

Andrea Ramazzina, Mario Bijelic, Stefanie Walz, Alessandro Sanvito, Dominik Scheuble, Felix Heide

ICCV 2023

Diffusion-SDF: Conditional Generative Modeling of
Signed Distance Functions

Gene Chou, Yuval Bahat, Felix Heide

ICCV 2023

Single Depth-Image 3D Reflection Symmetry
and Shape Prediction

Zhaoxuan Zhang, Bo Dong, Tong Li, Felix Heide, Pieter Peers, Baocai Yin, Xin Yang

ICCV 2023

Multi-view Spectral Polarization Propagation for Video Glass Segmentation

Yu Qiao, Bo Dong, Ao Jin, Yu Fu, Seung-Hwan Baek, Felix Heide Pieter Peers, Xiaopeng Wei, Xin Yang

ICCV 2023

∇-Prox: Differentiable Proximal Algorithm Modeling for Large-Scale Optimization

Zeqiang Lai, Kaixuan Wei, Ying Fu, Philipp Härtel, Felix Heide

SIGGRAPH 2023

In the Blink of an Eye: Event-based Emotion Recognition

Haiwei Zhang, Jiqing Zhang, Bo Dong, Pieter Peers, Wenwei Wu, Xiaopeng Wei, Felix Heide, Xin Yang

SIGGRAPH 2023

Stochastic Light Field Holography

Florian Schiffers, Praneeth Chakravarthula, Nathan Matsuda, Grace Kuo, Ethan Tseng, Doug Lanman, Felix Heide, Oliver Cossairt

ICCP 2023

Gated Stereo: Joint Depth Estimation from Gated and Wide-Baseline Active Stereo Cues

Stefanie Walz, Mario Bijelic, Andrea Ramazzina, Amanpreet Walia, Fahim Mannan, Felix Heide

CVPR 2023

LiDAR-in-the-loop Hyperparameter Optimization

Félix Goudreault, Dominik Scheuble, Mario Bijelic, Nicolas Robidoux, Felix Heide

CVPR 2023

Shakes on a Plane: Unsupervised Depth Estimation from Unstabilized Photography

Ilya Chugunov, Yuxuan Zhang, Felix Heide

CVPR 2023

The Differentiable Lens: Compound Lens Search over Glass Surfaces and Materials for Object Detection

Geoffroi Côté, Fahim Mannan, Simon Thibault, Jean-François Lalonde, Felix Heide

CVPR 2023

Seeing With Sound: Acoustic Beamforming for Multimodal Scene Understanding

Praneeth Chakravarthula, Jim Aldon D'Souza, Ethan Tseng, Joe Bartusek, Felix Heide

CVPR 2023

Pupil-aware Holography

Praneeth Chakravarthula, Seung-Hwan Baek, Florian Schiffers, Ethan Tseng, Grace Kuo, Andrew Maimone, Nathan Matsuda, Oliver Cossairt, Douglas Lanman, Felix Heide

SIGGRAPH Asia 2022

Differentiable Point-Based Radiance Fields for Efficient View Synthesis

Qiang Zhang, Seung-Hwan Baek, Szymon Rusinkiewicz, Felix Heide

Siggraph Asia 2022

Neural Photo-Finishing

Ethan Tseng, Yuxuan Zhang, Lars Jebe, Cecilia Zhang, Zhihao Xia, Yifei Fan, Felix Heide, Jiawen Chen

SIGGRAPH Asia 2022

GenSDF: Two-Stage Learning of Generalizable
Signed Distance Functions

Gene Chou, Ilya Chugunov, Felix Heide

NeurIPS 2022 (Featured)

Biologically Inspired Dynamic Thresholds for Spiking Neural Networks

Jianchuan Ding, Bo Dong, Felix Heide, Yufei Ding, Yunduo Zhou, Baocai Yin, Xin Yang

NeurIPS 2022

All You Need is RAW:
Defending Against Adversarial Attacks with Camera Image Pipelines

Yuxuan Zhang, Bo Dong, Felix Heide

ECCV 2022

Centimeter-Wave Free-Space Neural Time-of-Flight Imaging

Seung-Hwan Baek, Noah Walsh, Ilya Chugunov, Zheng Shi, Felix Heide

SIGGRAPH 2022

Seeing Through Obstructions with Diffractive Cloaking

Zheng Shi, Yuval Bahat, Seung-Hwan Baek, Qiang Fu, Hadi Amata, Xiao Li, Praneeth Chakravarthula, Wolfgang Heidrich, Felix Heide

SIGGRAPH 2022

Handheld Multi-Frame Neural Depth Refinement

Ilya Chugunov, Yuxuan Zhang, Zhihao Xia, Cecilia Zhang, Jiawen Chen, Felix Heide

CVPR 2022 (Oral)

LiDAR Snowfall Simulation for Robust 3D Object Detection

Martin Hahner, Christos Sakaridis, Mario Bijelic, Felix Heide, Fisher Yu, Dengxin Dai, Lux Van Gool

CVPR 2022 (Oral)

Hogel-free Holography

Praneeth Chakravarthula, Ethan Tseng, Henry Fuchs, Felix Heide

SIGGRAPH 2022

Gated2Gated:
Self-Supervised Depth Estimation from Gated Images

Amanpreet Walia, Stefanie Walz, Mario Bijelic, Fahim Mannan, Frank Julca-Aguilar, Michael Langer, Werner Ritter, Felix Heide

CVPR 2022 (Oral)

Neural Point Light Fields

Julian Ost, Issam Laradji, Alejandro Newell, Yuval Bahat, Felix Heide

CVPR 2022

All-photon Polarimetric Time-of-Flight Imaging

Seung-Hwan Baek and Felix Heide

CVPR 2022

Glass Segmentation using Intensity and Spectral Polarization Cues

Haiyang Mei, Bo Dong, Wen Dong, Jiaxi Yang, Seung-Hwan Baek, Felix Heide, Pieter Peers, Xiaopeng Wei, Xin Yang

CVPR 2022

Spiking Transformers for Event-based Single Object Tracking

Jiqing Zhang, Bo Dong, Haiwei Zhang, Jianchuan Ding, Felix Heide, Baocai Yin, Xin Yang

CVPR 2022

Polarimetric Spatio-Temporal Light Transport Probing

Seung-Hwan Baek, Felix Heide

SIGGRAPH Asia 2021

Neural Nano-Optics for High-quality Thin Lens Imaging

Ethan Tseng, Shane Colburn, James Whitehead, Luocheng Huang, Seung-Hwan Baek, Arka Majumdar, Felix Heide

Nature Communications

Gated3D: Monocular 3D Object Detection From Temporal Illumination Cues

Frank Julca-Aguilar, Jason Taylor, Mario Bijelic, Fahim Mannan, Ethan Tseng, Felix Heide

ICCV 2021

Differentiable Compound Optics and Processing Pipeline Optimization for End-to-end Camera Design

Ethan Tseng, Ali Mosleh, Fahim Mannan, Karl St-Arnaud, Avinash Sharma, Yifan Peng, Alexander Braun, Derek Nowrouzezahrai, Jean-François Lalonde, Felix Heide

SIGGRAPH 2021

Low-Cost SPAD Sensing for Non-Line-Of-Sight Tracking, Material Classification and Depth Imaging

Clara Callenberg, Zheng Shi, Felix Heide, Matthias B. Hullin

SIGGRAPH 2021

Dirty Pixels: Towards End-to-End Image Processing and Perception

Steven Diamond, Vincent Sitzmann, Frank Julca-Aguilar, Stephen Boyd, Gordon Wetzstein, Felix Heide

SIGGRAPH 2021

Thallo – Scheduling for High-Performance Large-scale Non-linear Least-Squares Solvers

Michael Mara, Felix Heide, Michael Zollhöfer, Matthias Nießner, Pat Hanrahan

SIGGRAPH 2021

Polka Lines: Learning Structured Illumination and Reconstruction for Active Stereo

Seung-Hwan Baek, Felix Heide

CVPR 2021 (Oral)

Neural Scene Graphs for Dynamic Scenes

Julian Ost, Fahim Mannan, Nils Thuerey, Julian Knodt, Felix Heide

CVPR 2021 (Oral)

Mask-ToF: Learning Microlens Masks for Flying Pixel Correction in Time-of-Flight Imaging

Ilya Chugunov, Seung-Hwan Baek, Qiang Fu, Wolfgang Heidrich, Felix Heide

CVPR 2021

Adversarial Imaging Pipelines

Buu Phan, Fahim Mannan, Felix Heide

CVPR 2021

Neural Auto Exposure for High-Dynamic Range Object Detection

Emmanuel Onzon, Fahim Mannan, Felix Heide

CVPR 2021

End-to-end High Dynamic Range Camera Pipeline Optimization

Nicolas Robidoux, Luis Eduardo García Capel, Dong-eun Seo, Avinash Sharma, Federico Ariza, Felix Heide

CVPR 2021

ZeroScatter: Domain Transfer for Long Distance Imaging and Vision through Scattering Media

Zheng Shi, Ethan Tseng, Mario Bijelic, Werner Ritter, Felix Heide

CVPR 2021

Learned Feature Embeddings for Non-Line-of-Sight Imaging and Recognition

Wenzheng Chen, Fangyin Wei, Kyros Kutulakos, Szymon Rusinkiewicz, Felix Heide

SIGGRAPH Asia 2020

Learned Hardware-in-the-loop Phase Retrieval for Holographic Near-Eye Displays

Praneeth Chakravarthula, Ethan Tseng, Tarun Srivastava, Henry Fuchs, Felix Heide

SIGGRAPH Asia 2020

Learning Rank-1 Diffractive Optics for Single-shot High Dynamic Range Imaging

Qilin Sun, Ethan Tseng, Qiang Fu, Wolfgang Heidrich, Felix Heide

CVPR 2020 (Oral)

Hardware-in-the-loop End-to-end Optimization of Camera Image Processing Pipelines

Ali Mosleh, Avinash Sharma, Emmanuel Onzon, Fahim Mannan, Nicolas Robidoux, Felix Heide

CVPR 2020 (Oral)

Seeing Around Street Corners: Non-Line-of-Sight Detection and Tracking In-the-Wild Using Doppler Radar

Nicolas Scheiner, Florian Kraus, Fangyin Wei, Buu Phan, Fahim Mannan, Nils Appenrodt, Werner Ritter, Jürgen Dickmann, Klaus Dietmayer, Bernhard Sick, Felix Heide

CVPR 2020

Seeing Through Fog Without Seeing Fog: Deep Multimodal Sensor Fusion in Unseen Adverse Weather

Mario Bijelic, Tobias Gruber, Fahim Mannan, Florian Kraus, Werner Ritter, Klaus Dietmayer, Felix Heide

CVPR 2020

Single-shot Monocular RGB-D Imaging using Uneven Double Refraction

Andreas Meuleman, Seung-Hwan Baek, Felix Heide, Min H. Kim

CVPR 2020 (Oral)

Defending Against Universal Attacks Through Selective Feature Regeneration

Tejas Borkar, Felix Heide, Lina Karam

CVPR 2020

Deep-Inverse Correlography: Towards Real-Time High-Resolution Non-Line-Of-Sight Imaging

Christopher A. Metzler, Felix Heide, Prasana Rangarajan, Muralidhar Madabhushi Balaji, Aparna Viswanath, Ashok Veeraraghavan, Richard G. Baraniuk

Optica 2020

Learned Large Field-of-View Imaging With Thin-Plate Optics

Yifan Peng, Qilin Sun, Xiong Dun, Gordon Wetzstein, Wolfgang Heidrich, Felix Heide

SIGGRAPH Asia 2019

Wirtinger Holography for Near-Eye Displays

Praneeth Chakravarthula, Yifan Peng, Joel Kollin, Henry Fuchs, Felix Heide

SIGGRAPH Asia 2019

Gated2Depth: Real-time Dense Lidar from Gated Images

Tobias Gruber, Frank Julca-Aguilar, Mario Bijelic, Werner Ritter, Klaus Dietmayer, Felix Heide

ICCV 2019 (Oral)

Pixel-Accurate Depth Evaluation in Realistic Driving Scenarios

Tobias Gruber, Mario Bijelic, Felix Heide, Werner Ritter, Klaus Dietmayer

3DV 2019

Hyperparameter Optimization in Black-box Image Processing using Differentiable Proxies

Ethan Tseng, Felix Yu, Yuting Yang, Fahim Mannan, Karl St. Arnaud, Derek Nowrouzezahrai, Jean-François Lalonde, Felix Heide

SIGGRAPH 2019

Non-line-of-sight Imaging with Partial Occluders and Surface Normals

Felix Heide, Matthew O’Toole, Kai Zang, David Lindell, Steven Diamond, Gordon Wetzstein

ACM TOG, presented at SIGGRAPH 2019

Steady-state Non-Line-of-Sight Imaging

Wenzheng Chen, Simon Daneau, Fahim Mannan, Felix Heide

CVPR 2019 (Oral)

DeepVoxels: Learning Persistent 3D Feature Embeddings

Vincent Sitzmann, Justus Thies, Felix Heide, Matthias Nießner, Gordon Wetzstein, Michael Zollhöfer

CVPR 2019 (Oral)

Sub-picosecond photon-efficient 3D imaging using single-photon sensors

Felix Heide, Steven Diamond, David B. Lindell, Gordon Wetzstein

Nature Scientific Reports

Transient Convolutional Imaging

Felix Heide

IEEE Dissertation Hightlights

Discriminative Transfer Learning for General Image Restoration

Lei Xiao, Felix Heide, Wolfgang Heidrich, Bernhard Schölkopf, Michael Hirsch

IEEE Transactions on Image Processing (TIP) 2018

End-to-end Optimization of Optics and Image Processing for Achromatic Extended Depth of Field and Super-resolution Imaging

Vincent Sitzmann, Steven Diamond, Yifan Peng, Xiong Dun, Stephen Boyd, Wolfgang Heidrich Felix Heide, Gordon Wetzstein

SIGGRAPH 2018

Deep End-to-End Time-of-Flight Imaging

Shuochen Su, Felix Heide, Gordon Wetzstein, Wolfgang Heidrich

CVPR 2018

Focal Sweep Imaging with Multi-focal Diffractive Optics

Yifan Peng, Xiong Dun, Qilin Sun, Felix Heide, Wolfgang Heidrich

ICCP 2018

Snapshot Difference Imaging using Correlation Time-of-Flight Sensors

Clara Callenberg, Felix Heide, Gordon Wetzstein, Matthias B. Hullin

SIGGRAPH Asia 2017

IDEAL: Image DEnoising AcceLerator

Mostafa Mahmoud, Bojian Zheng, Alberto Delmás Lascorz, Felix Heide, Jonathan Assouline, Paul Boucher, Emmanuel Onzon, Andreas Moshovos

MICRO-50 2017

Consensus Convolutional Sparse Coding

Biswarup Choudhury, Robin Swanson, Felix Heide, Gordon Wetzstein, Wolfgang Heidrich

ICCV 2017

Reconstructing Transient Images from Single-Photon Sensors

Matthew O'Toole, Felix Heide, David B. Lindell, Kai Zang, Steven Diamond, Gordon Wetzstein

CVPR 2017

Convolutional Sparse Coding for RGB+NIR Imaging

Xuemei Hu, Felix Heide, Qionghai Dai, Gordon Wetzstein

IEEE Transactions on Image Processing (TIP) 2017

Structure-Aware Computational Imaging

Felix Heide Ph.D. dissertation, November 2016 won the Alain Fournier Award won the ACM SIGGRAPH outstanding doctoral dissertation award

Encoded diffractive optics for full-spectrum computational imaging

Felix Heide, Qiang Fu, Yifan Peng, Wolfgang Heidrich

Nature Scientific Reports

ProxImaL: Efficient Image Optimization Using Proximal Algorithms

Felix Heide, Steven Diamond, Jonathan Ragan-Kelley, Matthias Niessner, Wolfgang Heidrich, Gordon Wetzstein

SIGGRAPH 2016

Computational Imaging with Phased Time-of-Flight Camera Arrays

Shikhar Shrestha, Felix Heide, Wolfgang Heidrich, Gordon Wetzstein

SIGGRAPH 2016

The Diffractive Achromat: Full Spectrum Computational Imaging with Diffractive Optics

Yifan Peng, Qiang Fu, Felix Heide, Wolfgang Heidrich

SIGGRAPH 2016

Material Classification Using Raw Time-of-Flight Measurements

Shuochen Su, Felix Heide, Robin Swanson, Jonathan Klein, Clara Callenberg, Matthias Hullin, Wolfgang Heidrich

CVPR 2015

Convolutional Sparse Coding for High Dynamic Range Imaging

Ana Serrano, Felix Heide, Diego Gutierrez, Gordon Wetzstein, Belen Masia

Computer Graphics Forum (presented at Eurographics 2016)

Computational Imaging Using Lightweight Diffractive-Refractive Optics

Yifan Peng, Hadi Amata, Shuochen Su, Felix Heide, Wolfgang Heidrich

Optics Express, 2015

Doppler Time-of-Flight Imaging

Felix Heide, Wolfgang Heidrich, Matthias B. Hullin, Gordon Wetzstein

SIGGRAPH 2015

Fast and Flexible Convolutional Sparse Coding

Felix Heide, Wolfgang Heidrich, Gordon Wetzstein

CVPR 2015 (Oral)