Sensing Depth with 3D Computer Vision - Dr. Benjamin Busam

Our world is 3D. However, most computer vision sensing happens on a 2D pixel grid. To reason about distances, one needs to understand how far object pixels are in the real world where distance estimation from sensor input provides an essential tool for computer vision applications. For an autonomous vehicle to drive, it is essential to know the distance to cars, pedastrians and obstacles to not cause accidents; and for augmented reality applications to look convincing, 3D understanding of the scene is key to realistically hide augmentations behind scene content.
In this talk, we will give a background on different depth sensing technologies such as multi-view stereo, time-of-flight sensing, and monocular approaches - and look how sensor fusion, synthetic data and self-supervision can boost the performance of depth estimation. We identify major obstacles and present first proposals to overcome some of them.

Lecture slides: https://www.dropbox.com/s/j67kq0jqwspz7kj/busam_sensing_depth_3DCV.pdf?dl=0

The talk will take a tour through the literature in the field of depth estimation and exemplify some tricks and solutions with ideas of recent papers by the speaker such as:

Lopez-Rodriguez, Busam, Mikolajczyk. ACCV 2020.
Project to Adapt: Domain Adaptation for Depth Completion from Noisy and Sparse Sensor Data
https://github.com/alopezgit/project-adapt

Jung, Brasch, Leonardis, Navab, Busam. 3DV 2021.
Wild ToFu: Improving Range and Quality of Indirect Time-of-Flight Depth with RGB Fusion in Challenging Environments
https://arxiv.org/abs/2112.03750

Gasperini, Koch, Dallabetta, Navab, Busam, Tombari. 3DV 2021.
R4Dyn: Exploring Radar for Self-Supervised Monocular Depth Estimation of Dynamic Scenes
https://arxiv.org/abs/2108.04814

Ruhkamp, Gao, Chen, Navab, Busam. 3DV 2021.
Attention meets Geometry: Geometry Guided Spatial-Temporal Attention for Consistent Self-Supervised Monocular Depth Estimation
https://github.com/DaoyiG/TC-Depth

Curriculum Vitae:

Benjamin Busam is a Senior Research Scientist with the Technical University of Munich coordinating the Computer Vision activities at the Chair for Computer Aided Medical Procedures. Formerly Head of Research at FRAMOS Imaging Systems, he led the 3D Computer Vision Team at Huawei Research, London from 2018 to 2020. Benjamin studied Mathematics at TUM. In his subsequent postgraduate programme, he continued in Mathematics and Physics at ParisTech, France and at the University of Melbourne, Australia, before he graduated with distinction at TU Munich in 2014. In continuation to a mathematical focus on projective geometry and 3D point cloud matching, he now works on 2D/3D computer vision for pose estimation, depth mapping and mobile AR as well as multi-modal sensor fusion and collaborative robotics. For his work on adaptable high-resolution real-time stereo tracking he received the EMVA Young Professional Award 2015 from the European Machine Vision Association and was awarded Innovation Pioneer of the Year 2019 by Noah's Ark Laboratory, London. Benjamin is part of the programme commitee for CVPR, ICCV, and ECCV, and was recently awarded with the 3DV Outstanding Reviewer Award consecutively in 2020 and 2021.

https://www.in.tum.de/campar/members/benjamin-busam/

***

Find us at:

All lectures are uploaded to our Youtube channel ➜ https://www.youtube.com/channel/UCHObHaxTXKFyI_EI8HiQ5xw

Newsletter for updates about more events ➜ http://eepurl.com/gJ1t-D

Sub-reddit for discussions ➜ https://www.reddit.com/r/2D3DAI/

Discord server for, well, discord ➜ https://discord.gg/MZuWSjF

Blog ➜ https://2d3d.ai