Skip to content

Details

This is a two parts talk. It is based on the papers "3D Scene Reconstruction from a Single Viewport" presented at this years ECCV and the "BlenderProc" paper. The speaker is the main author of both papers.

Part 1:
A novel solution will be presented to volumetric scene reconstruction based on single color images.

git : https://github.com/DLR-RM/SingleViewReconstruction
YouTube: https://youtu.be/wL6aPEb0Gsc
paper: https://tinyurl.com/singleviewreconstruction

Part 2:
BlenderProc will be highlighted - a procedural pipeline to generate images for the training of neural networks.

arxiv: https://arxiv.org/abs/1911.01911
git: https://github.com/DLR-RM/BlenderProc
YouTube: https://youtu.be/tQ59iGVnJWM

Finally, we will talk about an outlook what interesting fields of research lie ahead.

The talk is 2 hours long with a 10 minutes break in between the two parts.

Lecture abstract:

Part 1:
We present a novel approach to infer volumetric reconstructions from a single viewport, based only on a RGB image and a reconstructed normal image. The main contributions of reconstructing full scenes including the hidden and occluded areas will be discussed and their advantages in contrast to prior works which focused either on shape reconstruction of single objects floating in space or on complete scenes where either a point cloud or at least a depth image were provided. We propose to learn this information from synthetically generated high-resolution data. To do this, we introduce a deep network architecture that is specifically designed for volumetric TSDF data by featuring a specific tree net architecture. Our framework can handle a 3D resolution of 512³ by introducing a dedicated compression technique based on a modified autoencoder. Furthermore, we introduce a novel loss shaping technique for 3D data that guides the learning process towards regions where free and occupied space are close to each other.

Part 2:
We present BlenderProc, which is a modular procedural pipeline, helping in generating real looking images for the training of convolutional neural networks. These can be used in a variety of use cases including segmentation, depth, normal and pose estimation and many others. A key feature of our extension of blender is the simple to use modular pipeline, which was designed to be easily extendable. By offering standard modules, which cover a variety of scenarios, we provide a starting point on which new modules can be created.

Presenter BIO:

Maximilian Denninger is currently pursuing his PhD at the German Aerospace Center (DLR), where he is a full-time researcher. His research goal is to improve the computer vision on mobile robots, where the training data is always scarce. At the DLR he heads the vision part of an exciting project called SMiLE, where the goal is to design and implement robots, which are able to assist people working in elderly homes. This includes a variety of tasks from semantic segmentation to scene reconstruction. As robots need a natural understanding of their environment to fulfill any kind of task. For that he and his colleagues created BlenderProc, which helps in the generation of data for the training of neural networks. He is advised for his PhD by his department head Dr. Rudolph Triebel, which also works for the Technical University of Munich (TUM), where Max also works as a teaching assistant to help teach the course "Maching Learning for Computer Vision".

Linkedin: https://www.linkedin.com/in/maximilian-denninger/
Twitter: https://twitter.com/DenningerMax

This is a technical talk, prior knowledge of deep learning is advised.

-------------------------
Find us at:

All lectures are uploaded to our Youtube channel ➜ https://www.youtube.com/channel/UCHObHaxTXKFyI_EI8HiQ5xw

Newsletter for updates about more events ➜ http://eepurl.com/gJ1t-D

Sub-reddit for discussions ➜ https://www.reddit.com/r/2D3DAI/

Discord server for, well, discord ➜ https://discord.gg/MZuWSjF

Blog ➜ https://2d3d.ai

You may also like