Learning 3D Representations from 2D Images
Details
View synthesis has been a long-standing problem in the computer graphics and vision community. Finding an efficient and expressive representation is the key to addressing this problem.
In this talk, I will introduce two of our recent projects in this frontier. The first is Neural Light-transport Fields (NeLF), which enables simultaneous view synthesis and relighting from casual portrait photos.
The other work is Deep 3D Mask Volume, a way to enable flicker-free view synthesis for dynamic scenes. I will talk about how we design the representations and the underlying network to generate them.
Also, I will outline some possible research directions in this area.
Lecture slides: https://drive.google.com/file/d/1JZhyywqU0yyun8EdNeX0zaukMmCz7fMT/view?usp=sharing
The talk is based on our recent papers:
(1) Deep 3D Mask Volume for View Synthesis of Dynamic Scenes, ICCV 2021
Project page: http://zhiqiangshen.com/projects/LS_and_KD/index.html
Git: https://github.com/ken2576/deep-3dmask
(2) NeLF: Neural Light-transport Field for Portrait View Synthesis and Relighting, EGSR 2021
Project page: https://cseweb.ucsd.edu//~viscomp/projects/EGSR21NeLF/
Git: https://github.com/ken2576/nelf
Presenter Bio:
Kai-En Lin is a 4th-year PhD student at UC San Diego advised by Prof. Ravi Ramamoorthi. Before that, he graduated with a bachelor degree in Electrical Engineering from National Taiwan University.
His research interests cover computer vision, image-based rendering and view synthesis. To be more specific, he focuses on how to represent the 3D visual world given a sparse set of 2D images.
He is a recipient of the Qualcomm FMA fellowship.
More information can be found at: https://cseweb.ucsd.edu/~k2lin/
-------------------------
Find us at:
All lectures are uploaded to our Youtube channel ➜ https://www.youtube.com/channel/UCHObHaxTXKFyI_EI8HiQ5xw
Newsletter for updates about more events ➜ http://eepurl.com/gJ1t-D
Sub-reddit for discussions ➜ https://www.reddit.com/r/2D3DAI/
Discord server for, well, discord ➜ https://discord.gg/MZuWSjF
Blog ➜ https://2d3d.ai
Learning 3D Representations from 2D Images