Implicit Neural Representation Networks Fitting Signals, Derivatives & Integrals

Implicit Neural Representation Networks for Fitting Signals, Derivatives, and Integrals

Julien Martel, Postdoctoral Research Fellow at Stanford University in the Computational Imaging Lab
David B. Lindell, Postdoctoral Scholar at Stanford University and incoming Assistant Professor in the Dept. of CS at University of Toronto

Join Zoom Meeting
https://us02web.zoom.us/j/85283292799?pwd=K3FXYzJoSC9rUVUrbzFNTGxNSU1HUT09
Meeting ID: 852 8329 2799
Passcode: 634270
One tap mobile +16699006833,,85283292799# US (San Jose)
Find your local number: https://us02web.zoom.us/u/koPLgXz94

YouTube
https://youtu.be/IZw-Fbgyf38

Abstract
Implicitly defined, continuous, differentiable signal representations parameterized by neural networks have emerged as a powerful paradigm, offering many possible benefits over conventional representations and new capabilities in neural rendering and view synthesis. However, conventional network architectures for such implicit neural representations are incapable of modeling signals at scale with fine detail and fail to represent derivatives and integrals of signals. In this talk, we describe three recent approaches to solve these challenging problems. First, we introduce sinusoidal representation networks or SIREN, which are ideally suited for representing complex natural signals and their derivatives. Using SIREN, we can represent images, wavefields, video, sound, and their derivatives, allowing us to solve differential equations using this type of neural network. Second, we introduce a new framework for solving integral equations using implicit neural representation networks. Our automatic integration framework, AutoInt, enables the calculation of any definite integral with two evaluations of a neural network. This allows fast inference and rendering when applied to neural rendering techniques based on volume rendering. Finally, we introduce a new architecture and method for scaling up implicit representations, called Adaptive Coordinate Networks (ACORN). The approach relies on a hybrid implicit–explicit representation and a learned, online multiscale decomposition of the target signal. We use ACORN to demonstrate the first experiments that fit gigapixel images to nearly 40 dB peak signal-to-noise ratio (an 1000x increase in scale over previous experiments), and we reduce training times for 3D shape fitting from days to hours or minutes while improving memory requirements by over an order of magnitude.

Bios
Julien Martel (http://www.jmartel.net/) is a Postdoctoral Research Fellow at Stanford University in the Computational Imaging Lab led by Gordon Wetzstein. His research interests are in unconventional visual sensing and processing. More specifically, his current topics of research include the co-design of hardware and algorithms for visual sensing, the design of methods for vision sensors with in-pixel computing capabilities, and the use of novel representations for visual data such as neural implicit representations.

David B. Lindell (https://davidlindell.com) is a postdoctoral scholar at Stanford University and an incoming Assistant Professor in the Department of Computer Science at University of Toronto. He received a PhD in Electrical Engineering from Stanford University and is the recipient of the ACM SIGGRAPH 2021 Outstanding Dissertation Honorable Mention Award. His research spans the areas of computational imaging, computer vision, and machine learning with a focus on new methods for active 3D imaging and physics-based machine learning.

Joint Event with SF Bay ACM Chapter
https://www.meetup.com/SF-Bay-ACM/