Tom Tirer - GANs N' Denoisers: Appetite for Reconstruction
Details
Join Zoom Meeting
https://zoom.us/j/6043600514?pwd=VTFuU2VSTTNhTE1RRFJTZjhZNTN1Zz09
Meeting ID: 604 360 0514
Password: 703769
Ill-posed inverse problems appear in many image processing applications, such as deblurring, super-resolution and compressed sensing. Traditional reconstruction strategies, which involve minimizing a composition of fidelity and prior terms, exhibit limited performance due to the hardness in the mathematical modeling of natural images. Recently, many works have mitigated this difficulty by (exhaustively) training deep neural networks to learn the inverse mappings of given observation models. However, these methods suffer from a huge performance drop when the observation model used in training is inexact. In this talk, I focus on a promising line of work that uses deep learning models, such as CNN denoisers and GANs, for handling only the prior in inverse problems, and is therefore not restricted by assumptions made in the training phase. Our contributions include a back-projection (BP) fidelity term, which is an alternative for the traditional least squares (LS) objective. Using the simple proximal gradient method with the BP term and off-the-shelf denoisers (a scheme that we term IDBP) gives excellent results, requires less parameter tuning than LS-based methods, and is accompanied with theoretical motivations. Another contribution is an image-adaptive approach, where we tune CNN denoisers or GANs in test-time to specialize them on the image at hand. This approach leads to a significant performance boost, especially for GANs which often suffer from limited representation capabilities (known in the literature also as mode collapse).