From fast moving objects to video compression (Filip Sroubek, Jan Kotera)

Speakers:
Doc. Filip Šroubek and Dr. Jan Kotera
(Dept. of Image Processing, UTIA, Czech Academy of Sciences)

Abstract:
The talk consists of two parts that show applications of neural networks, particularly autoencoders, in practical problems of image and video processing. In the first part, we will summarize results of a successful GACR project jointly investigated by UTIA and CVUT that have unveiled intricate world of fast moving objects. Tracking objects that move with high speed is a challenging problem on which state-of-the-art methods frequently fail. Fast moving objects are excessively blurred and appear as semi-transparent streaks. Such phenomena are common in sports videos, yet we encounter fast moving objects also in various physical and biological experiments. Moving objects photographed in front of a static background are blurred and blend with the background. Estimating the object shape leads to two intertwined inverse problems of blind deblurring (both the object and blur are unknown) and image matting (separating foreground from background), which we call deblatting. We rigorously tackle both inverse problems simultaneously. From a single image and known background the proposed method estimates the object shape and appearance as well as object 2D motion blur and rotation. We will demonstrate a long-term tracker, in which deblatting plays a central role. The model-based approach to restoration can be fully replaced with the learning-based approach. Autoencoder neural net with novel loss functions can recover sharp objects even if trained only on synthetic data.
Image and video compression is a fundamental application of image processing used virtually every time you use your computer or cell phone. Many hand-designed and very well performing methods were developed in the last three decades and are now being used as standards. Quite recently, learned image compression became a very active research topic and the state-of-the-art methods can already in many ways compete with or even outperform traditional established standards while having several other advantages, such as for example fast and usually cheap domain adaptation. In the second part, we will introduce the general concepts and principles of image compression, present the base structure of learned image compression codec based on autoencoder neural net, and give overview of the techniques used by the latest state of the art methods as well as current challenges and research trends.