New feature representations for image retrieval and semantic matching

Bonsoir à tous,

Nous profitons de ce lendemain de Noël pour vous faire part du prochain meetup ML à venir! Pour bien commencer l'année, nous allons recevoir le 15 janvier prochain Rafael Sampaio de Rezende qui nous parlera de nouvelles méthodes pour la reconnaissance d'image et les correspondances sémantiques. Vous trouverez un résumé de sa présentation ci-après.

En attendant, nous vous souhaitons de belles fêtes de fin d'année.

A l'année prochaine!

Résumé: The problem of image representation is at the heart of computer vision. The choice of feature extracted of an image changes according to the task we want to study. Large image retrieval databases demand a compressed global vector representing each image, whereas a semantic segmentation problem requires a clustering map of its pixels. The techniques of machine learning are the main tool used for the construction of these representations. In this presentation, we address the learning of visual features for two distinct problems: Image retrieval and semantic correspondence.

Our first approach proposes an extension to the exemplar SVM feature encoding pipeline. We first show that, by replacing the hinge loss by the square loss in the ESVM cost function, similar results in image retrieval can be obtained at a fraction of the computational cost. We call this model square-loss exemplar machine, or SLEM. Secondly, we introduce a kernelized SLEM variant which benefits from the same computational advantages but displays improved performance. We present experiments that establish the performance and efficiency of our methods using a large array of base feature representations and standard image retrieval datasets.
In the second half, we propose a deep neural network for the problem of establishing semantic correspondence. We employ object proposal boxes as elements for matching and construct an architecture that simultaneously learns the appearance representation and geometric consistency. We propose new geometrical consistency scores tailored to the neural network’s architecture. Our model is trained on image pairs obtained from key points of a benchmark dataset and evaluated on several standard datasets, outperforming both recent deep learning architectures and previous methods based on hand-crafted features.