Deep Learning Meetup #13

Dear DeepLearners,

Our next Meetup is scheduled on Wednesday March 7 at 7:00 pm. This meetup is a little special, since it is a bit more focused into research than usual... To those afraid of math, beware! ---------------------------

Olivier Grisel (ML Expert - Software Engineer at INRIA) - Generalization in Deep networks

Abstract: This talk will give an overview to some recent theoretical results and experiments on why deep learning models work so well (when they work). In particular we will discuss expressive power, optimization and generalization and their interaction. We will illustrate some of the main insights with empirical experiments.

-----------------------------

Arthur Mensch (PhD Candidate at INRIA Parietal) - Differentiable Dynamic Programming for Structured Prediction and Attention

Abstract: Dynamic programming (DP) solves a variety of structured combinatorial problems by iteratively breaking them down into smaller subproblems. In spite of their versatility, DP algorithms are usually non-differentiable, which hampers their use as a layer in neural networks trained by backpropagation. To address this issue, we propose to smooth the max operator in the dynamic programming recursion, using a strongly convex regularizer. This allows to relax both the optimal value and solution of the original combinatorial problem, and turns a broad class of DP algorithms into differentiable operators. Theoretically, we provide a new probabilistic perspective on backpropagating through these DP operators, and relate them to inference in graphical models. We derive two particular instantiations of our framework, a smoothed Viterbi algorithm for sequence prediction and a smoothed DTW algorithm for time-series alignment. We showcase these instantiations on two structured prediction tasks and on structured and sparse attention for neural machine translation.

-----------------------------

Diogo Luvizon (PhD Candidate at ETIS - Université de Cergy) - 2D/3D Pose Estimation and Action Recognition using Multitask Deep Learning

Abstract: Action recognition and human pose estimation are closely related tasks since both problems depend on the human body understanding and additionally, action recognition benefits from precise estimated poses. Despite that, both problems are generally handled as distinct tasks in the literature. In this work, we propose a multitask framework for jointly 2D and 3D pose estimation from still images and human action recognition from video sequences. We show that a single architecture can be used to solve the two problems in an efficient way and still achieves state-of-the-art results. Additionally, we demonstrate that optimization from end-to-end leads to significantly higher accuracy than separated learning. The proposed architecture can be trained with data from different categories simultaneously in a seamlessly way. The reported results on four datasets (MPII, Human3.6M, Penn Action and NTU) demonstrate the effectiveness of our method on the targeted tasks.

---------------------------

As usual, pizzas and drinks will be provided after the talks.

Heuritech Meetup Team.