Feature Selection with Deep Neural Networks - Dr. Ofir Lindenbaum

The talk is based on the paper: “Feature Selection using Stochastic Gates” recently published at ICML 2020. In this talk, Ofir, the paper's author, will present a solution for using NNs for feature selection. Feature selection is an important problem in machine learning, and it can lead to several benefits, such as interpretability, reduced overfitting, and computational complexity. He will explain the derivation of the and demonstrate its use with several examples.

git: https://github.com/runopti/stg
arxiv: https://arxiv.org/abs/1810.04247

Lecture abstract:

Feature selection problems have been extensively studied in the setting of linear estimation (e.g. LASSO), but less emphasis has been placed on feature selection for non-linear functions. In this study, we propose a method for feature selection in neural network estimation problems. The new procedure is based on probabilistic relaxation of the L0 norm of features or the count of the number of selected features. Our L0-based regularization relies on a continuous relaxation of the Bernoulli distribution; such relaxation allows our model to learn the parameters of the approximate Bernoulli distributions via gradient descent. The proposed framework simultaneously learns either a nonlinear regression or classification function while selecting a small subset of features. We provide an information-theoretic justification for incorporating Bernoulli distribution into feature selection. Furthermore, we evaluate our method using synthetic and real-life data to demonstrate that our approach outperforms other commonly used methods in both predictive performance and feature selection

Presenter BIO:

Ofir Lindenbaum is a postdoc fellow at Yale University at the applied math department, working with Prof. Ronald Coifman and Prof. Yuval Kluger. His primary research is in the field of machine learning and computational biology. He pursues research on signal processing, music and audio analysis, manifold learning, spectral methods for data mining, and dimensionality reduction.

His Website: https://ofirlin.wixsite.com/ofirlindenbaum

This is a technical talk, prior knowledge of deep learning is advised.

** ** Please register through the zoom link right after your RSVP. We will send the links to the zoom event via email only to those who have registered through zoom. ** **

-------------------------
Find us at:

All lectures are uploaded to our Youtube channel ➜ https://www.youtube.com/channel/UCHObHaxTXKFyI_EI8HiQ5xw

Newsletter for updates about more events ➜ http://eepurl.com/gJ1t-D

Sub-reddit for discussions ➜ https://www.reddit.com/r/2D3DAI/

Discord server for, well, discord ➜ https://discord.gg/MZuWSjF

Blog ➜ https://2d3d.ai