Skip to content

Details

🆂🅻🅾🆃 #1: Contrastive Representation Learning: A Framework and Review - Phúc Lê

Contrastive Learning has recently received interest due to its success in self-supervised representation learning in the computer vision domain. However, the origins of Contrastive Learning date as far back as the 1990s and its development has spanned across many fields and domains including Metric Learning and natural language processing. In this paper we provide a comprehensive literature review and we propose a general Contrastive Representation Learning framework that simplifies and unifies many different contrastive learning methods. We also provide a taxonomy for each of the components of contrastive learning in order to summarise it and distinguish it from other forms of machine learning. We then discuss the inductive biases which are present in any contrastive learning system and we analyse our framework under different views from various sub-fields of Machine Learning. Examples of how contrastive learning has been applied in computer vision, natural language processing, audio processing, and others, as well as in Reinforcement Learning are also presented. Finally, we discuss the challenges and some of the most promising future research directions ahead.

👨🏻‍🔬 PHUC H. LE-KHAC is currently pursuing the Ph.D. degree with ML-Labs, Dublin City University. Since 2018, he has been a Deep Learning
Engineer. His major interest is in representation learning, with a focus on learning representation from video from uncurated sources without human annotations.

📌As always, we are super happy to have you with a lightning talk on your small or big successes, open source projects or just something amazing you would like to share.

📌Do not miss out updates on twitter - https://twitter.com/hack_ai and join Munich ML community on Slack channel https://ai-hack-inviter.herokuapp.com/

You may also like