This session we will discuss three recent papers which focus on self-supervised learning.

EVERYONE SHOULD TAKE THE TIME TO READ AT LEAST ONE OF THE PAPERS IN DETAIL SEVERAL DAYS BEFORE THE MEETUP (they are not in any particular order so focus on the ones that most interest you). Ideally, read two in detail and skim the rest. Since this week was posted late, we've chosen papers that have YouTube video introductions (posted below) which might make things a lot easier for most.

This is an online session. Details will be available to previous attendees in the DLSG Slack channel. Additional attendees by invitation. If you are completely new to the group or not from the Bay Area, please ping me on LinkedIn first - https://www.linkedin.com/in/jeffcoggshall/

We will focus on but not limit ourselves to the following papers:

** SCAN: Learning to Classify Images without Labels **
paper link: https://arxiv.org/abs/2005.12320

** A critical analysis of self-supervision, or what we can learn from a single image **
paper link: https://arxiv.org/abs/1904.13132

** Big Self-Supervised Models are Strong Semi-Supervised Learners (SimCLRv2) **
paper link: https://arxiv.org/abs/2006.10029

KEY PREREQUISITE REFERENCES

A Simple Framework for Contrastive Learning of Visual Representations (SimCLR v1)
link: https://arxiv.org/abs/2002.05709

Deep Clustering for Unsupervised Learning of Visual Features
link: https://arxiv.org/abs/1807.05520

OPTIONAL READING

** Debiased Contrastive Learning **
paper link: https://arxiv.org/abs/2007.00224

Additional Resources

• A Visual Exploration of DeepCluster - https://amitness.com/2020/04/deepcluster/
• Yannic Kilcher's video on SimCLRv2: https://www.youtube.com/watch?v=2lkUNDZld-4
• Yannic Kilcher's video on Learning to Classify Images without Labels: https://www.youtube.com/watch?v=hQEnzdLkPj4
• Yannic Kilcher's video on A critical analysis of self-supervision, or what we can learn from a single image https://www.youtube.com/watch?v=l5he9JNJqHA

Background & Key References