Benchmarking Self Supervised Learning Methods

This talk will cover a brief introduction to the various families of Self Supervised Learning methods such as Masked Language Modelling for training LLMs and DINO, VICReg and SwAV for vision. We'll broadly compare these families of methods and understand their similarities and key differences. We shall also explore the difficulties which emerge while implementing these methods and release open source implementations in Tensorflow and Keras along with pretrained models and training logs.