Extreme Few-Shot Classification to Achieve Near-SOTA Performance

Booking.com Machine Learning Webinar Series is all about bringing our extensive ML community knowledge to ML practitioners around the world.
We invite Booking.com brightest ML scientists and engineers to share their experience and insights on a wide variety of topics across Booking's ML ecosystem.

Agenda

Extreme Few-Shot Classification to Achieve Near-SOTA Performance
Manos Stergiadis - Senior Machine Learning Scientist, Booking.com
Satendra Kumar - Senior Machine Learning Scientist, Booking.com
Ilya Gusev - Senior Machine Learning Engineer, Booking.com

AI is undergoing a paradigm shift. Deep learning models like GPT-3, BERT, and CLIP now demonstrate much better understanding of text and images, even enabling impressive Zero-Shot performance on a wide range of downstream tasks (e.g. image classification) and domains (e.g. travel, medical).
However, in most real-world settings, practitioners still often need to turn to Fine-Tuning on top of these foundational models to achieve acceptable performance for their specific task & domain.
Whilst well established, Fine-Tuning for each task still scales poorly with respect to the number of tasks. Each task still needs its own labelled data (often human-annotated), which costs time and money. At Booking.com, we currently need to classify an image into at least 120 tags (‘bedroom’, ‘dishwasher’, ‘sauna’, ‘barbeque’). And in the ever-evolving world of travel, this list is also ever-evolving.
In this talk, we share some nuts and bolts of our multimodal algorithm to address the above pitfalls. Versus vanilla Fine Tuning on large human-annotated data, we demonstrate that our algorithm achieves near-SOTA using as few as 5 Positive labelled examples across these 120 tasks. This can cut costs by 2 orders of magnitude.