DataTalks #30: Model Distillation 🧠⚗️

DataTalks #30: Model Distillation 🧠⚗️

Our 30th DataTalks meetup will be held online in cooperation with Windward, and will focus on model distillation.

𝗭𝗼𝗼𝗺 𝗹𝗶𝗻𝗸: https://us02web.zoom.us/j/87493432159?pwd=bjlLSk1GaGZ2VWFBQVdLUkk3VzYvZz09

𝗔𝗴𝗲𝗻𝗱𝗮:
🔶 16 :30 - 17:15 – Model Distillation for Object Detection – Shani Gamrian
🔴 17:20 - 18:05 – Distilling Maritime Insights with Deep Learning – Gilad Landau, Windward

---------------------

𝗠𝗼𝗱𝗲𝗹 𝗗𝗶𝘀𝘁𝗶𝗹𝗹𝗮𝘁𝗶𝗼𝗻 𝗳𝗼𝗿 𝗢𝗯𝗷𝗲𝗰𝘁 𝗗𝗲𝘁𝗲𝗰𝘁𝗶𝗼𝗻 – 𝗦𝗵𝗮𝗻𝗶 𝗚𝗮𝗺𝗿𝗶𝗮𝗻

Object Detection networks are commonly used on many applications and products nowadays and are capable of achieving very high performances in different real-life scenarios. However, when implementing these networks on limited resources, real-time solutions are required.

Model Distillation refers to the idea of model compression by teaching a smaller network, how to behave using a bigger, pre-trained network. There are two types of knowledge representations that can be transferred from teacher to student. The first is knowledge from direct outputs (also known as Knowledge Distillation) and the second is knowledge transferred from intermediate layers. In this talk, we will discuss the ideas and approaches of both types and the differences between them. We will also cover recent distillation works and solutions designed specifically for object detection networks such as SSD and FPN that show significant improvement of the results.

𝗣𝗮𝗽𝗲𝗿 𝗹𝗶𝗻𝗸: https://arxiv.org/abs/1906.03609

𝗕𝗶𝗼: Shani is an Applied Machine Learning and Computer Vision researcher at Brodmann17.

---------------------

𝗗𝗶𝘀𝘁𝗶𝗹𝗹𝗶𝗻𝗴 𝗠𝗮𝗿𝗶𝘁𝗶𝗺𝗲 𝗜𝗻𝘀𝗶𝗴𝗵𝘁𝘀 𝘄𝗶𝘁𝗵 𝗗𝗲𝗲𝗽 𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴 – 𝗚𝗶𝗹𝗮𝗱 𝗟𝗮𝗻𝗱𝗮𝘂, 𝗪𝗶𝗻𝗱𝘄𝗮𝗿𝗱

I will present Windward's process of developing and deploying a deep learning pipeline in the maritime domain. The lecture will focus on the real-world challenges of training a deep learning model with a small amount of labeled data by utilizing distillation and active learning techniques.

𝗣𝗮𝗽𝗲𝗿 𝗹𝗶𝗻𝗸:
https://arxiv.org/abs/1503.02531
https://arxiv.org/abs/1711.00941
https://arxiv.org/abs/1609.03499

𝗕𝗶𝗼: Gilad is a Technologist and a Senior Data Scientist Windward. He is enthusiastic about creating real business value with Deep Learning.

---------------------

𝗭𝗼𝗼𝗺 𝗹𝗶𝗻𝗸: https://us02web.zoom.us/j/87493432159?pwd=bjlLSk1GaGZ2VWFBQVdLUkk3VzYvZz09