Etsy – Learning to rank with multimodal data
Details
In this talk Kamelia and Nishan from Etsy (https://www.etsy.com) are going to talk about Etsy's learning to rank production pipeline and challenges from two perspectives: (i) a single modality model using traditional text-based features, and (ii) a multimodal approach that includes both text and visual clues from Etsy listings.
Abstract
Search is an important problem for modern e-commerce platforms such as Etsy. As a result, the task of ranking search results automatically or the so-called learning to rank is a multibillion dollar machine learning problem.In this talk, we first review Etsy's approach to learning to rank using a few hand-constructed features based on the Etsy listing's text-based representation.
We then discuss a multimodal learning to rank model that combines these traditional text-based features with visual semantic features transferred from a deep convolutional neural network. We show that a multimodal approach to learning to rank can improve the quality of ranking in an experimental setting.
About the speakers
Kamelia Aryafar is a senior data scientist with Etsy's data science team since 2013. She works on building scalable machine learning and computer vision tools to curate a personalized experience for Etsy users. Prior to Etsy she was doing a Ph.D. in computer science in Drexel University, building large-scale music classification models.
Nishan Subedi is a senior software engineer with Etsy's search ranking team since 2014. Nishan works on improving ranking and relevance of search results and building tooling around this at Etsy. Prior to this he was an infrastructure wizard in Wayfair.