DIR 2021 ½: Week 4

DIR 2021 ½ has teamed up with SEA and we are proud to be organizing DIR 2021 ½ on four consecutive Fridays. This fourth and final day of DIR 2021 ½ features a keynote by Gabriella Kazai. Details to be announced.

The fourth and final week of DIR 2021 ½ will consist of a keynote talk given by Gabriella Kazai from Microsoft.

Gabriella Kazai (Microsoft)
Title: IR Evaluation - An Industry Perspective

Abstract: Measurement is key in the development of any product or service, including information retrieval (IR) systems. The challenge of developing IR metrics from which reliable conclusions can be drawn is no trivial matter. For example, issues in sampling, instrumentation, label collection or metric computation methodologies can invalidate an evaluation experiment or even lead to problems in the training of machine learning systems. In this talk I will reflect on some of the challenges when developing IR metrics and building offline evaluation pipelines at Bing.

Bio: I am a Principal Applied Scientist at Microsoft, tackling a range of IR evaluation challenges as part of the Bing Web and AI Science Group, focusing on offline evaluation, crowdsourcing, and metric development for various search scenarios, including organic web search, news, autosuggestions, and covering aspects from relevance to source credibility, trust and bias. Prior to this, I worked at two startups: As Lead Data Scientist at Mudano Ltd, I worked on an AI-driven project management system to deliver large-scale IT change projects in the financial sector; and as VP of Data Science at Semion Ltd, I led a team of data scientists implementing the AI behind the Lumi news recommender app, providing out-of-the-box personalised media content to users based on interests discovered from their Twitter streams. Before that, I was a postdoc at Microsoft Research after obtaining my PhD from the University of London under the supervision of Prof Mounia Lalmas. My research interests include information retrieval (IR), IR evaluation, human computation, gamification, recommender systems, social IR, information seeking behaviour, activity based personal information management.

---
For those of you who are not tired of counting yet. This is talk #200 hosted by SEA.