Marco Ribeiro | Beyond Accuracy: Behavioral Testing of NLP Models with CheckList

Virtual London Machine Learning Meetup - 04.11.20 @ 18:30

We would like to invite you to our next Virtual Machine Learning Meetup. Please read the papers below and help us create a vibrant discussion.

The discussion will be facilitated by Sebastian Riedel, currently a Researcher at Facebook AI Research. He is also a Professor at University College London as a proud member of the UCL NLP lab, and an Allen Distinguished Investigator.

Agenda:

• 18:25: Virtual doors open
• 18:30: Talk
• 19:10: Q&A session
• 19:30: Close

Sponsors
https://evolution.ai/ : Machines that Read - Intelligent data extraction from corporate and financial documents.

• Title: Beyond Accuracy: Behavioral Testing of NLP Models with CheckList (Marco Ribeiro is a Senior Researcher at Microsoft Research)
• Papers/Resources:
  https://homes.cs.washington.edu/~marcotcr/

Abstract: We will present CheckList, a task-agnostic methodology and tool for testing NLP models inspired by principles of behavioral testing in software engineering.

We will show a lot of fun bugs we discovered with CheckList, both in commercial models (Microsoft, Amazon, Google) and research models (BERT, RoBERTA for sentiment analysis, QQP, SQuAD). We'll also present comparisons between CheckList and the status quo, in a case study at Microsoft and a user study with researchers and engineers. We show that CheckList is a really helpful process and tool for testing and finding bugs in NLP models, both for practitioners and researchers.

Bio: Marco Tulio Ribeiro is a Senior Researcher at Microsoft Research. His work is on facilitating the communication between humans and machine learning models, which includes interpretability, trust, debugging, feedback, robustness, testing, etc. He received his PhD from the University of Washington.