Anna Rogers | BERTology nuggets: what we have learned about how BERT works

Virtual London Machine Learning Meetup - 15.12.2021 @ 18:30

We would like to invite you to our next Virtual Machine Learning Meetup.

Agenda:

• 18:25: Virtual doors open
• 18:30: Talk
• 19:10: Q&A session
• 19:30: Close

Sponsors
https://evolution.ai/ : Machines that Read - Intelligent data extraction from corporate and financial documents.

• Title: BERTology nuggets: what we have learned about how BERT works (Anna Rogers is a post-doctoral researcher scientist at the University of Copenhagen)

Abstract: BERT and its Transformer-based cousins are at the core of most state-of-the-art NLP systems, yet it is still unclear why they work so well, and what could be our next step. This talk presents a collection of findings on how BERT learns and generalizes, where it fails, and whether the lottery ticket hypothesis holds for this architecture.

Bio: Anna Rogers is a post-doctoral researcher at the University of Copenhagen, focusing on analysis and evaluation of deep learning models for NLP, as well as their impact on society. She is also known for her work on NLP methodology, peer review, and as an organizer of the Workshop on Insights from Negative Results.

The discussion will be facilitated by Alessandro Raganato. Alessandro is a post-doctoral researcher at the University of Helsinki, working on Natural Language Processing (NLP) with a focus on crosslingual applications. Current research interests include Lexical Semantics, specifically Word Sense Disambiguation, and Neural Machine Translation.