A Brief Overview of the Success Story of Large Language Models

In our next Meetup, we are happy to host Timo Denk in a joint event with Freiburg AI. Timo is an Applied Scientist at Zalando and presents some recent advances in NLP. For more information about Timo, please check out his website https://timodenk.com/ and for more information about this Meetup have a look at https://freiburg.ai/.

Abstract:
BERT here, GPT there, [you name it] is all you need... Self-supervised, large-scale language models have been vastly successful in recent years. Today's top scorers on NLP benchmarks are almost exclusively Transformer-based architectures. This DSC talk will give a brief overview of the success story of large language models. We will look into the most prominent papers that are built on top of the Transformer, discuss current research directions, and develop an intuition for why ultra-large language models work so exceptionally well.