[webMeetup] How to Pretrain Your Transformer

Join the next NLP Zurich tech shindig online! Lewis Tunstall (Machine Learning Engineer @Huggingface) and Leandro von Werra (Machine Learning Engineer @Huggingface) will give us an overview of one of the chapters of their new book “Natural Language Processing with Transformers”.

The meeting will find place via Zoom (link will be added soon).

Agenda:
17:55 Join the Zoom Meeting
18:00 Lewis Tunstall (ML Engineer@Huggingface) and Leandro von Werra (ML Engineer@Huggingface): How to Pretrain Your Transformer
18:35 Q&A
18:50 Virtual Hugs and Kisses ⊂(◉‿◉)つ

Talk Summary:
In this session, Leandro von Werra and Lewis Tunstall will discuss their new O'Reilly book "Natural Language Processing with Transformers" and highlight some of the techniques and challenges that arise when training these models from scratch. A focus of this session will be on the creation of CodeParrot - a large GPT-2 model that is trained on (almost) all of the Python source code on GitHub!

They will ll also be giving away 3 electronic copies of their book. Details on the raffle will be provided at the event.

About the speakers:
Lewis Tunstall is a Machine Learning Engineer at Hugging Face, focused on developing open-source tools and making them accessible to the wider community. A former theoretical physicist, he has over 10 years experience translating complex subject matter to lay audiences and has taught machine learning to university students at both the graduate and undergraduate levels.

Leandro von Werra is a Machine Learning Engineer at Hugging Face. He has years of industry experience bringing NLP projects to production by working across the whole machine learning stack, and is the creator of a popular Python library that combines Transformers with reinforcement learning.

NLP Zurich:
Meetup.com, https://www.meetup.com/NLP-Zurich/
Linkedin, https://www.linkedin.com/company/nlp-zurich
YouTube, https://www.youtube.com/channel/UCLLX-5j9UNYassOwS0nveDQ/featured
Twitter, https://twitter.com/nlp_zurich