Fundamentals of Scaling LLM training

Vedant from Aleph Alpha will talk about Fundamentals of Scaling LLM training.

If you want to join this event, please sign up on our Luma page: https://lu.ma/l2sc9gt9
​⚠️ Registration is free, but required due to building security.

Abstract:
Scale is one key ingredient that has enabled the success of modern LLMs. In this talk I will go over different kinds of parallelisms (model, pipeline, data, sequence, expert) and how they compose with each other to enable training of models with extremely large number of parameters.

🔈 Speakers:

• Vedant Nanda

Agenda:
✨ 18:30 Doors open: time for networking with fellow attendees
✨ 19:00 Talk and Q&A
✨ 20:00 Mingling and networking with pizza and drinks
✨ 21:00 Meetup ends
- Where: In person, Aleph Alpha Heidelberg, Speyerer Str. 14
- When: Tuesday, July 22nd
- Language: English