Skip to content

Details

PLEASE REGISTER FOR THE ZOOM MEETING IN THE RIGHT COLUMN!

Our second lunch-and-learn session will be covering the fundamentals of distributed training! Join us on April 13 for lunch on us and a hands-on tutorial from Armand McQueen, a Senior Software Engineer at Determined.

Overview for the session:

Distributed training is an extremely valuable tool for speeding up the training of deep learning models. However, it isn't used as frequently as it could be. This hands-on tutorial will start by going over how distributed training works, covering the core concepts and terminology. Then we will move into an interactive session where we take a specific model and walk through how to speed it up by an order of magnitude through distributed training. This part will focus on practical tips and tricks that can be applied to any model. It will teach you about some of the fundamental problems in distributed training (e.g. large batch size convergence, scaling inefficiencies) and how you might think about (or avoid!) those problems when training your own model.

Goals for the session:

  • Establish a solid baseline of understanding so that all attendees can have meaningful discussions about distributed training.
  • Hands-on demonstration of how to take a model that trains on one GPU in days and use distributed training to train it on many GPUs in minutes.
  • Come away with an understanding of the steps needed to do distributed training with your model and how to approach the unavoidable problems in distributed training.

Want to attend? Follow these steps:

  1. Register for the meetup
  2. Click on the Zoom link and register in advance of the session!

After the lunch-and-learn session has concluded, Determined staff will be staying on the Zoom to answer questions and offer support, as a part of our ongoing community syncs. Bring your challenges with distributed training or just general questions about deep learning - we'd love to help!

Community sync: https://www.meetup.com/determined-ai-meetup/events/277310166/

As we get closer, we will be emailing everybody with housekeeping items and details on (free!) lunch for the session, so please register for the Zoom. Thanks!

You may also like