From One GPU to Many: Understanding Distributed Training with PyTorch DDP

In this presentation Demdah will present us how to do training on multiple GPU.

Resume:

We will discuss the fundamental concepts of distributed deep learning, including data parallelism, ranks, collective communication, gradient synchronization, and fault recovery. These concepts will be illustrated through a real-world DDP experiment in PyTorch on two nodes, using WSL2, NCCL, heterogeneous NVIDIA GPUs, and training a ResNet18 model on the CIFAR-10 dataset.

The presentation will be held at the grande bibliotheque

475 Boul. de Maisonneuve E, Montréal, QC H2L 5C4
room 2.130.1
The goal of this "informal" event is to learn about AI and discuss it together with the host.