Group Study building LLMs at scale, Stanford cs336

Incredible material on LLMs. Quite possibly the best hands on class in the last 20y. Relevant to current job market with state of the art assignments. Riveting lectures.

First meetup: we are going to prequalify people and see if they are qualified to do the work. If nobody has pytorch experience we are going to cancel the remaining events.

The instructors have modified the course to make it open for self study.

All of the lectures are on YT
Course website

The assignments and pacing are quite difficult. Most of these are well beyond the range of the typical CS undergraduate.

The homeworks are substantial and require background and skills most people on meetup don't have so we are going to try something easier. You are free to form a group and do the hw assignments without looking at the solutions.

New format, going to try to review individual slides or demos as we review the cs336 LLM videos.

The format is participants view the lecture and come with a slide or 3 to discuss a particular topic or demo running code or progress towards the assignments.

As we progress over the course of 2 months we can separate people into working groups to make either summary slides to summarize key points or do code demos.

We can cover each lecture and see how far we get in 2months. These are the first 6 lectures.
cs336 Lecture 1 Tokenization
cs336 Lecture 2 Pytorch and Resource Accounting
cs336 Lecture 3 arch and hyperparams
cs336 lecture 4 MoE
cs336 Lecture 5 GPUs
cs336 Lecture 6 Kernels and Triton