AI Study Group @ Block: Andrej Karpathy's Zero to GPT Hero
Details
***
Please enrol in the study group via the following link:
https://forms.gle/L4u3TMfTs5TjqWpt7
***
The GPT phenomenon is largely responsible for putting AI on the map. However many Developers & Data Scientists largely do not understand how the technology works on a deep level.
100% free, for ~2 hours in the evening, every Wednesday at 6pm, over the course of 4 weeks we will be working through Andrej Karpathy's build your own GPT course from scratch as a group
The Youtube materials shall act as the “textbook” content that students have to follow. The GPT Study group will however provide the missing techniques used by educational institutions to motivate students to the end of a syllabus including collaboration, tutors!,
Guest speakers and many other methods to keep students motivated towards the end of the course material.
For those looking for a refresh or beginners, we will be going over the fundamentals all the way through to building out your own GPT from first principles.
Attendees of the study group can expect to learn:
- How neural networks & back propagation work
- The basics of how language modelling works
- The makemore neural net
- Bigram language models to learn how next token prediction works
- MLPs
- Limitations of Bigram models
- Multilayer Neural Nets
- How Activations, Gradients & Batchnorm work
- understanding the activations and gradients in a neural network during training
- Recurrent neural networks
- Batch normalization
- How Wavenet works
- Defining the building blocks of a Wavenet including linear layers, convolutional layers, and batch normalisation layers
- Implement a way to concatenate multiple characters into a single character embedding
- How GPT works
- Explanation of the Transformer Architecture
- Training a transformer on tiny Shakespeare data
- How the GPT tokensizer works.
- different algorithms for tokenization, eg byte pair encoding (BPE) etc
- Tokenization hyperparameter optimisation
What we’re looking for:
You are comfortable with basic linear algebra. It’s fine if you have to brush up on these skills before course commencement.
You are comfortable programming in Python (other languages are helpful, but you’ll spend the time writing Python).
You are comfortable with Git & basic command line calls. eg git push.
We’ll use these criteria for selection:
Impact on you. We want to understand why this course will help you achieve something you couldn’t otherwise.
Self-motivation & communication. We’re looking for people who will work hard through those 4 weeks, and who will inspire others (in the course and externally) to endeavour to learn how modern LLMs work.
More information will be emailed out to those who are successful in their application to the course,
Best of luck & regards,
Paul & Bo
~~~
Enrol via the link below: