Build LLMs From Scratch - AiA
Details
Join our event-series to learn how to build Large Language Models (LLMs) from scratch.
This is our FIRST meeting.
***
Have you used LLMs but remain curious/dubious about how they work?
You want to learn about LLMs but find it overwhelming?
You see the potentials of LLMs but don't know where to start?
Join us to build your LLM from scratch! Learn by doing together with a supportive community.
Full details: check out this page
***
## What
This is a hands-on learning bootcamp focusing on Large Language Models (LLMs), spanning several months. You will learn to design, pretrain, and fine-tune your own GPT-like model.
## Who
The program is suitable for people with a quantitative background. Working knowledge of Python, Pytorch, and Machine Learning is a plus but not mandatory.
Two main prerequisites are:
- be comfortable with computers & maths (bachelor level)
- willing to commit ~40h/month to focused learning
## How
- Study: read/watch materials, run the code, and solve the exercises
- Research: read papers, explore and experiment, try to break things
- Group meeting: monthly meetings, summarize key insights, Q&A
- Discord: discuss about anything, share resources, ask questions
## Wanna join?
IMPORTANT:
- Check out [program repo on Github]
- Complete the [Meeting-0 materials]
- Please fill [Google Form] to join WhatsApp & Discord group
- Logistics: meetings are in-person, location & time will be updated in Discord & Meetup
See you there! Dan, Mark & Hal
***
# Program & Schedule
We plan to meet every 3/4 weeks, around the last week of the month.
- Month 0 Getting started - Mon, Nov 4
- Month 1 Tokenization & Embeddings - Mon, Nov 25
- Month 2 Project: build your own tokenizer - Mon, Jan 6
- Month 3 Attention Mechanisms - Mon, Jan 27
- Month 4 Transformer & GPT Architecture - Mon, Feb 24
- Month 5 Pretraining LLMs - Mon, Mar 24
- Month 6 Fine-tuning LLMs - Mon, Apr 21
- Month 7 Final Project: build your own GPT-2 - Mon, May 26
The first 3 months have optional materials allowing for everyone to acquire the fundamental knowledge required to build LLMs.
***
## About Us
Our goal is to democratize Machine Learning and AI. We experiment with hands-on projects on LLMs like RAGs, quantization, and other real-life applications. We believe that it does not matter who you are, where you come from, you can build and contribute to shaping the future with better and safer AI technologies.
***
## FAQs
1.Is there any fee?
No fee, no hidden cost, aside from the textbook. If you can't afford it, reach out to us. Most materials are publicly available.
2.What will I learn after completing the program?
At minimum, you'll gain a much deeper understanding of LLMs than from YouTube and blog-posts. At best, with the right resources, you'll spin out new custom LLMs every month.
3.How can I join the Discord and WhatsApp group?
Fill out the form in the event description to receive invitations to join the groups.
4.What kind of support is available?
Support is available in real-time on Discord. We may organize co-working days on top of the monthly meeting.
5.What if I miss a meeting?
Meeting discussions and resources will be available on Discord and GitHub to help you catch up.
6.Can I join after the series has started?
It's best to join from the start, but you can join later if you have the required time/knowledge to catch up.
Build LLMs From Scratch - AiA