Build a Large Language Model (From Scratch)
Details
We are going through Hands-On Build a Large Language Models (from Scratch) by Sebastian Raschka.
The emphasis during the meetups will be to discuss key aspect of the Chapter being covering. Code focused discussions should be done over Discord.
Raschka provides a step-by-step guide coding up your own foundation LLM ground up, spanning initial design and creation stages, to pretraining on a general corpus, and on to fine-tuning for specific tasks.
Pages being discussed:
Please see the latest message (also pinned) in the #current-reading channel in our Discord chat space to see which pages we'll be reviewing in this session.
Please note that the session is not recorded and participants are responsible for obtaining their own copy of the text.
Discord joining instructions:
Buy the book (affiliate links):
Book overview:
In Build a Large Language Model (from Scratch) bestselling author Sebastian Raschka guides you step by step through creating your own LLM. Each stage is explained with clear text, diagrams, and examples. You'll go from the initial design and creation, to pretraining on a general corpus, and on to fine-tuning for specific tasks.
Build a Large Language Model (from Scratch) teaches you how to:
- Plan and code all the parts of an LLM
- Prepare a dataset suitable for LLM training
- Fine-tune LLMs for text classification and with your own data
- Use human feedback to ensure your LLM follows instructions
- Load pretrained weights into an LLM
Build a Large Language Model (From Scratch)