2025-08: Building Transformers from Scratch — and What’s Next

Ever wondered what really happens under the hood of your favorite LLM like ChatGPT or DeepSeek? Join Jen Wei for a hands-on journey through Transformer architecture: learn how it works, why it matters, and where the latest innovations are taking us.

Where: Platform Calgary, East Annex
When: Wednesday, August 27, at 5:30pm

Transformers are the foundation of today’s AI models, but most of us only interact with them through APIs. In this talk, Jen will walk us through a step-by-step implementation of the original Transformer architecture (“Attention Is All You Need”), highlighting how each component works and how they fit together.

She’ll also introduce some recent innovations from DeepSeek — such as Multi-head Latent Attention and Mixture of Experts (MoE) — and explain why these changes matter for scaling and inference efficiency. Whether you’re a data scientist, ML engineer, or just curious about what happens under the hood, this talk will give you both a hands-on view of the basics and a glimpse of the cutting edge.

You can find the "build a transformer from scratch tutorial" here
🤗 https://huggingface.co/datasets/bird-of-paradise/transformer-from-scratch-tutorial

Schedule:
5:30 - Food and Networking
6:00 - Presentation and Discussion
7:30 - Wrap up

Speaker Bio:
Jen Wei is an independent researcher and open-source contributor focused on foundational AI. She maintains a series of Hugging Face tutorials on implementing Transformer architectures and related innovations from scratch, with an emphasis on clarity, testing, and reproducibility. Her goal is to make advanced AI concepts more accessible while encouraging curiosity about the “why” behind the models.

Hugging Face: https://huggingface.co/bird-of-paradise