Linear Algebra for AI Engineers: From Vectors to Transformers
Details
This is practical hands-on and does not require any laptops. Please bring your books and pens.
This workshop provides a focused but deep introduction to the linear algebra required to understand modern machine learning and AI research papers. Rather than treating linear algebra as an abstract mathematics subject, we will build intuition around how vectors, matrices, transformations, and high-dimensional spaces are actually used inside ML models.
At a high level, we will cover vectors and embeddings, matrices and matrix multiplication, vector spaces, basis and rank, dot products and similarity, orthogonality and projections, eigenvalues and eigenvectors, SVD, PCA, and low-rank representations.
We will then connect these concepts directly to modern AI architectures, including neural-network weight matrices, embeddings, attention, Q/K/V projections, multi-head attention, LoRA, and compressed representations such as DeepSeek's MLA.
The final goal of the workshop is not to make participants linear-algebra mathematicians, but to make them comfortable looking at equations such as:
Y=XW
QK^T
A=UΣV^T
ΔW=BA
and
Attention(Q,K,V)=softmax(QK^T/sqrt(dk))V
and understanding what the objects represent, why their dimensions matter, what mathematical operation is occurring, and what that operation means inside a machine-learning model.
### By the end of the meetup
Participants should have a strong enough working understanding of linear algebra to follow the mathematical notation and matrix operations used in modern ML and LLM papers, including papers on Transformers, attention mechanisms, model compression, LoRA, PCA/SVD, and architectures such as DeepSeek.