LLM Architecture in 2026 - DeepSeek Technical Report
Details
We'll take a first pass at the DeepSeek Technical Reports
2024-2025 up through V3 and R1 and maybe even this year's V4. These are excellent windows into frontier LLMs architecture and training.
Before we dive deep on one feature, let's do a survey and pull out the vocab from what has changed since vanilla transformers and GPT-3 chattiness. The variety is dizzying, from low level details like attention variants to post training with RLVR, but starting with DeepSeek publications should give us plenty to talk about.
The setting is informal and open to everyone!
DLRTP is back with weekly Wednesday lunchtime sessions at Frontier RTP.
Suggestions for future papers and topics welcome.
Find us in the "Eagles" conference room that is attached to the main Coworking space.
A Zoom link will be included in the comments at least 30 minutes before the event.