LLM Architecture in 2026 - DeepSeek Technical Report

DLRTP is back with weekly Wednesday lunchtime sessions at Frontier RTP.

Let's take a first pass at the DeepSeek Technical Reports
2024-2025 up through V3 and R1 and maybe even this year's V4. These are excellent windows into the current batch of LLMs.

Before we dive deep on one feature, let's do a survey and pull out the vocab from what has changed since vanilla transformers and GPT-3 chattiness. The variety is dizzying, from low level details like attention variants to post training with RLVR, but starting with DeepSeek publications should give us plenty to talk about.

The setting is informal and open to everyone!

Suggestions for future papers and topics welcome.

A Zoom link will be included in the comments at least 30 minutes before the event.