Deep Learning Architectures & Systems

Let's continue with our survey of developments since GPT-3.

Useful starting points:

• A Visual Tour of LLM of Modern LLM Architectures (youtube 38min)
• A slop summary of Reiner Pope's interview on how LLMs are actually trained and served. (Has some useful diagrams)
• The Deepseek family of models are the most well documented of the lot for the last two years.

Informal, all levels welcome. Suggestions for future papers and topics welcome.