Open Source LLM Model Reach 1 Trillion Parameters

Finally open-sourced LLM model had reached 1 Trillion parameters. We will look at the performance and training of Kimi K2. We will look at their super stable training phase with the new optimiser MuonClip.

Paper: Kimi K2