[PDG 491] Evolution Strategies at the Hyperscale (Eggroll)

Link to article: https://arxiv.org/pdf/2511.16652
Title: Evolution Strategies at the Hyperscale (Eggroll)
Content: EGGROLL is a low-rank Evolution Strategies method that makes black-box optimization far more GPU-efficient by replacing unstructured random perturbations with rank-r matrix perturbations, achieving up to 91% of pure batch-inference throughput and a hundredfold training-speed increase for billion-parameter models at large population sizes. The paper theoretically shows that EGGROLL remains consistent with Gaussian ES in high dimensions and experimentally demonstrates strong performance across integer-only recurrent language model pretraining, LLM reasoning post-training, and tabula rasa RL.
Slack link: ml-ka.slack.com, channel: #pdg. Please join us -- if you cannot join, please message us here or to mlpaperdiscussiongroupka@gmail.com.

In the Paper Discussion Group (PDG) we discuss recent and fundamental papers in the area of machine learning on a weekly basis. If you are interested, please read the paper beforehand and join us for the discussion. If you have not fully understood the paper, you can still participate – everyone is welcome! You can join the discussion or simply listen in. The discussion is in German or English depending on the participants.