The Art of Scaling Reinforcement Learning Compute for LLMs - review

# The Art of Scaling Reinforcement Learning Compute for LLMs

This Saturday we're reviewing "The Art of Scaling Reinforcement Learning Compute for LLMs", a great new paper from Meta sharing insights into running RL at large scale.

Preparation:
There will not be a prepared presentation but more of a structured conversation.
Spend some time ahead of the meeting skimming the paper so we can have a productive conversation.

Resources:
Paper: https://arxiv.org/abs/2510.13786

Algorithm improvement over GRPO
- CISPO https://arxiv.org/abs/2506.13585

RL infrastructure
- Pipeline RL - https://huggingface.co/blog/ServiceNow/pipelinerl
- Magistral - https://arxiv.org/abs/2506.10910