Towards a Neural Debugger for Python

Maximilian Beck will give a remote talk on May 13th at 15:00.

Details follow:
"Towards a Neural Debugger for Python"

Training large language models (LLMs) on Python execution traces grounds them in code execution and enables the line-by-line execution prediction of whole Python programs, effectively turning them into neural interpreters (FAIR CodeGen Team et al., 2025). However, developers rarely execute programs step by step; instead, they use debuggers to stop execution at certain breakpoints and step through relevant portions only while inspecting or modifying program variables. Existing neural interpreter approaches lack such interactive control. To address this limitation, we introduce neural debuggers: language models that emulate traditional debuggers, supporting operations such as stepping into, over, or out of functions, as well as setting breakpoints at specific source lines. We show that neural debuggers—obtained via fine-tuning large LLMs or pre-training smaller models from scratch—can reliably model both forward execution (predicting future states and outputs) and inverse execution (inferring prior states or inputs) conditioned on debugger actions. Evaluated on CruxEval, our models achieve strong performance on both output and input prediction tasks, demonstrating robust conditional execution modeling. Our work takes first steps towards future agentic coding systems in which neural debuggers serve as a world model for simulated debugging environments, providing execution feedback or enabling agents to interact with real debugging tools. This capability lays the foundation for more powerful code generation, program understanding, and automated debugging.

Bio:
Maximilian Beck is an AI Scientist in the CodeGen team at Meta AI (FAIR).
He recently received his PhD from Johannes Kepler University Linz, where he was advised by Sepp Hochreiter. His research focuses on efficient architectures for large language models, including recurrent and sub-quadratic approaches, such as the xLSTM (Extended Long Short-Term Memory), as well as applications to code generation. He received his Bachelor’s and Master’s degrees in Mechatronics and Information Technology from Karlsruhe Institute of Technology in 2017 and 2021. In 2025, he was a research intern with the CodeGen team at Meta AI, where he worked on code world models, code execution prediction, and neural debuggers --- topics related to the work presented in this talk.

Zoom link: https://kth-se.zoom.us/j/61111843462