*in-person* Roberta Raileanu (Google DeepMind) - Automating Scientific Discovery

Title: Automating Scientific Discovery: How Far Are We?

Speaker: Roberta Raileanu (Google DeepMind)

Abstract: This talk provides an overview of using LLMs to automate scientific discovery, arguing that current models fall short of their ambitious goals. To rigorously measure these limitations, we introduce MLGym, a comprehensive benchmark for diverse, open-ended AI research tasks, and the LLM Speedrunning Benchmark, which specifically tests an agent's ability to reproduce known innovations. Our findings show that even frontier models struggle significantly on these benchmarks. Beyond just evaluation, we demonstrate how to improve AI research agents by formalizing their process as a search problem, showing that enhancing exploration, generalization, and search strategies can lead to state-of-the-art performance. Ultimately, this work presents a comprehensive view of current limitations while offering promising directions and approaches needed to advance the field.

Bio: Roberta Raileanu is a Senior Staff Research Scientist at Google DeepMind and Honorary Lecturer at UCL where she’s co-teaching a course on Open-Endedness and Artificial General Intelligence. Her work focuses on designing open-ended learning systems drawing from different fields such as reinforcement learning, self-supervised learning, evolutionary search, and foundation models. Previously, Roberta was a Research Scientist at Meta, where she led tool use for Llama and worked on accelerating AI research. Roberta received her PhD in computer science at NYU, where she worked on generalization in deep reinforcement learning.

Agenda:
5:15pm - Registration opens
6pm - Talk
6:40pm - Q&A session
7pm - Networking + refreshments (soft/alcoholic drinks + pizza)
8pm - Close

Please note that the live talk will be recorded and uploaded to the Meetup YouTube channel.
Many thanks to our sponsors, Man Group and ArcticDB, for hosting this Meetup.