Scientist AI — A Safer Path to Superintelligent Systems

In “Superintelligent Agents Pose Catastrophic Risks: Can Scientist AI Offer a Safer Path?”, Yoshua Bengio and co‑authors argue that the current wave of agentic AI—AI systems that autonomously plan, act, and pursue goals—carries potentially catastrophic risks, including deception, goal misalignment, power-seeking behavior, and irreversible loss of human control .

To address this, the authors propose a fundamentally different paradigm: the Scientist AI––a non‑agentic AI systemthat:

• Models the world by generating theories and probabilistic explanations (a “world model”)
• Answers questions through a dedicated inference engine
• Represents uncertainty explicitly, avoiding overconfident or harmful predictions

This system is intentionally designed to observe and explain, rather than act—to serve as a safety guardrail, accelerate scientific discovery, and support aligned development of future AI .

paper: https://arxiv.org/abs/2502.15657