State management for LLMs

As the AI space matures, the interface at which developers interact with these models will shift from stateless APIs (e.g., ChatCompletions) to stateful APIs (e.g., “agent” APIs). This talk discusses state management for building AI applications for complex real-world applications.

In what ways does this software layer resemble an operating system? How to intelligently manage AI state to overcome the inherent limitations of the underlying foundation models, such as reliability, planning, noisy feedback loops, retrieval, agent drift/derailment, personalization, context pollution, and context overflow?

About the speaker
Charles Packer is a PhD candidate at UC Berkeley and part of the Berkeley AI Research Lab and Sky Computing Lab. His current research focus is on building agentic systems driven by large language models (LLMs).

His recent work, MemGPT (MemoryGPT), proposes a novel “operating system for LLMs” that allows LLMs to manage their own memory and represents a promising new direction for maximizing the capabilities of LLMs within their fundamental limits.