Lessons Learned from Building Effective AI Agents for Millions of Users

Context Engineering for AI Agents: Lessons from Building Manus

We are going to take an in-depth look at context engineering as a core strategy for building effective AI agents for millions of users, drawing lessons from the development of the "Manus" project. It explains why the team chose context engineering over training end-to-end models due to the need for fast iterative improvements, referring to this trial-and-error process as "Stochastic Graduate Descent." The article then outlines several critical, non-obvious principles for optimization, such as designing agents to maximize the KV-cache hit rate to reduce latency and cost, and employing a strategy to mask token logits instead of dynamically removing tools to maintain cache stability. Furthermore, it discusses using the file system as external, persistent memory to overcome context window limits and improve recovery by leaving failed actions in the context so the model can learn from errors rather than repeating them. Finally, the text advises against overly uniform few-shotting, recommending structured variation to prevent the agent from getting stuck in repetitive behavioral patterns.

Slides for past meetups posted: Github
Recordings have been posted at: YanAITalk

Feel free to reach out if you want to present a paper or a use case at upcoming meetups!

Note: You must have a Zoom account to login (free account is sufficient). Link and password will be shared three days before the meeting.