Nebius Academy Israel

Agentic AI systems can plan, use tools, and act across multi-step workflows, but reliability in production remains an open problem. Current limitations in scaling trustworthy agents aren't just a model issue. They're a data problem.

This session covers where and why human data matters across the agent development lifecycle: from pre-training and trajectory optimization to production monitoring and real-time expert escalation. Expect concrete use cases, a live demo, and time for questions.

What we'll cover:

1. The data gap.
   Why training data built for standard LLMs is insufficient for agents operating across multiple steps and modalities.
2. Human feedback for planning and optimization.
   Verifying tool selection, reasoning chains, and intermediate steps; adapting preference ranking to sequential task evaluation.
3. Production reliability.
   How human annotators catch failure modes that automated metrics miss, and how Tendem via MCP makes verified expert judgment a callable layer inside live agent workflows.
4. Live demo.
   Toloka Arena, RL Gym walkthroughs, and self-service presets you can apply to your own projects.

Speakers:
Vladislav Davydkin, Technical Solutions Engineer at Toloka
Renaud de la Guéronnière, Program Director at Toloka
Emeli Dral, Co-founder and CTO at Evidently AI
Alexander Golubev, Research Lead (LLM, RL & Agents) at Nebius