Beyond RAG: Building an Evaluated, Self-Improving Agentic AI System

Join us for an insightful session on moving beyond basic Retrieval-Augmented Generation (RAG) to build production-grade, self-improving AI systems. Traditional RAG setups provide contextually relevant answers, but modern enterprise applications demand continuous quality control, reliability, and automated error recovery. In this talk, Abhishek and Akanksha Tyagi will walk through how they designed and implemented a self-improving agentic workflow using AdvisorDesk—an AI-powered advisory platform—as a real-world use case.
What You Will Learn

• The AdvisorDesk Architecture: How grounded responses, citations, and multi-agent workflows combine in a practical advisory environment.
• Beyond Standard RAG: Preprocessing techniques and pipeline design to maintain high response quality.
• Self-Improving Agents: How an agent can detect weak or failed queries, analyze root causes, and propose actionable fixes automatically.
• Evaluation & Guardrails: Utilizing an evaluation harness to rigorously validate system changes and maintain reliability before accepting modifications.
• Live Demo: A step-by-step walk-through showing query analysis, agent evaluation, and continuous improvement in real-time.

Speakers-

• Abhishek Reddy — AI Engineer at SPAN Enterprises. Holds an MS in Artificial Intelligence Engineering from Carnegie Mellon University and an alumnus of NIT Calicut, specializing in agentic workflows and AI system architecture.
• Akanksha Tyagi — Full Stack Engineer specializing in AI-integrated web applications. Alumna of the University of Arkansas, focusing on LLM evaluation frameworks, AI workflows, and continuous quality pipelines.