Orlando AWS UG July Meetup

Beyond the Demo: Engineering AI Agents That Actually Work in Production

Most AI agent demos work perfectly on stage. In production, they hallucinate, retry endlessly, burn through budgets, and fail silently. This talk covers the engineering behind a production-grade AI agent that replaced 12 AWS Lambda functions with a single containerized system processing 1,300+ contracts for an enterprise customer. You'll learn how to build self-healing extraction that auto-corrects 79% of errors without human intervention, a Prompt Lab where subject matter experts test prompt changes against real documents for $0.03 without touching production, an Insights engine that surfaces cost anomalies and quality regressions across thousands of extractions, and a developer dashboard that provides full pipeline observability from a single command with zero deployed infrastructure. The talk also covers automated health checks that detect and restart stuck processing, graceful shutdown and cancellation patterns, adaptive concurrency with circuit breakers, and the cost optimization strategies that keep the system stable and affordable at scale. Whether you're building your first agent or hardening an existing one, this session gives you the architecture decisions and production patterns that separate demos from systems that run unattended.