From Demo to Production: Building Enterprise AI Agents on Databricks

Format: 60 minute session
Structure: 30 minute talk + 30 minute hands on lab
Audience: Practicing engineers and Students
Level: Intermediate
Track: AI Engineering / Applied ML
Prerequisites: Familiarity with LLMs
Most enterprise AI agent projects don’t fail in production, they fail before production because they were never the right solution for the problem in the first place. While demos often look impressive, many teams struggle to bridge the gap between prototype and a system that can be evaluated, governed, deployed, and trusted in a real enterprise environment.
This session explores the practical patterns required to take AI agents from experimentation to production using Databricks. Attendees will learn how to identify enterprise problems that genuinely warrant an agent and how to distinguish agents from workflows, RAG applications, and function calling systems. The session also examines the seven major production failure points that derail enterprise agent projects, including orchestration, tool governance, retrieval, evaluation, observability, deployment, and cost management.
The presentation introduces a practical decision rubric and a challenge to capability framework teams can use to evaluate whether an agent is the right architectural choice before investing engineering effort. It also walks through production ready patterns using Databricks capabilities such as Foundation Models, Agent Framework, Unity Catalog tools, Vector Search, MLflow, and Agent Evaluation.
The second half of the session is fully hands on. Participants will build and test a working AI agent within a provided Databricks notebook using either a sandbox environment or their own free tier workspace. The lab is designed so attendees who fall behind can still complete the exercise later using the provided notebook and supporting resources.

Opening (2 mins)
A short story about an AI agent project that demoed beautifully but never shipped and the key question the rest of the session answers.
Part 1 — When Does an Agent Earn Its Keep? (6 mins)
A five question rubric and two axis framework for distinguishing agents from workflows, RAG applications, and simple function calling systems.
Part 2 — Seven Things That Break in Production (12 mins)
A breakdown of the most common production challenges including orchestration, tool governance, retrieval, evaluation, observability, deployment, and cost management.
Mapped directly to Databricks capabilities with selected deep dives into real implementation patterns.
Part 3 — Reference Architecture (5 mins)
How enterprise agent systems fit together using Foundation Models, Agent Framework, Unity Catalog tools, Vector Search, MLflow, and Agent Evaluation.
Part 4 — Lab Framing (5 mins)
Introduction to the hands on use case, prerequisites check, and walkthrough of the lab structure.
Part 5 — Hands On Lab (30 mins)
Attendees follow along in a provided Databricks notebook to build a working AI agent using production oriented design patterns.

Proposed Take Home for Attendees
A completed notebook runnable in any Databricks workspace.
Supporting code and sample datasets.
A reusable decision rubric for enterprise AI agents.
A practical challenge to capability mapping framework.
Hands on experience building and evaluating a production style AI agent system.