😶‍🌫️ Blind Spots & Broken Bots: AI Agent Observability

Blind Spots & Broken Bots

Getting an AI agent running is one thing.
Understanding what it's doing when something goes wrong is another.

As agents become more complex, debugging failures gets harder. A failed tool call, an unexpected LLM response, or an agent stuck in a loop might not trigger a traditional alert.

Without visibility into what's happening inside the workflow, troubleshooting quickly turns into guesswork.

For this Prompt Engineers AI session, we'll explore how to bring observability to AI agents using an open-source, vendor-neutral stack built around LangChain, OpenTelemetry, and the OpenTelemetry Collector.

We'll look at how to instrument agents, collect telemetry, and route traces, metrics, and logs into Prometheus, Tempo, Loki, and Grafana. We'll also cover how using the OpenTelemetry Collector helps keep observability backends separate from application code, whether you're running agents in Docker, Kubernetes, or another containerized environment.

What we'll cover:

• Agent tracing: Follow agent execution from LLM interactions through tool calls to better understand behavior and failures.
• Telemetry collection: Use OpenTelemetry and the Collector to route traces, metrics, and logs.
• Visualizing agent behavior: Build Grafana views to help identify failures and troubleshoot workflows.
• Vendor-neutral observability: Understand how to change observability backends at the Collector without modifying agent instrumentation.
• Starter implementation: Walk away with an open-source boilerplate implementation you can explore, extend, and use as a foundation for your own projects.

Who should attend?
Platform engineers, SREs, DevOps practitioners, and developers interested in building or operating AI agents.

No prior OpenTelemetry experience is required.

Whether you're already running agents or just starting to explore what it takes to operate them reliably, this session will provide a practical introduction to agent observability.

You'll also leave with a working starting point, so you can continue experimenting and building beyond the meetup.

Come learn, ask questions, and connect with others working through similar challenges!

About the Speaker: Mithun Banerjee
Mithun Banerjee is a senior technical leader at Splunk specializing in observability, platform engineering, and AI infrastructure. He brings hands-on experience with OpenTelemetry, Kubernetes, and AI agent observability, and has presented at Cisco Live and CNCF community events. He's also active in the Dallas engineering community, leading technical workshops and sharing practical approaches to cloud-native engineering.

Details:
📍 Location: Improving Plano — 5445 Legacy Dr #100, Plano, TX
🕕 Time: Wednesday, October 28th · 6:00–9:00 PM
💻 Livestream Link

Agenda:
6:00 – 6:30 PM
Check-in, networking & pizza

6:30 – 7:30 PM
Blind Spots & Broken Bots: AI Agent Observability
Mithun Banerjee

7:30 - 9:00 PM
Q&A and Networking

🌐 Stay Connected
If this is your first time joining us, welcome! Join the Prompt Engineers AI community, follow us on LinkedIn, and connect in Slack to stay involved between meetups.
www.promptengineers.ai