Virtual: Shift-Left Performance Engineering for Agentic AI with CI/CD

Speaker: Kandasamy Selvaraj | LinkedIn

Topic: Shift-Left Performance Engineering for Agentic AI with CI/CD Performance Signatures

Agenda:
🕛 9:00–9:05 — Welcome
🕛 9:05–9:55 — Kandasamy Selvaraj | LinkedIn - Shift-Left Performance Engineering for Agentic AI with CI/CD Performance Signatures
🕛 9:55–10:00 — Closing
Summary:
Summary :
Years ago, I built automated performance gates into CI/CD pipelines that cut production defects by 80% and sped up deployments by 40%. The idea was simple. Catch a problem in the pipeline, where it's cheap to fix, instead of waiting for production to find it for you. That discipline is now standard in traditional software. It barely exists yet in agentic AI.

Response time, error rate, and throughput have been the standard SLO dimensions for as long as most of us have been doing this. They're necessary for an agentic pipeline, but they're not sufficient. An agent can meet every one of those three and still be a problem: it took six tool calls and three LLM calls to get a simple answer, a supervising agent delegated a task to a subagent that never properly completed it, one agent in a chain timed out while the overall request still returned something, or it pushed average token usage per trace well past what the system was designed to handle. None of that shows up in a response time chart. All of it shows up in the trace.

This talk is about treating trace data as the source for a new generation of SLOs built specifically for agentic pipelines part of shift left performance gate. I'll walk through what these new dimensions look like in practice: tool call count and LLM call count per trace, chain depth, tool failure and retry rate, delegation errors between agents, agent-level timeouts, and average tokens per trace. I'll cover a concrete starting point I use, flagging any trace that exceeds five tool calls or two LLM calls as a real gate condition, using trace data from a Langfuse integration wired directly into the performance gate rather than a dashboard someone checks after the fact.

I'll also cover why average tokens per trace deserves a place at the promotion gate, not just at monitoring runtime. If a change to a prompt, a tool, or a chain pushes average tokens per trace past an agreed SLO, that pipeline doesn't get promoted to the next environment until it's back in range, the same way a latency regression would block a deploy in a traditional pipeline. The same thinking applies to delegation errors and timeouts. If a subagent quietly fails to finish what it was asked to do, catching that before promotion is a quick fix. Finding it in production is an incident. The same goes for a timeout that never shows up in your metrics because the system quietly returned a default answer instead of surfacing the failure, the request looks successful on paper, but something underneath actually broke.

What I have is a proven performance engineering discipline from one domain, and a genuine, worked-through case for what the new SLO dimensions need to be for agentic systems, and how to enforce them with the trace data teams are often already collecting.

Takeaways :
What attendees will learn:
1. Why response time, error rate, and throughput aren't enough to know if an agentic pipeline is healthy
2. The new SLO dimensions agentic systems need: tool call count, chain depth, tool failure rate, delegation errors, agent timeouts, and tokens per trace
3. How to turn trace data, using a Langfuse integration, into real pipeline gates instead of a dashboard someone checks after a complaint
4. A concrete starting threshold you can adapt, such as flagging traces over five tool calls or two LLM calls
5. Why delegation errors and timeouts need their own gate, separate from a top-level error rate that can mask them
6. Why average tokens per trace belongs at the promotion gate, and how that differs from watching spend at runtime
7. Real Demo of Agentic AI Shift left implementation

🎙️ Speaker Opportunities at Our Meetups

Have expertise in QA, leadership, or tech skills you'd love to share? We're always looking for engaging speakers for our monthly meetups. This is a fantastic platform to showcase your knowledge, connect with the community, and inspire others. If you're interested submit your topic here —we’d love to hear from you!