Can You Trust Your AI? The Eval Mindset

Three months after the verdict in The Nilgiri Taj Theft, the case returns to court.

The prosecution's star witness is Athena AI. After reviewing the evidence, Athena is 97% confident that the accused is guilty.

Case closed, right? Not quite.

The defense has one question: how do we know Athena is right?

In this session, we'll put Athena on trial and use the case to explore how an AI system can sound convincing, yet still get things wrong. We'll uncover some of AI's most common failure modes, from hallucinations and prompt injection to misleading confidence scores and flawed evaluation frameworks.

We'll also look at how modern teams use evals, guardrails, LLM-as-judge, golden datasets, monitoring, and evaluation pipelines to build AI systems that are safe, reliable, and production-ready.

After all, "sounds confident" and "is correct" aren’t always the same thing.

What You’ll Learn:

• Why traditional testing fails for AI systems
• The different types of AI evals and where they fit
• Common LLM failure modes like hallucinations and prompt injection
• How teams measure AI quality and reliability
• The role of guardrails, datasets, and monitoring in production AI

Seats are limited and available on a first-come, first-served basis. RSVP is mandatory, so be sure to register early!

Once you secure your spot, we’ll send a confirmation email your way.

Will Athena's verdict hold up in court? There's only one way to find out.

See you at the meetup!