When AI Systems Fail: Building Reliable, Explainable Systems in the Real World

Join us for an evening exploring how we move beyond AI hype to build systems that actually work, and are fixable when they fail.

Together, our two expert speakers offer complementary visions: first we will learn how to build systems that adapt and self-correct in production. Next, we'll hear how to build explainability and trustworthiness into the foundation. Both challenge us to think beyond the hype and build AI that actually serves human needs.

We will have free pizza sponsored by Customer Paradigm (customerparadigm.com), a Boulder, Colorado company that specializes in AI integration, application development and digital marketing. Since 2002, Customer Paradigm has been led by founder Jeff Finkelstein, and has worked directly with the AWS machine learning team to build AI tools for eCommerce sites.

We'll open with Marisa Ferrara Boston: "Self-Healing AI Systems: From Detection to Automatic Repair"

Marisa Ferrara Boston is the Founder and CEO of Simthetic AI, a Denver-based company building a synthetic agent and simulation platform to test and refine generative AI systems before deployment.

Marisa will challenge a fundamental assumption in AI deployment: that detecting failures is hard. Drawing from her work with financial auditing firms and healthcare organizations, she'll demonstrate why treating AI agents like traditional software is not just ineffective, it's a losing strategy.

Key insights you'll gain:
- Why complex systems require new thinking: AI agent systems operate more like aviation or financial markets than traditional software. Users deploy them in unexpected ways, and failures have cascading consequences we can't predict.
- The fraud detection challenge: Marisa will walk through a concrete example showing how auditors struggle to detect fraud in financial systems, and why even when AI helps, outdated knowledge bases and policy mismatches create systematic failures. She'll demonstrate how her team automatically patches these systems by updating underlying content and verifying improvements through synthetic testing.
- Self-healing systems in practice: Learn how her approach converts AI failures into "synthetic twins," ships repair packets with automated tests, and verifies system improvements, all without human intervention. This isn't theoretical: it's happening now with major firms.

Next, Beth Rudden will cover, "From Magical Thinking to Explainable AI: Building Systems We Can Actually Trust"

Beth brings an unconventional lens, archaeologist turned IBM Distinguished Engineer turned AI CEO, to challenge how we think about AI development. Her talk addresses the 95% failure rate of AI pilots and makes the case for a fundamentally different approach.

What Beth will explore:
- Data as human artifact: Drawing from her archaeological background, Beth will show how our data isn't neutral, it's embedded with human biases and cultural contexts that get encoded into AI systems at scale. Understanding this is the first step toward building trustworthy systems.
- Deterministic AI through ontological scaffolding: Her company Bast.ai achieves 100% certainty in data retrieval using W3C standards and version control. She'll demonstrate how this enables multi-persona outputs, delivering the same medical protocol appropriately for physicians, families, and children.
- Real-world healthcare applications: Beth will share examples from healthcare environments where ambient intelligence doesn't just retrieve information, it knows when a patient searching for insurance details at 2 AM needs sleep recommendations instead, alerting nurses and adjusting the environment.

'When AI Systems Fail: Building Reliable, Explainable Systems in the Real World'
CU Boulder East Campus: In-person
Aerospace Engineering Sciences Building (Room 120)
3775 Discovery Dr., Boulder CO 80303
Wed, Oct 15th
6:00 - 8:00 PM - Meeting
5:30 PM Free pizza (thanks to our pizza sponsor, Customer Paradigm!)
Parking: https://www.rmaiig.org/parking

At the end of the meeting, we'll assemble the speakers for a Q&A panel. Bring lots of questions!

Special thank you to Bobby Hodgkinson for arranging the meeting space!