13th SF Bay Area Meetup – Eval. Agentic AI & Shifting Quality Left for AI Code

13th SF Bay Area Meetup – Evaluating Agentic AI and Shifting Quality Left for AI-Generated Code

AI is changing how software is built, tested, and delivered. But how do we ensure AI agents behave as expected, and how do we maintain quality when AI starts generating code, skills, and entire workflows?

At The Test Tribe’s 13th SF Bay Area Meetup, Ojasvi Agarwal and Noel Yap will explore two important dimensions of AI-driven Quality Engineering: evaluating the behaviour of agentic AI systems and embedding quality practices into AI-generated code, skills, and agents.
Through practical approaches and engineering examples, both sessions will examine how teams can move beyond traditional testing methods to build more reliable, measurable, and trustworthy AI-driven systems.

Session 1
Deep Dive on DeepEval

Speaker: Ojasvi Agarwal, Systems/Software Engineer 2

Ojasvi works on validation engineering and AIOps for Data Center Assurance, HPE's datacenter networking platform. She built the DCA Simulator, which models over 8,000 network devices and 100,000 flows per cycle to stress-test the platform and generate training data for failure-prediction models. She also developed a Day-0 setup validation tool adopted into Juniper's official product documentation.

A speaker at mabl Experience 2026 and author of a single-authored paper at ACM Creativity & Cognition 2024, Ojasvi holds an MS in Computer Science from UIUC.

In this session, she will introduce DeepEval, an evaluation framework designed to assess agentic AI systems whose dynamic responses and decisions cannot always be validated through traditional assertion-based testing. The session will demonstrate how LLM-as-a-judge and configurable evaluation thresholds can measure not just an agent's final response but also its behaviour, tool usage, and performance across multi-turn conversations.

The session will cover:

• Beyond Traditional Test Automation: Understanding why static expected outputs and conventional assertions are insufficient for evaluating nondeterministic AI responses.
• Evaluating AI Outputs: Using DeepEval, LLM-as-a-judge, configurable criteria, and acceptance thresholds to assess relevance, factual accuracy, and task alignment.
• Measuring Agent Behaviour: Evaluating task completion, tool selection, argument correctness, step efficiency, hallucinations, and multi-turn conversations.
• Testing MCP-Based Agents: Assessing whether agents select and use MCP tools effectively to complete the intended tasks.

Session 2
Shifting Quality Left for AI-Generated Code with Skills and Agents

Speaker: Noel Yap, T-Shaped Software Engineer and OODA Loop Tightener

Noel brings extensive experience in improving software engineering and testing practices across organisations including JPMorgan, Morgan Stanley, Google, Netflix, GRAIL, and QAD Redzone.

Throughout his career, he has worked on version control, testability, CI/CD, build and release processes, and automation. His work includes teaching testability techniques at Google, implementing CI/CD practices at Netflix, and building software engineering and testability best practices into AI skills and agents at QAD Redzone.

In this session, Noel will explore how skills and agents can help reduce AI-generated code quality issues and bring stronger engineering discipline into AI-assisted development. He will also discuss how statistical methods can be used to test the effectiveness and reliability of AI skills and agents.

The session will cover:

• Reducing AI Slop: Using skills and agents to improve the quality and consistency of AI-generated code.
• Building Quality into AI Workflows: Applying software engineering and testability best practices through AI skills and agents.
• Testing Skills and Agents: Exploring statistical methods to evaluate the effectiveness and reliability of AI-powered workflows.

Event Details

• Date: 2 December 2026
• Time: 06:00 PM – 08:30 PM PDT
• Venue: 1631 N First St, San Jose, CA

Agenda

• 6:00 PM – 6:15 PM: Welcome Coffee & Opening Remarks
• 6:15 PM – 7:15 PM: Session 1 by Ojasvi Agarwal on “Deep Dive on DeepEval”
• 7:15 PM – 8:15 PM: Session 2 by Noel Yap on “Shifting Quality Left for AI-Generated Code with Skills and Agents”
• 8:15 PM – 8:30 PM: Closing Remarks & Networking over Snacks
• 8:30 PM: Event Wrap

The evening will bring together QA and Engineering professionals to exchange ideas on evaluating AI agents, improving AI-generated code, and making quality an integral part of AI-driven software development.

About The Test Tribe

The Test Tribe is the world's largest software testing community, empowering testers globally since 2018. With 980+ events and 176K+ lives touched across 130+ countries, we bring testers together to learn, grow, and connect through meetups, conferences, cohorts, masterclasses, and more.

Become a Speaker at The Test Tribe Events

Wish to share your insights with the community at a future event?
Submit your talk idea here, and if it resonates with the perspective we are trying to build at our events, our team will connect with you.

By RSVPing, you agree to our Terms and Conditions and Privacy Policy and consent to be contacted by The Test Tribe and our event partners.