Skip to content

Details

# Tuning the Tuners: Building a Multi-Agent System for Search Relevance Optimization

Featuring: Eric Pugh, OpenSource Connections

Eric co-founded OpenSource Connections, shapes the OpenSearch project's tech direction, is worried about AI, and enjoys woodworking, especially Danish furniture reproduction.

***

## About the Talk

I finished a project to build a multi-agent system to tune an incredibly complex system, a Search Engine, and I want to share what I learned!

In this talk, I’ll introduce how I used a multi-agent architecture built with the Strands SDK, following the “Agents as Tools” pattern. The project successfully demonstrated that this multi-agent structure—featuring a single Orchestrator Agent managing multiple specialized agents—is a viable and effective path forward for complex search relevance tuning processes.

We’ll look at the codes and play with the online demo (plus peek at the server logs).

Finally, I’ll share critical lessons learned regarding model sensitivity, managing user expectations with asynchronous interactions, and the necessary steps to make existing OpenSearch APIs “agent-ready” for effective, high-velocity search optimization.

***

## Event Details

Date: Tuesday, Dec 9th
Time: 6:00 PM – 8:00 PM
Location: Studio IX – 969 2nd St SE, Charlottesville, VA

***

## Schedule

5:30 PM – Setup begins
5:45 PM — (Optional) Book Club Track
6:00 PM – Doors open (feel free to bring snacks or grab something nearby)
6:30 PM – Eric Pugh
7:30 PM – Breakout Track Reports and 3-Minute Pitches
7:45 PM – Open Forum Discussion
Share your projects, demo tools, or bring a topic for group discussion.
8:00 PM – Wrap-up and networking

Organizerss

Owen Zanzal - https://www.linkedin.com/in/owenzanzal/
Mike Powers - https://www.linkedin.com/in/michael-powers-4262033/
Chat with us on Cville Slack #cvilleaiexplorers
If you aren't a member of Cville Slack, DM me your email, and I'll send you an invite.

Sponsors

Studio IX
Coworking space in downtown Charlottesville, Virginia, designed for freelancers, entrepreneurs, and remote workers.
Studio IX offers a flexible shared office environment with options for open workspaces, dedicated desks, and private offices. The space fosters a collaborative community through weekly events and mixers, featuring a 'warm industrial' atmosphere with ample natural light.
Amenities include high-speed internet, soundproof phone booths, conference rooms, and premium coffee. Studio IX is staffed by a community manager on weekdays and provides 24/7 access for members.
Tours of the space are available; interested individuals can contact them via:

Events in Charlottesville, VA
AI Algorithms
AI and Society
New Technology
Professional Networking
Technology

Members are also interested in