Building a Production-Grade Cloud Advisor / Vector Database Benchmarking

For the February LA DevOps meetup we have two interesting presentations leading into DevOpsDayLA at SCALE23x which happens the following week.

The first presentation is from Nicolas Micali Co-Founder and CTO of [CloudGo.ai](https://cloudgo.ai/) who will be discussing Turning Context into Decisions which goes into detail on how CloudGo.ai built a production grade cloud advisor.

For the second presentation Reza Rassool founder of Kwaai Labs will be giving a quick update on the Kwaai Summit at SCALE23x on March 5, and then discussing the results of the Vector Database Benchmarking Research rececntly completed intern group.

See the Presentation Details section below for more info.

**Sponsored by CloudGo.ai!!!!**

Thanks to CloudGo.ai for sponsoring LA DevOps!

Event Details:
6:00 - 6:30 Attendees arrive. Networking/Mingle
6:30 - 6:45 LA DevOps Welcome,Attendee Introductions
6:45 - 7:15 Nicolas Micali - Turning Context into Decisions: How We Built a Production-Grade Cloud Advisor
7:15 - 8:00 Reza Rassool - Vector Database Benchmarking
8:00 - 9:00 Networking/Mingle

Presentation Details:

Title: Turning Context into Decisions: How We Built a Production-Grade Cloud Advisor
**Presented By: Nicolas Micali CTO CloudGo.ai**

Description:
Nicolas will walk through why they built CloudGo.ai and what it takes to ship modern AI agents that can make real-world decisions to manage modern cloud infrastructure.

About Nicolas:
Nicholas Micali is the Co-Founder / CTO of [CloudGo.ai](https://cloudgo.ai/)

Title: Vector Database Benchmarking
Presented By: Reza Rassool
Description:
Reza will be discssuing the results of Kwaai Labs research of Vector Database Benchmarks.

Following is the abstract of the final report.

This study presents a comprehensive performance evaluation of seven production vector databases (FAISS, Chroma, Qdrant, Weaviate, Milvus, OpenSearch, pgvector) across nine corpus sizes ranging from 175 to 2.2 million chunks. Using rigorous N=10 statistical methodology with multi-pass outlier detection (91 cold-start outliers removed, 2% of measurements), we measured query latency, throughput, ingestion performance, and resource utilization under controlled conditions. Our findings reveal distinct performance classes: Chroma achieves near-constant time query performance (α=0.02) with 7.7-8.4ms latency and 141 QPS at medium scale with exceptional consistency (CV=2.3% after outlier removal), pgvector (HNSW) delivers exceptional 9.9ms latency and 101 QPS at 50k scale—outperforming all dedicated vector DBs except Chroma, while FAISS demonstrates exceptional sub-linear scaling (α=0.48) to 2.2M chunks with remarkable consistency (CV=2.5%). We quantify the HNSW "warm-up phenomenon" showing latency reductions of up to 74% as corpus size increases from 1k to 50k chunks. Pgvector's dual-index support (IVFFlat vs HNSW) provides unique flexibility with 2.3× performance difference at scale. Resource analysis reveals consistent 12-16GB memory footprint across databases with CPU utilization ranging from 16% (OpenSearch) to 25% (Chroma). OpenSearch exhibits catastrophic variance (CV=45-94%) making it unsuitable for production vector workloads. These results provide quantitative guidance for database selection based on scale requirements, latency tolerance, and consistency needs.

About Reza: Reza Rassool, Chair of Kwaai
After a successful start-up career, including award-winning products, two exits to Google, 75 patents, Reza retired as CTO of RealNetworks.
He then founded Kwaai, a nonprofit with the mission to democratize AI.

About O'Brien's Pub on Wilshire:
The covered patio at Obrien's Pub has been a great place to host LA DevOps meetups for our post covid revival. The food and drink are great and the staff are very friendly. O'brien's is also the LA home of Manchester United Football Club Supporters.
You may want to bring a jacket for the cooler months.
Regarding parking, there are meters on Wilshire and the side streets that expire at 6.

Present at an Upcoming LA DevOps Meetup:
If you would like to present at an upcoming LA DevOps meetup please send a message in meetup to the organizers with a proposed title and a sentence or two about the talk.

DevOpsDayLA at SCALE23x is happening on Friday March 6 at the Pasadena Convention Center
Registration to SCALE includes access to DevOpsDayLA.
We are looking for sponsors, please see our prospectus for details.
Use promo code DEVOP.

Subscribe to the LA DevOps Meetup Calendar at lu.ma
The LA DevOps events are also listed on lu.ma which has an interesting interface. Subscribe to LA DevOps on lu.ma at https://lu.ma/calendar/cal-F7BPqBLAe80T5nS