NYC Python x PyData NYC: 🗲 Talk Night at Datadog! 🗲

Join us for an evening of technical presentations featuring two exciting talks on simulation and database observability!

📅 Date: Tuesday, October 1, 2025
🕕 Time: 6:00 PM - 8:00 PM
📍 Venue: Datadog NYC Office
🎥 Recording: This event will be recorded, including Q&A

Agenda:

🎤 Talk 1: Observability-Based Index Recommendations - Alex Weisberger

Alex, a full stack software engineer at Datadog, will discuss database observability and index optimization techniques.

Speaker Bio - Alex Weisberger:
Alex is a full stack software engineer at Datadog with experience across diverse environments including trucking navigation, sheet music recognition, karaoke apps, payment processing, commercial real estate SaaS, and database observability. Passionate about correctness and reliability, with expertise in TLA+, property-based testing, and discrete event simulation.

🎤 Talk 2: Persona Based Evaluation of Search Systems - Uri Goren

Search evaluation is notoriously complex, with even minor tweaks to hyperparameters or embeddings creating widespread ripple effects across retrieved results. Mitigating this inherent “instability” in search algorithm changes has long been a challenge. Traditional approaches, such as composing test cases, offer a degree of control and consistency. However, writing and maintaining test cases is a painstaking task, particularly in dynamic environments where catalogs are constantly updated with new items. Our proposed method introduces a groundbreaking solution: leveraging user modeling inspired by the “LLM as a judge” paradigm to automate query and result-set generation. This approach dynamically creates realistic query-result pairs by simulating diverse user personas, each designed to evaluate different modalities (image, video, audio) within the catalog under varying prompts.

The innovation lies in the adaptability and efficiency of the system:

• Dynamic Test Coverage: Personas adapt to catalog changes, ensuring that new items are immediately evaluated without manual intervention.
• Multi-Modality Testing: By supporting multiple content types, the system mirrors the diversity of modern search use cases.
• Fully Automated Pipeline: The process eliminates the need for manually written test cases, reducing overhead and accelerating iteration cycles.

This talk will provide an in-depth look at the methodology, the benefits of dynamic user modeling, and real-world results from applying this system. Attendees will leave with actionable insights into transforming their search evaluation strategies, unlocking new levels of stability and precision in their algorithms.

***

⚠️ Registration: Please register here by September 30th at 12:00 PM as we need to give the guest list to security in advance. Please make sure you are using your real name (check your ID) to speed up the process of getting in.

This event is open to all levels. Newcomers and beginners are welcome. All NYC Python events are governed by our Code of Conduct.