NYC Airflow Meetup at Datadog!

Join fellow Airflow enthusiasts and leaders at Datadog's offices for an evening of engaging talks, great food and drinks, and awesome swag, co-hosted by Astronomer and Datadog!

PRESENTATIONS
Talk 1: Multi-Tenant Airflow at Scale
Speaker: Conlan Cesar, Analytics & Data Platforms, Software Engineer at Datadog
Datadog is a world-class data platform ingesting more than a 100 trillion events a day, providing real-time insights. Since our internal adoption of Airflow following the release of 3.0.0, the number of teams relying on our internal Airflow platform have grown organically and quickly. This internal Airflow adoption came with a number of platform challenges, requiring novel solutions which could support multi-tenancy, scalability, and bespoke runtime environments.

In this talk, we cover the changes we've implemented to allow for a secure and scalable multi-tenant Airflow. This includes plugins for the secrets backend to make different credentials available to Operators transparently, how we inject short-lived JWT tokens without needing to modify operators, methods to scale workers according to live demand with Datadog Autoscaling, and upstream contributions to the triggerer to allow for specific queue assignment.

Talk 2: Apache Airflow for Production-ready AI
Speaker: George Yates, Field Engineering Manager at Astronomer

Running AI workloads in production takes more than a model call. This talk shows how Apache Airflow's Common AI provider brings production-ready AI orchestration into your DAGs: single-turn LLM calls, multi-turn agents with tools, and building blocks for RAG, SQL generation, and file analysis. You'll also see how to productionize it all with durable execution, usage limits, and human-in-the-loop review.

AGENDA
5:30-6:30 PM: Networking, food and drinks
6:30-7:00 PM: Talk 1 - Multi-Tenant Airflow at Scale, Conlan Cesar (Datadog)
7:00-7:30 PM: Talk 2 - George Yates (Astronomer)
7:30-8:30 PM: Networking