Apache Airflow Meetup - Bangalore

### Meetup: Scaling Apache Airflow – Community Meetup in Collaboration with Astronomer

📅 Date: September 20, 2025
🕜 Time: 1:30 PM – 4:00 PM IST
📍 Location: Cloudera Office, Bangalore
👥 Hosted by: Future of Data – Bangalore, in collaboration with Astronomer

***

Join us for an insightful in-person Apache Airflow Meetup, hosted at the Cloudera Bangalore office, where we’ll dive deep into real-world challenges, innovative solutions, and best practices from industry experts and contributors. This is a fantastic opportunity to meet fellow Airflow users, partners, and engineers, and discover how organizations are scaling Airflow for critical workloads.
Whether you're an Airflow user, contributor, or just getting started — you're welcome!

***

### 🔥 Agenda

1:30 PM – 2:00 PM: Arrival & Settling In

2:00 PM – 2:30 PM: Talk 1 – The Evolution of Orchestration: Airflow’s Journey and It’s AI-Powered Future
Speaker: Shubham Raj Software Engineer II - CDE at Cloudera & Shreya Sareen Product Manager at Cloudera

2:30 PM – 3:00 PM: Talk 2 – Breaking the Connection: Why Airflow Tasks No Longer Touch the Database
Speaker: Amogh Desai, Senior Software Engineer at Astronomer & Apache Airflow PMC Member

3:00 PM – 3:30 PM: Talk 3 – GenAI based log summarization and mitigation steps at Uber
Speaker: Sameer Raj & Atish Anand, Uber

3:30 PM – 4:00 PM: Networking & Snacks

***

### 🎤 Featured Talk Highlight

####

Talk 1- The Evolution of Orchestration: Airflow’s Journey and It’s AI-Powered Future
Speaker: Shubham Raj & Shreya Sareen
Workflow orchestration has been on quite a journey. It started with tools, which gave data teams a way to manage complex Hadoop jobs at scale. Then came Airflow, bringing Pythonic DAGs, a flexible scheduler, and an active community quickly becoming the go-to platform for modern data pipelines. Now, a new chapter is opening: AI is beginning to reshape how we design and interact with workflows. From helping generate DAGs to making orchestration more accessible, AI is pushing us toward a more intuitive developer experience.
This talk traces that evolution, from foundational tools to Airflow’s rise, and the AI-powered frontier ahead.

Talk 2- Breaking the Connection: Why Airflow Tasks No Longer Touch the Database
Speaker: Amogh Desai, Senior Software Engineer, Astronomer & Apache Airflow PMC Member
Airflow tasks traditionally had direct access to the internal database — a design that caused scalability, security, and dependency issues. With AIP-72, Airflow introduces a radical improvement: running tasks in isolated environments that communicate with Airflow via a lightweight API. Learn how this new approach enables:

• Better security with isolated execution
• Elimination of dependency conflicts
• Task execution on any infrastructure
• Simplified development experience

Talk 3 - Piper (an Apache Airflow fork) serves hundreds of teams at Uber, with ~200k pipelines, and 800k tasks run in a day.
Speaker: Sameer Raj & Atish Anand, Uber
With huge scale comes the huge responsibility of supporting users, with >300 support queries a week. This huge inflow of queries is overwhelming for the on-call engineer to tackle, leading to delays in resolution. The queries, for example, can be like "Why did my task fail?", "Why was my pipeline paused?", "Can you help me with writing a task to upload a file to HDFS?", etc.
Thus, to solve this, we leveraged GenAI and developed agents.
The support resolution rate is 45%.

***

💬 Want to present, collaborate, or volunteer at future meetups?
Feel free to reach out — we’re always open to community contributions!
🔗 RSVP now and block your seat – limited capacity!

***