Bangalore Apache Airflow® Meetup at Cloudera!

Meet up with fellow Airflow users at Cloudera's Bangalore Office for an afternoon of awesome talks and networking.

This is the perfect opportunity to get to know like-minded Airflow users, partners, and company experts, and learn how customers are utilizing Airflow within their organizations.

🔥 Agenda

• 1:30 PM – 2:00 PM: Arrival & Settling In
• 2:00 PM – 2:30 PM: Talk 1 – The Evolution of Orchestration: Airflow’s Journey and It’s AI-Powered Future
• Speaker: Shubham Raj Software Engineer II - CDE at Cloudera & Shreya Sareen Product Manager at Cloudera
• 2:30 PM – 3:00 PM: Talk 2 – Breaking the Connection: Why Airflow Tasks No Longer Touch the Database
• Speaker: Amogh Desai, Senior Software Engineer at Astronomer & Apache Airflow PMC Member
• 3:00 PM – 3:30 PM: Talk 3 – GenAI based log summarization and mitigation steps at Uber
• Speaker: Sameer Raj & Atish Anand, Uber
• 3:30 PM – 4:00 PM: Networking & Snacks

🎤 Featured Talk Highlights

Talk 1- The Evolution of Orchestration: Airflow’s Journey and It’s AI-Powered Future
Speakers: Shubham Raj & Shreya Sareen
Workflow orchestration has been on quite a journey. It started with tools, which gave data teams a way to manage complex Hadoop jobs at scale. Then came Airflow, bringing Pythonic DAGs, a flexible scheduler, and an active community quickly becoming the go-to platform for modern data pipelines. Now, a new chapter is opening: AI is beginning to reshape how we design and interact with workflows. From helping generate DAGs to making orchestration more accessible, AI is pushing us toward a more intuitive developer experience.
This talk traces that evolution, from foundational tools to Airflow’s rise, and the AI-powered frontier ahead.

Talk 2- Breaking the Connection: Why Airflow Tasks No Longer Touch the Database
Speaker: Amogh Desai, Senior Software Engineer, Astronomer & Apache Airflow PMC Member
Airflow tasks traditionally had direct access to the internal database — a design that caused scalability, security, and dependency issues. With AIP-72, Airflow introduces a radical improvement: running tasks in isolated environments that communicate with Airflow via a lightweight API. Learn how this new approach enables:

• Better security with isolated execution
• Elimination of dependency conflicts
• Task execution on any infrastructure
• Simplified development experience

Talk 3 - Piper (an Apache Airflow fork) serves hundreds of teams at Uber, with ~200k pipelines, and 800k tasks run in a day.
Speakers: Sameer Raj & Atish Anand, Uber
With huge scale comes the huge responsibility of supporting users, with >300 support queries a week. This huge inflow of queries is overwhelming for the on-call engineer to tackle, leading to delays in resolution. The queries, for example, can be like "Why did my task fail?", "Why was my pipeline paused?", "Can you help me with writing a task to upload a file to HDFS?", etc.
Thus, to solve this, we leveraged GenAI and developed agents.
The support resolution rate is 45%.
***