Bangalore Airflow Meetup at Cloudera HQ

📣📣 Attention Airflow Users and Big Data Professionals 📣📣

Meet up with fellow Airflow users at Cloudera HQ for a night of awesome talks and networking.

This is the perfect opportunity to get to know like-minded Airflow users, partners, and company experts, and learn how customers are utilizing Airflow within their organizations.

If you'd like to give a presentation at or host a future meetup, please email briana.okyere@astronomer.io

AGENDA

• 5:00 PM - 5:30 PM: Arrivals and networking
• 5:30 PM - 7:30 PM: Presentations
• 7:30 PM - 8:00 PM: Networking

PRESENTATIONS

Talk #1: Airflow under the hood: DAG Serialization

• Speaker: Utkarsh Sharma, Senior Software Engineer at Astronomer and Apache Airflow Committer
• Abstract: Contributing to a robust codebase like Airflow can seem daunting, especially when delving into intricate elements like DAG serialization. This talk aims to unravel the complexities surrounding DAG serialization within Airflow, offering a clear understanding of its fundamentals, and key mechanisms. Intended for Python enthusiasts and seasoned Airflow users, this session aims to empower potential contributors, providing them with the confidence and knowledge to navigate and contribute effectively to this crucial facet of Airflow's architecture.

Talk #2: Supercharging Airflow with Custom Python Packages

• Speaker: Amogh Desai, Software Engineer II at Cloudera and Apache Airflow Committer
• Abstract: Ever struggled with Airflow when handling your custom Python code and packages? To overcome this, we propose a solution: creating a dedicated Airflow Python environment using a resource builder service. This service generates a requirements file, sets up a version-compatible Python virtual environment adhering to Airflow’s policies, and handles custom pip repository authentication.

Talk #3: Data Workflows at Scale

• Speakers: Sumit Maheshwari, Tech Lead at Uber and PMC at Apache Airflow, and Rajat Srivastava, Senior Software Engineer at Uber
• Abstract: Sumit & Rajat will walk attendees through how Uber manages thousands of pipelines and executes millions of tasks each day via their custom workflow management tool called Piper.