Airflow in Action: Exploring Use Cases Across Companies

We are excited to announce an upcoming Meetup focused on Apache Airflow, scheduled to take place on the 13th of February at the Cloudera Bangalore office. This event promises to be an insightful gathering for anyone interested in mastering the art of workflow orchestration and automation using Apache Airflow.

If you'd like to give a presentation at or host a future meetup, please email sbhatnagar@cloudera.com

Event Details:

• Date: 13th February
• Time: 05:00 PM IST to 08:00PM IST
• Location: Cloudera Bangalore Office

Speakers:
1. Utkarsh Sharma - Senior Software Engineer at Astronomer and Apache Airflow Committer
Aspiring Machine Learning Engineer currently pursuing Master's in AI/Ml from IIITB, with experience in web development as a Full Stack Developer, with a demonstrated history of working in the internet industry. Skilled in Regression Models, NLP, Python, GoLang, and Frontend tech/framework(Angular, HTML, CSS, JS). Strong engineering professional with a Bachelor of Engineering (BE) focused on Computer Science

Abstract: Contributing to a robust codebase like Airflow can seem daunting, especially when delving into intricate elements like DAG serialization. This talk aims to unravel the complexities surrounding DAG serialization within Airflow, offering a clear understanding of its fundamentals, and key mechanisms. Intended for Python enthusiasts and seasoned Airflow users, this session aims to empower potential contributors, providing them with the confidence and knowledge to navigate and contribute effectively to this crucial facet of Airflow's architecture.

2. Amogh Desai - Software Engineer II at Cloudera and Apache Airflow Committer
Seasoned engineer with a 3-year track record in cloud-native product development for data engineers, including familiarity in big data technologies. Apache Airflow Committer contributing across project domains. Skilled in Python, Golang, AngularJS, Helm Charts, Kubernetes, Apache Spark, and other big data technologies. Passionate about open source and driving innovation in tech.

Abstract: Ever struggled with Airflow when handling your custom Python code and packages? To overcome this, we propose a solution: creating a dedicated Airflow Python environment using a resource builder service. This service generates a requirements file, sets up a version-compatible Python virtual environment adhering to Airflow’s policies, and handles custom pip repository authentication.

3. Sumit Maheshwari - Engineer at Uber and PMC at the Apache Software Foundation
Experienced engineering leader with a passion for building high-performing teams and innovative products. Strong technical background with a demonstrated history of working in Big data technologies, Open source software, and scalable SAAS/PAAS products.

Rajat Srivastava, Senior Software Engineer at Uber
Sumit is a seasoned engineer in Big data space and involved with Airflow development since 2016. He is currently a Tech Lead in Uber’s workflow management team which handles thousands of pipelines and millions of task executions every day

Abstract: Sumit and Rajat will walk us through how Uber manages thousands of pipelines and executes millions of tasks each day via their custom workflow management tool called Piper.

Agenda:

• Introduction to Apache Airflow: A brief overview for both beginners and enthusiasts.
• Best Practices and Use Cases: Learn from industry experts about optimizing your workflows and real-world use cases.
• Networking and Knowledge Sharing: Connect with like-minded professionals, share your experiences, and expand your network in the Airflow community.
• Q&A Session: Get your queries addressed by our knowledgeable speakers.

Registration:
Attendance is free, but space is limited. To secure your spot, please RSVP by signing up here.
Who Should Attend?

• Data Engineers
• Data Scientists
• DevOps Engineers
• Anyone interested in workflow automation and orchestration

This Meetup is a fantastic opportunity to deepen your understanding of Apache Airflow, exchange ideas with industry peers, and gain practical insights from experienced professionals. We look forward to seeing you there!