Apache Airflow Virtual Meetup


Details
Hello all Airflow-lovers in Bangalore/India.
It's been almost a year since we hosted our first Apache Airflow meetup in Bangalore. It was quite a day to see so many energetic faces and 4 great talks on Airflow.
Given the pandemic situation, we would be going fully virtual this time. Unfortunately, we gonna miss the physical presence, lunch, and swags but a large number of attendees should be able to square that feeling off.
==========
Schedule:
Talk 1: Workflow Platform at Cloudera
Abstract: The Workflow Platform at Cloudera is build upon Airflow. In this talk, Vivek will talk about various features of the platform which he & his team has built at Cloudera. Some of the key features are:
- Running Rest API for internal microservices using the custom-built operator.
- Running test jobs using the K8S operator.
- Running existing test Jenkins job execution.
- S3 Log streaming for k8S pods and S3 log integration for task logs.
- Notification for failed tasks.
- Re-running tasks with the support of reporting.
- Integration of airflow tasks reporting with internal Cloudera engineering dashboard.
Speaker: Vivek Pemawat is a Staff SWE from the Infrastructure team at Cloudera and leads the "Workflow Platform".
---------------
Talk 2: Airflow Artifacts deployment mechanism at Societe Generale
Abstract: This talk will cover the presentation of service offer which provide on-demand managed airflow instances using Airflow at Societe Generale. Also, covers the deployment strategy developed at SG to deploy DAGs, Python packages, System packages, etc.
The main topics of this talk would be the following:
- Service Offer overview.
- Dogfooding Airflow (use Airflow to provision Airflow).
- Image-Based Artifact deployment (DAGs, python, and system dependencies).
- "One Page UI".
Speaker: Jilan Kothakota - Lead Developer & Scrum master for Airflow team at Societe Generale.
---------------
Talk 3: Oozie to Airflow - A migration story
Abstract: Inmobi recently switched from Oozie to Airflow. Amit would talk about how they are using Airflow, what challenges they have faced and the features/plugins they have developed while migrating.
Speaker: Amit Khanna - Sr SWE at Inmobi
===============
Thanks,
Apache Airflow Bangalore Moderators.

Sponsors
Apache Airflow Virtual Meetup