Airflow Meetup @ Paypal

Hosted by Bay Area Apache Airflow Meetup

Public group
Bay Area Apache Airflow Meetup
Bay Area Apache Airflow Meetup
Public group

Paypal - Town Square ( Room: Round Table )

2161 N 1st St · San Jose, CA

How to find us

Town Square is a building on east side of the Paypal's main campus entrance.

Location image of event venue


Hello Airflow community,

Paypal is hosting Airflow meetup !!

Here are more details.

Presentations & Talks:

1) Paypal using Airflow to remediate Live Hadoop cluster to keep it highly available.

Speaker: Jayesh Senjaliya, Software Engineer @ Paypal

Description: Paypal has one of the biggest Hadoop clusters in the industry. ~ It's challenging to keep up with sporadical hardware failures to keep the cluster highly available. Also its humanly impossible to manually take care of bad nodes and replace with new ones at such a large scale. So Paypal takes a workflow based approach to automate the entire process to remediate nodes on live Hadoop clusters.

2) Predictive Analytics with Airflow and PySpark

Speaker: Russell Jurney, Principal Consultant, Data Syndrome

Description: Russell will talk about how to wrangle PySpark into a full blown analytics application using Airflow.

3) Advanced Data Engineering patterns with Apache Airflow

Description: Analysis automation and analytic services is the future of data engineering! Apache Airflow's DSL makes it natural to build complex DAGs of tasks dynamically, and Airbnb has been leveraging this feature in intricate ways, creating a wide array of services as dynamic workflows. In this talk, we'll explain the mechanics of dynamic pipeline generation using Apache Airflow, and present advanced use cases that have been developed at Airbnb.

Speaker: Maxime Beauchemin, Airbnb

4) Panel discussion on 2017 Roadmap for Apache Airflow

Speakers: Apache Airflow PMC members


Schedule :

5:30-6:30 Meet, greet, eat, and drink

7:00-8:00 Talks

8:00-8:30 Q&A, Updates, Roadmap

8:30-9:00 Community open-mic, updates, etc.