NYC Airflow Meetup #3
Details
SecurityScorecard will be hosting the 3rd Airflow Meetup at SecurityScorecard HQ on July 26th, at 6:30pm.
Timeframe
- 6:30pm - 9:00pm
Agenda
-
How Betterment uses Airflow
-
Airflow Setup at Blueapron
-
Analytics at SecurityScorecard using Airflow
- How Betterment uses Airflow
Betterment is the largest independent online financial advisor with more than $9 billion in assets under management. The service is designed to help increase customers’ long-term returns and lower taxes for retirement planning, building wealth, and other financial goals. Learn more. (https://www.betterment.com/)
Steve Spagnola from Betterment will be talking about Airflow usage at Betterment. In his own words - "As Betterment has grown, we’ve needed to maintain a highly available and scalable data pipelining infrastructure. We’ve adopted Airflow to achieve this, building a distributed and elastic data processing cluster. We’ll discuss some of the challenges we’ve faced in our migration to Airflow, parts of Airflow we extended, and the deployment, testing and development processes we’ve found to be most effective."
Speaker : Steve Spagnola - Software Engineer, Betterment
- Airflow Setup at Blue Apron
Blue Apron has run Airflow in production for nearly two years. Being a growing enterprise with diverse problems, Blue Apron has decided to deploy isolated instances of Airflow that are owned by their respective teams. Joe Napolitano from Blue Apron will walk us through the Airflow image that is being used within Blue Apron, as well as an unopinionated base image for use by the community. Joe will present the project setup and communicate the benefits of using Blue Apron's base image in your org.
Speaker: Joe Napolitano - Sr. Data Engineer, Blue Apron
- Airflow at SecurityScorecard
SecurityScorecard has been rearchitecting their collection and analytics systems from the ground up into a big data platform over the past several months. Nick Matviko and Patrick D'Souza will walk through how Airflow fits into the overarching Analytics platform at SecurityScorecard and how Security analytics and conclusions are generated using Airflow and Spark.
Speaker:
Nick Matviko - Director of Engineering, Data Engineering
Patrick D'Souza - Sr Engineer, Data Engineering
