At this edition of Apache Airflow meetup, we welcome you at the Google (https://cloud.google.com) office in San Francisco!
=======
Agenda
18:00 - Registrations, speed networking, food and drinks.
18:30 - Observability for Airflow at Sentry by Mike Clarke
19:15 - Running a data pipeline Infrastructure with 100s of DAGs at Credit Karma by Harish Gaggar & Jorge Lee
20:00 - Networking
Talks
1st talk
Abstract:
Moving from "works on my laptop" to "mission-critical data pipelines" presents unique challenges for data teams. We'll share our experience over the last 18 months of running production Airflow at Sentry and dive into the open-source tooling we've built. Join us to learn how we ship data pipelines with confidence.
Bio:
Mike Clarke is an engineering manager at Sentry, an open-source error monitoring tool that helps developers ship better software, faster. Mike's passion is bringing Sentry's monitoring solutions to data engineers & data scientists. Connect with Mike and leverage Sentry on your next project.
2nd talks
Abstract:
As data grow in complexity, the need of having flexible and scalable infrastructure is important for the business. Please join us to learn how we leverage Google Cloud Infrastructure to build highly scalable Airflow Celery Infrastructure framework to support hundreds of data pipeline in daily operation.
Bio:
Harish Gaggar at Credit Karma Engineering, responsible for managing Analytics Airflow data pipeline system. He has hands-on knowledge of designing large scale infrastructure on open source and cloud based applications.
Jorge Lee, Staff engineer at Credit Karma has hands-on experience backend, frontend, data engineering, devops, and anything else needed to build a production system. Passionate about code quality and all kinds of automation.