Berlin Apache Airflow Meetup at GetYourGuide!

Name: Berlin Apache Airflow Meetup at GetYourGuide!
Start: 2025-09-17T18:00:00+02:00
End: 2025-09-17T21:00:00+02:00
Location: GetYourGuide Office

Hosted By

Airflow M.

Details

Join fellow airflow enthusiasts and leaders at GetYourGuide's Offices for an evening of engaging presentations, delicious food and drinks, and awesome swag!

PRESENTATIONS

Airflow & dbt: dynamic creation of inter-dag sensors with cosmos

Speaker: Giovanni Corsetti Silva, Data Core Engineer at GetYourGuide

Apache Airflow is the go-to platform for data orchestration, while dbt is widely recognized for analytical data transformations. Using astronomer-cosmos library, integrating dbt projects into Airflow becomes straightforward, allowing each dbt model to be transformed into individual tasks or task groups equipped with Airflow features like retries and callbacks.
However, organizing dbt models into separate Airflow DAGs based on domain or user-defined filters presents challenges in maintaining dependencies across these distinct DAGs. Ensuring that downstream dbt tasks only execute after the corresponding upstream tasks in different DAGs have successfully completed is crucial for data consistency—yet this functionality is not supported by default.
Join GetYourGuide as we explore our method for dynamically creating inter-DAG sensors in Airflow using Astronomer Cosmos for dbt. We will show how we maintained dbt model dependencies across multiple DAGs, making our pipeline modular, scalable, and robust.

Safeguarding your workflows in Apache Airflow

Speaker: Robert Bemmann, Senior Data Engineer at GetYourGuide

This presentation highlights practical validation techniques to prevent misconfigurations and enhance reliability in Apache Airflow environments. We cover two key safeguards: validating that sensors are correctly tied to their upstream tasks, and checking that critical DAGs have PagerDuty alerting enabled. Both validations are automated and integrated into our CI/CD pipeline using GitHub Actions, ensuring continuous enforcement and early detection of potential issues before deployment. In addition, we’ve implemented a solution to track Service Level Objectives (SLOs) for our DAGs, enabling better insight into reliability and performance over time. These checks form a practical defense against operational blind spots, promoting workflow reliability and robust incident response. Join us as we uncover practical strategies to streamline workflow monitoring and enhance system resilience using Apache Airflow's robust capabilities.

From Chaos to Cosmos: Simplifying DAG Creation in Apache Airflow