Przejdź do treści

How Oracle manages Airflow in research & production & highlights on Airflow 2.0

Zdjęcie użytkownika Databand, an IBM Company
Hosted By
Databand, an IBM C.
How Oracle manages Airflow in research & production & highlights on Airflow 2.0

Szczegóły

**This is a free online meetup via Zoom - must RSVP to get the link.
*This event will be in English!

How Oracle manages Airflow as a service for research and production data teams:

In this meetup session, we will have a special guest speaker, Gita Ferber, Oracle’s Senior Software Developer. Gita will dive deeply into Apache Airflow at Oracle: Airflow for production and Airflow for research teams. Get a sneak peek into how Oracle built out their projects, so you can gain some tips and best practices to implement on your own Airflow project.

Airflow for production:
Oracle’s use case and why they use Airflow
Working with Airflow over time: how their architecture changed
Airflow in large scale: how they run Airflow cluster with over 30k tasks
Automating the CI/CD process

Airflow orchestration as a service for research teams:
The different requirements for research
Running heavy research pipelines using Airflow + DBND open source
Using multiple compute environments

Airflow then and now:

Evgeny, Databand’s co-Founder and CTO, will then discuss the pains that Airflow 1.0 has that Airflow 2.0 will solve.

Apache Airflow 2.0 highlights and deep dive:
Functional DAGs
Airflow Scheduler (HA and Performance )
Serialized DAGs and Versioned Dags
KubernetesExecutor and KEDA
Packaging
What's the current status? What’s missing? What’s Next?

Photo of Tel Aviv Apache Airflow Meetup group
Tel Aviv Apache Airflow Meetup
Zobacz więcej wydarzeń