Orchestrating Hybrid (Batch and Near Real-Time) Pipelines with dbt and Dagster

• Topic: Orchestrating Hybrid (Batch and Near Real-Time) Pipelines with dbt and Dagster
• Date: Thursday 29th May at 6.00 pm - 8.00 pm
• Networking: 6 pm - 6.30 pm and 7.20 - 8 pm
• Presentation Time: 6.30 pm - 7.20 pm
• (includes welcome, presentation 30 mins & community marketplace)
• Location: Mantel, Level 2/452 Flinders St
• Ticket: Free of cost, however, registrations and RSVP are required!
• Sponsor: ThoughtWorks, Mantel

In this talk, we’ll explore how to orchestrate hybrid data pipelines using dbt and Dagster, two modern tools. As organisations demand fresher data with governed transformations, combining dbt’s transformation logic with Dagster’s orchestration engine offers a powerful solution for data engineers.
Key topics include:
• How to structure hybrid pipelines that combine streaming ingestion with dbt transformations.
• Using Dagster’s software-defined assets and dbt’s transformation capabilities together
• Leveraging materialized views to capture event arrival or new data.
• Observability best practices with Dagster: monitoring, lineage, and backfill strategies.
• A real implementation walkthrough to test the actual performance live and demonstrate the orchestration.

Speaker Intro:

Hasan is a seasoned Data Engineer at ProcurePro, an Australian start-up company. He has both academic and industry experience in Data Engineering and Data Science. He began with a bachelor’s degree in Industrial Engineering, where he focused on optimising production lines and decision systems. This led to a master’s in Data Mining, where he explored metaheuristic algorithms and clustering techniques.
He started my career in the industry as a Data Scientist, building models and running experiments. Over time, he became more interested in the underlying systems that power those models, which led him to begin a PhD in Distributed Stream Processing in Edge Environments. These days, he is particularly passionate about writing on Data Engineering concepts and practices through his blog, Pipeline To Insights.