What we’re about
Welcome to the Data Engineering Melbourne Meet Up Group!
Join us for engaging discussions and networking opportunities centered around data engineering in the vibrant city of Melbourne.
Our monthly events, held both in-person and in a hybrid format, provide a safe space for data enthusiasts to come together and share their knowledge and experiences.
At our meetups, you won't find any sales pitches or promotional activities. Instead, we focus on facilitating genuine connections and meaningful conversations. Whether you're a seasoned data engineer or just starting your journey, our events offer valuable insights, discussions, and learning opportunities.
We value the spirit of community, networking, learning, and empowerment. Our aim is to build a supportive and collaborative community where data professionals can learn from each other and grow together.
Our events are completely free of charge and open to everyone.
Come and join us at the Data Engineering Melbourne Meet-Up Group, we look forward to meeting you!
Join our slack channel here: Data Engineering Melbourne on Slack
Upcoming events (1)
See all- Orchestrating Hybrid (Batch and Near Real-Time) Pipelines with dbt and Dagsterlevel 2/452 Flinders St, Melbourne
- Topic: Orchestrating Hybrid (Batch and Near Real-Time) Pipelines with dbt and Dagster
- Date: Thursday 29th May at 6.00 pm - 8.00 pm
- Networking: 6 pm - 6.30 pm and 7.20 - 8 pm
- Presentation Time: 6.30 pm - 7.20 pm
- (includes welcome, presentation 30 mins & community marketplace)
- Location: Mantel, Level 2/452 Flinders St
- Ticket: Free of cost, however, registrations and RSVP are required!
- Sponsor: ThoughtWorks, Mantel
In this talk, we’ll explore how to orchestrate hybrid data pipelines using dbt and Dagster, two modern tools. As organisations demand fresher data with governed transformations, combining dbt’s transformation logic with Dagster’s orchestration engine offers a powerful solution for data engineers.
Key topics include:
• How to structure hybrid pipelines that combine streaming ingestion with dbt transformations.
• Using Dagster’s software-defined assets and dbt’s transformation capabilities together
• Leveraging materialized views to capture event arrival or new data.
• Observability best practices with Dagster: monitoring, lineage, and backfill strategies.
• A real implementation walkthrough to test the actual performance live and demonstrate the orchestration.Speaker Intro:
Hasan is a seasoned Data Engineer at ProcurePro, an Australian start-up company. He has both academic and industry experience in Data Engineering and Data Science. He began with a bachelor’s degree in Industrial Engineering, where he focused on optimising production lines and decision systems. This led to a master’s in Data Mining, where he explored metaheuristic algorithms and clustering techniques.
He started my career in the industry as a Data Scientist, building models and running experiments. Over time, he became more interested in the underlying systems that power those models, which led him to begin a PhD in Distributed Stream Processing in Edge Environments. These days, he is particularly passionate about writing on Data Engineering concepts and practices through his blog, Pipeline To Insights.
