Data Engineering Meetup

Join Equal Experts as we talk you through the journey from Software (or QA) Engineer to Data Engineer! Gain insights from those who have walked the path, learn about the best ways to use TDD practices for Data Engineering, and grow your knowledge and career!

RSVP today!

Move to data engineering
The need for people who know how to get data in the right format and unify heterogeneous sources of information has grown significantly. Data is everywhere and data engineers are just becoming more in demand. You might think that as a developer, analyst, or power user, it's impossible to make the move to data engineering, but you'd be wrong.

TDD for Data Engineering
Many practices known to deliver improved outcomes in software delivery (for example test automation, CI / CD, agile methods) can also be successfully applied to Data Engineering. Focusing on testing, my presentation will highlight the benefits of applying Test Driven Development techniques to data transformations. This can prove extremely useful when writing logic to cater for non-trivial business or technical requirements. TDD also acts as a guide to design data transformations that are more decoupled and easier to test. I will provide examples written with DBT where the tests and the transformation logic is written entirely in SQL. But the concepts are applicable to other data stacks.

Speakers:
Gerhard van Deventer - Senior Data Engineer
Guillaume Belrose - Senior Data Engineer