Skip to content

Data Kata: Unit testing in SQL and DBT for data validation

Photo of Lev Konstantinovskiy
Hosted By
Lev K.
Data Kata: Unit testing in SQL and DBT for data validation

Details

We meet online to improve our Data Craftsmanship skills.
Unusual to a data science gathering there is no leaderboard. ;)
We will play a "collaborative game" with the goals of solving a problem, having fun, and learning how to write better code, listen to our colleagues, be a supportive team, accept criticism and defend our ideas.

## Exercise:

This time we will use SQL and DBT to attempt the same validation exercise as we did last week in Pydantic. We will do in TDD way.

Yes, unit tests in SQL exist!

"Read data from a CSV and check data types, range of values, consistency between columns. Re-do in several frameworks - pandas, pydantic, Great Expectations, Pandera, mage, DBT, anything else."
See data and starting code in https://github.com/tmylk/data-kata/

### Format:
- Mob programming with a facilitator. We will all collaborate as one team, switching the person on the keyboard every 5 mins. (aka "Coding Dojo Randori Kata".)
- You don't need to have any previous experience with mob programming.

### Schedule:
- Intros - 20 mins
- Mob programming - 50 mins
- Interim Retrospective - 10 mins. What was great? What can be improved?
- Break - 10 mins
- Mob programming - 50 mins
- Final Retrospective - 10 mins
- Closing

### Things to note:
- We will be using SQL as our programming language. You don't need to have some experience with it to participate.
- We will work on Data as the broad domain of skills that we want to improve.
- We will use gitpod.io as our shared VS Code IDE work environment and MobTime timer https://mobtime.fly.dev to switch drivers.

Photo of Berlin Data Crafters group
Berlin Data Crafters
See more events