Introduction to the data build tool (dbt) w/ John Miner

Many people learn from the Jaffle Shop sample dbt project. However, the complexity of the project is quite simple. What I really wanted is a project that goes over the following dbt concepts:

· Seed raw data tables
· Transform raw data using views
· Support multiple databases with macros
· Capture slowly changing dimensions type
· Incremental loading of large data
· Use ephemeral tables as a place holder
· Use dbt utils package to make a date dimension
· Create a dimensional model for reporting
· Save pre-canned queries in analyses directory
· Test data quality using property/schema files
· Output custom data quality exceptions to a table
· Create documentation based on model design

In the end, I created my own sample dbt project using the Chinook database schema as the foundation. I must thank Luis Rocha for the OLTP database design and sample data.
This open source project was built using an Fabric SQL Database as the target. However, it can be built on any supported database using the correct dbt adapter and ODBC driver. At the end of this presentation, a data engineer will be ready to model data for any given database or warehouse.

Biography:

John Miner is a Senior Data Architect at Insight Digital Innovation helping corporations solve their business needs with various data platform solutions.

He has over thirty years of data processing experience, and his architecture expertise encompasses all phases of the software project life cycle, including design, development, implementation, and maintenance of systems.

His credentials include undergraduate and graduate degrees in Computer Science from the University of Rhode Island. Also, he has earned certificates from Microsoft for Database Administration (MCDBA), System Administration (MCSA), Data Management & Analytics (MCSE) and Data Science (MPP).

John has been recognized with the Microsoft MVP award ten times for his outstanding contributions to the Data Platform community.

When he is not busy talking to local user groups or writing blog entries on new technology, he spends time with his wife and daughter enjoying outdoor activities. Some of John’s hobbies include woodworking projects, crafting good beer, and playing a game of chess.