DBT (Data Build Tool) vs DLT (Delta Live Tables)

This time we will have an introduction to two ETL tools. DBT (Data Build Tool) and Databricks Delta Live Tables (DLT).

First up is Simon Whitley talking about DLT:
Automated ETL with Delta Live Tables
There is a lot of complexity in building an engineering framework - When should it run? How are dependencies managed? How does it track data quality & telemetry over time? Databricks have released Delta Live Tables to tackle just this - DLT is a prebuilt framework that allows you to describe sets of tables, in either SQL or Python, then it will build out the rest for you.

In this session, we will run through the core components of DLT, before building out a sample pipeline, complete with data quality measurement, inter-table dependencies and post-run logging. We will look briefly at some more complex topics, managing incremental updates and real-time datasets, before looking at the downsides of a black-box solution.

About Simon:
Director of Engineering & Owner of Advancing Analytics, a Microsoft Data Platform MVP and Databricks Beacon. Simon runs the popular Advancing Analytics Youtube channel, where he can often be found digging into the nerdy underbelly of Spark, Streaming and Delta Lakehouse architectures. When not ranting at a camera, Simon spends his time championing Data Engineering in the Microsoft space with a whole range of global clients, specifically uniting the worlds of "Big Data" and traditional Analytics Warehousing. When not tinkering with tech, Simon is a death-dodging London cyclist, a sampler of craft beers, an avid chef and a generally nerdy person.

Then Halvar Trøyel Nerbø will talk about dbt:
dbt - a game changer for Data Platforms and Data Warehouseing (everywhere)
dbt (data build tool) is an open source tool for helping you create data models running on your preffered data plattform / data warehouse (Snowflake, Databricks, Synapse, SQL database etc).

This session gives you the run through of what dbt is. How it works, where it fits in the Data Platform / Data Warehouse architecture and why you'd want to use it.

About Trøyel:
Data Platform Developer at Glitni
Trøyel digs creating data platforms, doing data engineering, DevOps and all that jazz! Trøyel enjoys himself the most when he gets to work the big picture as well as with both hands down into the 'porridge'. He loves to spin up resources using Terraform and Azure DevOps. Orchestrate and move in information in Data Factory and transform using it in dbt before throwing it into Power BI.