EtE Unity Catalog-enabled Databricks Lakehouse Deployment with Terraform

Welcome to the #6 meetup of the Databricks User Group Denmark!

We are organizing an in-depth session on a template project that leverages Terraform for deploying all necessary components for establishing a LakeHouse in Databricks. This setup includes the deployment of cloud components, using Azure as our example platform, along with account-level and workspace-level components within Databricks, specifically featuring a Unity Catalog-enabled workspace.
During this session, we will also showcase practical examples of the Unity Catalog governance model, implemented via Terraform, for one of our esteemed clients.
Finally, we close the meetup by presenting highlights of Data&AI Summit 2024.

Date and time: 19th June 2024 17:00-20:00.
Venue: Delegate, Hummeltoftevej 49, 2830 Virum
Agenda:
17:00 – 17:15 Doors open / welcome

17:15 – 18:30 Terraform-deployed Databricks LakeHouse template (Farzad Bonabi from Delegate)
- Terraform introduction
- Automated cloud deployment to host Databricks LakeHouse
- Automated Databricks account-level deployment
- Automated Databricks workspace-level deployment via:
- Fully tested ETL framework with SPETLR library
- ETL with Databricks Notebooks
- ETL with Databricks DLT
- Q&A

18:30 – 19:00 Sandwich / mingling

19:00 - 19.30 Unity Catalog governance model with Terraform for a customer (Simon Heisterkamp from Delegate)
- LakeHouse setup with Terraform
- Governance model

19:30 - 19:45 Data&AI Summit 2024 highlights (Nikodem Olsavsky)
- Hackathon
- Biggest news and announcements
- Talks Generative AI Topics
- Data Lakehouse topics
- Data governance topics (unity catalog)
- All other cool stuff and the vibe at the venue

19.45 -20.00 Wrap up and next meetup at twoday

Information:

• This agenda is subject to change.
• For now, the meeting is physical attendance only and it will not be possible to join online.
• Free parkings are available in the area (You can park at the guest parking spots in front of the main enterance in the upper level, or Delegate spots in the the lower level. Clever chargepoints are also available if you drive on an EV).