Plot Twists in Data: DBT’s Approach to Movie Data Transformation

Title - Practical Data Engineering in a Movie Recommendation System Using DBT

Details:
Movie recommendation systems are programs that suggest movies to users based on the user's ratings or preference towards a specific genre or type of movie/series. These are the suggestions that come up when you open YouTube or Netflix on the home screen; usually under sections like "What should you watch next?" OR “Because you watched…”, and these predictions are usually built using AI/ML models which are trained on massive data. In this session, we will be building a data pipeline using DBT for a movie recommendation system that can provide data to those AI/ML systems.

Why DBT?
DBT plays the T (transform) part in the ELT/ETL part of data engineering. The tool allows you to write all your transforms in SQL which enables us to create a repository and check in our transforms via a version control. Since the tool also has a CLI, we can trigger different DBT actions from CI or other cloud schedulers. As the DBT tool in itself does not handle data we don't have to worry about resource allocation and is handled by the data warehouse. If you're a data engineer or a software engineer, all you need is a little bit of knowledge of SQL to get started with DBT.

What’s in it for you?
You will learn about DBT and you should be able to apply it to an existing data project. Based on your learnings from the session, you should be able to compare DBT with the other tools that you're already using in your projects. In addition to the above, this session is not limited to learning about DBT or any new tech, but also networking with the other techies alongside fueling your curiosity.
What are we going to do?
We will be live-coding a data pipeline for a movie recommendation system using the movie database and thus learn the basic features of DBT in an interactive way. Whether a data engineer or not or a software engineer, there's something for you to learn in this talk.
Recommended prerequisites:
- Basic SQL knowledge
- Basic Shell
- Basic templating language knowledge
[Prerequisites are not compulsory]
It's okay even if you're just a software engineer not knowing any of the above. We will have a gentle introduction to each from the above.
We are inviting all enthusiasts & curious minds alike for this and a series of more such events.
This will be a quick session with a short break and refreshments in between.
We will be providing snacks and refreshments to all the attendees during the event.
Agenda
10:30 am - 11:00 am - Tea & Introduction
11:00 am - 11:30 am - Equal Experts Overview
11:30 am - 12:30 pm - Expert Talks on DBT
12:30 pm - 12:45 pm - Q & A session
12:45 pm - 1:30 pm - Networking over Lunch