This month we welcome Denny Lee, Developer Advocate, Databricks (former SQLCAT DW and BI) to the Redmond Pass User Group meeting.
Denny will be talking about "Building a Data Engineering Data Pipeline with Delta Lake"
A common data engineering pipeline architecture uses tables that correspond to different quality levels, progressively adding structure to the data: data ingestion (“Bronze” tables), transformation/feature engineering (“Silver” tables), and machine learning training or prediction (“Gold” tables). Combined, we refer to these tables as a “multi-hop” architecture. It allows data engineers to build a pipeline that begins with raw data as a “single source of truth” from which everything flows. In this session, we will show how to build a scalable data engineering data pipeline using Delta Lake.
Delta Lake is an open-source storage layer that brings reliability to data lakes. Delta Lake offers ACID transactions, scalable metadata handling, and unifies streaming and batch data processing. It runs on top of your existing data lake and is fully compatible with Apache Spark APIs.
We look forward to seeing you there - bring a friend, or three! :)
Here's the agenda for your reference:
5:30 PM - 6:00 PM Networking with beverages and snacks
6:00 PM - 6:10 PM User Group updates
6:10 PM - 6:30 PM Icebreaker & Speaker Introduction
6:30 PM - 7:30 PM Featured presentation
7:30 PM - 8:00 PM Q&A and continued networking
Looking for the presentation from a previous monthly meeting? Visit our meeting archive: https://redmond.pass.org/MeetingArchive.aspx
DeNisha Malone, Tho Le, and the Redmond SQL Server User Group Leadership Team
- - - -
We've love to keep in touch! Visit http://pass.org to join our larger user group organization to get access to free learning content, database and BI experts, and join our local Redmond PASS User Group at http://redmond.pass.org and our sister group across the lake http://seattle.pass.org.