What we’re about
The Utah Data Engineers Meetup showcases presentations for data engineers - data pipelining, streaming, architecture, governance, and much more.
Join us the 3rd Wednesday of every month.
If you would like to be added to our slack channel "Utah Data Engineering", please DM or email one of the organizers your preferred email address and we'll send an invite.
Upcoming events (2)See all
- Real world pipelines -Marc KeelingRecursion, Salt Lake City, UT
We will be demonstrating:
- How to set up a real-world data pipeline using MySQL, Airbyte, Clickhouse, and Dagster (maybe some dbt as well).
- A real project that simulates a real-world scenario, allowing you to see what it might take to solve practical data engineering challenges.
- By the end, you will have an overview of how to design, build, and run a data pipeline.
- I have used this pipeline professionally to process and analyze billions of rows of data in production environments
Join us on slack
- Beyond Tiered Storage: Serverless Kafka with No Local Disks - Richie Artuol276 E 12200 S, Draper, UT
Separation of compute and storage has become the de-facto standard in the data industry for batch processing.
The addition of tiered storage to open source Apache Kafka is the first step in bringing true separation of compute and storage to the streaming world.
In this talk, we'll discuss in technical detail how to take the concept of tiered storage to its logical extreme by building an Apache Kafka protocol compatible system that has zero local disks.
Eliminating all local disks in the system requires not only separating storage from compute, but also separating data from metadata. This is a monumental task that requires reimagining Kafka's architecture from the ground up, but the benefits are worth it.
This approach enables a stateless, elastic, and serverless deployment model that minimizes operational overhead and also drives inter-zone networking costs to almost zero.
Join us on slack