Beyond Tiered Storage: Serverless Kafka with No Local Disks


Details
Hello Streamers!
You are invited to attend a meetup on Wednesday, April 17th starting at 6:30pm MT hosted by our friends at Utah Data Engineering meetup group!
***PLEASE NOTE, VERY IMPORTANT***: If you are interested to attend this meetup, please register here. Registration for this meetup will not be open on this page, please visit the link below:
[https://www.meetup.com/utah-data-engineering-meetup/events/297700439/ ](https://www.meetup.com/utah-data-engineering-meetup/events/297700439/)
Schedule:
6:30pm-6:50pm pizza
6:50pm announcements
7:00pm speaker
7:45-8:30pm mingle
Separation of compute and storage has become the de-facto standard in the data industry for batch processing.
The addition of tiered storage to open source Apache Kafka is the first step in bringing true separation of compute and storage to the streaming world.
In this talk, we'll discuss in technical detail how to take the concept of tiered storage to its logical extreme by building an Apache Kafka protocol compatible system that has zero local disks.
Eliminating all local disks in the system requires not only separating storage from compute, but also separating data from metadata. This is a monumental task that requires reimagining Kafka's architecture from the ground up, but the benefits are worth it.
This approach enables a stateless, elastic, and serverless deployment model that minimizes operational overhead and also drives inter-zone networking costs to almost zero.
Join us on slack
COVID-19 safety measures

Beyond Tiered Storage: Serverless Kafka with No Local Disks