Apache Druid x Apache Kafka


Details
Join us for an Apache Druid® and Apache Kafka® meetup on July 23rd from 6:00pm in London hosted by Confluent!
06:00pm - Doors open, food and drink served
06:30pm - Welcome and introductions
06:35pm - “Kafka - No Rocks, Please”, Roman Kolesnev, Confluent
Using Kafka Streams With Alternative State StoresA popular data transformation library in the Apache Kafka® world - Kafka Streams - is bundled with a high-performance key/value store - RocksDB - with many advantages regarding near real-time, high throughput, and low latency processing.
But what if RocksDB key/value store semantics are just not enough? What if the use case requires features of other storage engines - document indexing, graph processing, or simply secondary indexes? Can we employ other embeddable storage engines or extend RocksDB-based state stores?Luckily, Kafka Streams supports custom state store implementations through a pluggable interface. Even better - we can delegate durability, data sharding, and failure recovery to Kafka Streams runtime. Of course, there are some associated restrictions as well. That, and more, is what we will explore in this session.
Roman Kolesnev is a Staff Customer Innovation Engineer at Confluent in the Customer Solutions & Innovation Division Labs team. His experience includes building business critical event streaming applications and distributed systems in the financial and technology sectors.
07:05pm - “Real-time plane-spotting”, Peter Marshall and Hugh Evans, Imply
Hidden from our eyes, aircraft in our skies are constantly transmitting data. Join us as we use some simple tech and the power of open source to fly through this data set. In this talk, see a Raspberry Pi, Apache Kafka, Apache Druid, and Grafana coming together for real-time data production, transport, OLAP, and interactive visualisation.
Peter Marshall is an award-winning speaker who leads developer relations at Imply, having worked with adopters and users of Apache Druid for five years. Peter has worked in enterprise architecture in both public and private sector for over 20 years. He has a BA in Theology and Computer Studies from the University of Birmingham.
Hugh Evans is a Developer Advocate at Imply, helping members of the Apache Druid community and bringing them together.
07:35pm - Break
07:35pm - “Indexing Kafka - Why point look ups matter and how we achieve them”, Tom Scott, Streambased
Have you ever wanted to know the count of messages on a topic? How about the count of messages with a particular property? Ever spent long hours searching for that poison pill message that killed your applications? All of these cases require looking at historical data in Kafka, something it is traditionally terrible at.
Join us in this talk as we explore historical Kafka data interactively and discuss the technologies that can make this possible.
Tom Scott is the founder of Streambased, Tom is building multi tenant, on-prem and cloud Kafka services to attack common Kafka pain points and break down barriers to starting your data journey.

Sponsors
Apache Druid x Apache Kafka