IN PERSON: Apache Kafka® Meetup Munich - Apr 2024

Join us for an Apache Kafka® meetup on Tuesday, April 23rd from 6:00pm in Munich hosted by apinity and supported by bakdata!

📍Venue:
apinity GmbH, Streitfeldstraße 17-19, 81673 München, Germany

***
🗓 Agenda:

• 6:00pm: Doors open
• 6:00pm - 6:30pm: Food, Drinks and Networking
• 6:30pm - 7:00pm: Bruno Cadonna, Software Developer, Confluent
• 7:00pm - 7:30pm: AJ, Backend Engineer, apinity & Ramin Gharib, Software and Data Engineer, bakdata
• 7:30pm - 8:00pm - Additional Q&A & Networking

***
💡 Speaker One:
Bruno Cadonna, Software Developer, Confluent

Talk:
Versioned State Stores in Kafka Streams

Abstract:
Versioned key-value stores in Kafka Streams enhance stateful processing capabilities by allowing users to store multiple record versions per key, rather than only the single latest version per key as is the case for existing key-value stores today. Storing multiple record versions per key unlocks use cases such as true temporal stream-table joins: when an out-of-order record arrives on the stream-side, Kafka Streams can produce the correct join result by looking "back in time" for the table state at the timestamp of the stream-side record. Foreign-key joins will see similar benefits, and users can also support custom use cases in their applications by running interactive queries to look up older record versions from versioned state stores, or by using them in custom processors.

This talk will introduce versioned state stores starting from the basics, discuss the stream-table join use case as motivation, operational considerations for users who'd like to use them, briefly touch on implementation in doing so, and also cover the timeline for when various pieces of functionality can be expected.

Bio:
Bruno is a committer and PMC member for Apache Kafka and a Software Developer at Confluent. At Confluent he has worked on Kafka Streams, Java Kafka clients, and ksqlDB. Prior to Confluent he was a Software Developer at SAP where he worked on a distributed in-memory computing engine for Big Data. Bruno holds a Ph.D. in Computer Science from the Free University of Bozen-Bolzano, Italy, and he held a postdoc position at Humboldt-Universität zu Berlin. His academic research focused on data stream and event processing. Bruno loves well-written code and hates inline comments.

***
💡 Speaker Two & Three:
AJ, Backend Engineer, apinity & Ramin Gharib, Software and Data Engineer, bakdata

Talk:
Apache Kafka’s Versioned State Store in charge of API analytics and consistent metering

Abstract:
Services and tools created by apinity extend API management with modern API publishing capabilities, business relationships, and flexible consumption modes.

Under the hood, it is a microservice architecture. Message queues enable asynchronous communication. Apache Kafka and Kafka Streams facilitate API event analytics to provide near real-time usage insights and metering to report API consumption for invoicing.

API offerings and business relationships are subject to change over time. For creating billing data, the system has to reflect the correct state at any point in time, including the past. Therefore, maintaining consistency across time and development cycles for seemingly simple filtering, grouping, and aggregation is more
challenging than it appears at first glance. Consider late arrivals, the necessity to partially reprocess data, or disaster recovery.

Together with bakdata, we will present our architecture and data pipelines using Kafka Streams’ versioned state stores that deal with evolving, late-arriving domain reference data. We will walk you through our journey and demonstrate our implementation that handles late arrivals on either side of a stream join.

Bios:
Ajinkya Khalwadekar (AJ) is a backend engineer at apinity. With a background in building cloud-based micro service architectures he’s now applying his experience to stream data processing.

Ramin Gharib is a software and data engineer at bakdata. He works on building data pipelines and distributed processing with Apache Kafka. He enjoys DevOps and the deployment of distributed processing to the cloud. Ramin is an active open-source developer contributing tools for Apache Kafka ecosystem.

***
DISCLAIMER
BY ATTENDING THIS EVENT IN PERSON, you acknowledge that risk includes possible exposure to and illness from infectious diseases including COVID-19, and accept responsibility for this, if it occurs.
NOTE: We are unable to cater for any attendees under the age of 21.
***
If you would like to speak or host our next event please let us know! community@confluent.io