Apache Kafka Use-Cases: Data Consumption & Data Integration

Join us on Wednesday, June 19, for a meetup focused on Kafka Consumers and Avro schema optimization. We have prepared two technical talks for all those interested in the mechanics and integration of data streaming. For starters, Danica Fine from Confluent will discuss the Kafka Consumer's journey from polling to data retrieval. This talk aims to provide a clear understanding of consumer request handling and performance metrics. Following this, Alexandru Dobrinescu of ING Hubs Romania will cover the optimization of Kafka streaming by enhancing Avro schemas. The session will explore strategies for data integration across various systems, aiming to improve data architecture efficiency.

Let's further meet our speakers and the topics they will tackle:

First talk: The Kafka Consumer: An Unexpected Journey of Data Consumption
Once your data is stored on your Apache Kafka® cluster, the next step is to consume that data and do something interesting with it. Enter: Kafka Consumers. We all know how to set up a Kafka Consumer to poll data… but do you know how a consumer fetches the data from the cluster? Let’s find out!
Every call to consumer.poll() is translated into a low-level request which is sent along to the brokers for fulfillment. In this session, we’ll join Kafka Consumers as they embark on their epic adventure to consume your data. First, see how these clients band together in a single fellowship and follow the guidance of their consumer group coordinator. Then, follow a request from an initial call to poll(), all the way to disk, and back to the client with your data via the broker’s final response. Along the way, we’ll explore a number of client and broker configurations that affect how these requests are handled and discuss the metrics that you can monitor to keep track of every stage of the consumer life cycle.
By the end of this session, you’ll know the ins and outs of your Kafka Consumer requests, making your next debugging or performance analysis session a breeze.

About the speaker:
Danica Fine is a Staff Developer Advocate at Confluent where she helps others get the most out of their event-driven pipelines. Prior to this role, she served as a software engineer on a streaming infrastructure team at Bloomberg where she predominantly worked on Kafka Streams- and Kafka Connect-based projects. Her expertise in streaming systems has taken her to a number of conferences and speaking engagements over the years, giving her the chance to express her love of Kafka to anyone who will listen. Danica is committed to increasing diversity in the technical community and actively serves as a mentor to a number of women in tech. She can be found on Twitter, tweeting about tech, plants, and baking @TheDanicaFine.

Second talk: Optimizing Kafka Streaming: Enriching Avro Schemas for Seamless Data Integration
Delve into the critical process of enriching Avro schemas within Kafka streaming environments to optimize data ingestion for data lakes and data marts. This session explores the boundaries of new and old technologies as we look at the significance of metadata augmentation. Together we will identify the missing elements needed to ensure seamless compatibility across diverse data repositories and answer the question ‘how can an organisation unlock the full potential of their streaming data architecture?

About the speaker:
Alexandru-Ioan Dobrinescu, Chapter Lead Engineer at ING Hubs Romania
Alex joined ING Hubs Romania in 2021 as DevOps engineer in the Data Lake ecosystem and now leads a chapter focused on Big Data solution development.
Even in his spare time, Alex is into deploying clusters for personal use and balances it out with other favourite activities, like gaming, biking, and traveling with his family.

Agenda
18:30 - 19:00 - Welcome & Networking
19:00 - 19:50 - Danica Fine - The Kafka Consumer: An Unexpected Journey of Data Consumption
19:50 - 20:30 - Alexandru Dobrinescu - Optimizing Kafka Streaming: Enriching Avro Schemas for Seamless Data Integration
20:30 - 21:30 - Networking

The event is hosted by ING Hubs Romania. Meet us all on Wednesday, June 19, at their curious office (174-176 Calea Victoriei).
This is an in-person event, presentations will be conducted in English. Please RSVP to secure your spot.

See you there!