IN PERSON! Apache Kafka® Meetup Bangalore- Apr 2025

Hello everyone! Join us for an IN PERSON Apache Kafka® meetup on April 19th from 10:30AM, hosted by Nokia in Bangalore!

📍 Venue:
Nokia L5 Manyata Business Park
1, Silver Fir, L5, Manyata Tech Park Rd, DadaMastan Layout, Manayata Tech Park, Nagavara, Bengaluru, Karnataka 560045, India

***
Agenda:

• 10:30 - 10:35: Welcome and Introduction
• 10:35 - 11:15: Shruti Mantri, Software Architect
• 11:15 - 11:55: Kumar Keshav, Engineering Manager, Confluent
• 11:55 - 12:10: Break
• 12:10 - 12:50: Ashwin Pankaj, Staff Software Engineer II, Confluent
• 12:50 - 14:00: Lunch

***
💡 Speaker:
Shruti Mantri, Software Architect

Talk:
Queues for Apache Kafka

Abstract:
Event streaming is great but sometimes it’s easier to use a queue, especially when parallel consumption is more important than ordering. Wouldn't it be great if you had the option of consuming your data in Apache Kafka just like a message queue? For workloads where each message is an independent work item, you’d really like to be able to run as many consumers as you need, cooperating to handle the load, and to acknowledge messages one at a time as the work is completed. You might even want to be able to retry specific messages. This is much easier to achieve using a queue rather than a topic with a consumer group.

Apache Kafka 4.0 introduces Queues as part of their offering. This talk covers in detail the concept of Kafka Queues, how they are implemented, its features and also the potential gaps in the offering.

-----
💡 Speaker:
Kumar Keshav, Engineering Manager, Confluent

Talk:
Monitor Kafka Clients Centrally with KIP-714

Abstract:
Apache Kafka introduced KIP-714 in 3.7 release, which allows the Kafka brokers to centrally track client metrics on behalf of applications. The broker can subsequently relay these metrics to a remote monitoring system, facilitating the effective monitoring of Kafka client health and the identification of any problems.

KIP-714 is useful to Kafka operators because it introduces a way for Kafka brokers to collect and expose client-side metrics via a plugin-based system. This significantly enhances observability by allowing operators to monitor client behavior (including producers, consumers, and admin clients) directly from the broker side.
Before KIP-714, client metrics were only available within the client applications themselves, making centralized monitoring difficult. With this improvement, operators can now access client performance data, detect anomalies, and troubleshoot issues more effectively. It also simplifies integrating Kafka with external monitoring systems like Prometheus or Grafana.

This talk covers setting up ClientOtlpMetricsReporter that aggregates OpenTelemetry Protocol (OTLP) metrics received from the client, enhances them with additional client labels and forwards them via gRPC client to an external OTLP receiver. The plugin is implemented in Java and requires the JAR to be added to the Kafka broker libs.

Be it a kafka operator or a client application developer, this talk is designed to enhance your knowledge of efficiently tracking the health of client applications.

-----
💡 Speaker:
Ashwin Pankaj, Staff Software Engineer II, Confluent

Talk:
Kafka Connect Runtime

Abstract:
In this session we will walk through the internals of Kafka Connect Runtime covering the following topicsConnect cluster

Connector basics
Plugin loading
Connector Lifecycle
Task assignment and rebalance