Melbourne Apache Kafka® Ecosystem Meetup by Confluent

Hello everyone! Join us for an Apache Kafka® meetup on Oct 22nd from 5:30pm, in Melbourne!

The address, agenda, and speaker information can be found below. See you there!

📍Venue:
394 High Street Northcote, Victoria 3070

***
🗓 Agenda:

• 5:30pm: Doors open
• 5:30pm - 6:00pm: Pizza, Drinks, and Networking
• 6:00pm - 6:30pm: Paras Sitoula, Technical Lead, Tabcorp
• 6:30pm - 7:00pm: ​Evelyn Bayes
• 7:00pm - 7:30pm: Additional Networking

***
💡 Speaker:
Paras Sitoula, Technical Lead, Tabcorp

Talk:
Who Watches the Streams? Building an AI Kafka Operator

Abstract:
In this talk, I’ll explore whether an AI agent can help troubleshoot common Kafka problems. Instead of an engineer manually checking dashboards, consumer groups, topic settings and schemas, we’ll give an AI agent access to this information and see if it can work out what is going wrong. We’ll test it with real examples such as growing consumer lag, schema errors and a consumer falling behind the topic’s retention period. We’ll also look at where AI can be useful, where it can get things wrong, and how much control we should actually give an AI agent over a Kafka environment.

Bio:
Paras is a seasoned software/date engineer with significant experience in developing and optimizing realtime scalable systems. At Tabcorp, Paras leads a high-performing team, developing and optimizing real-time transaction monitoring systems utilizing technologies like Apache Kafka and Neo4j. With a robust background in both real-time and batch data processing, Paras has extensive experience across various platforms and tools, including AWS, Databricks, and Spark, which he leverages to deliver high-performance data solutions.

-----
💡 Speaker:
Evelyn Bayes

Talk:
Stuck On You: How a Kafka Performance Fix Took Down a Fortune 500

Abstract:
KIP-480 promised Apache Kafka a nifty performance boost. Instead of spreading keyless records evenly across every partition, the producer would stick to one partition until a batch filled, then move on to another.

And it worked! Bigger batches meant better compression, fewer requests and lower latency. Everyone wins.

Until they don't.

In this talk, we'll dig into a real major incident at a Fortune 500 company where that optimization turned into a trap. A subtle feedback loop caused producers to get stuck sending a disproportionate share of traffic to the slowest brokers, which made those brokers slower still until everything fell apart. We'll cover how the issue was diagnosed, what was actually happening inside the producer, and how it led to a new approach to partitioning in Kafka.