What we’re about
New to Apache Kafka®? Start with these free resources: https://cnfl.io/learn-ak-mu
This is an open community - if you want to present, host or contribute in other ways follow this link (http://cnfl.io/get-involved-mu) - first time speakers welcome!
This meetup is for your fellow event streaming enthusiasts!
The topics discussed at our events are all about event streaming, including Confluent Platform, Confluent Cloud, Apache Kafka®, Kafka Connect, streaming data pipelines, ksqlDB, Kafka Streams as well as stream processing, Security, Microservices and a lot more!!
Code of conduct: https://cnfl.io/code-of-conduct-welcome
Beyond this group, we also have the following resources to help you learn and develop your skills! See them here:
*The Meetup Hub*
Find recordings of previous meetups around the world and see upcoming dates for many more at the Meetup Hub
https://cnfl.io/meetup-hub-desc
*Ask The Community:*
-Forum;
This is a place for all the community to ask the tough questions, share knowledge and win badges :D http://cnfl.io/forum-desc
-Slack;
Join tens of thousands of community members in this community cross-collaboration tool, exchanging thousands of messages every month:
cnfl.io/slack
*Confluent Community Catalysts*
Nominate the next Community Catalysts (MVPs) and find out more here:
*Confluent Training and Certification discounts!*
Learn Apache Kafka® and become Confluent Certified (with 20% off your certification exam with the code MU2021CERT): https://cnfl.io/train-cert
--
Also here’s a gift: Get $200 worth of free Confluent Cloud usage every month for your first 3 months; (that could be $600 worth, without spending a single penny) (Ts & Cs apply) http://cnfl.io/mu-try-cloud
If you’re already a user, you can get an extra $60 on top with the code: CC60COMM
Head to http://cnfl.io/get-involved-mu if you have any questions, ideas, concerns or if you want to contribute in some way!
Apache Kafka®, Kafka, and associated open source project names are trademarks of the Apache Software Foundation. The Apache Software Foundation has no affiliation with and does not endorse, or review the materials provided here or at any of our Meetups.
Sponsors
Upcoming events
1
•OnlineFrom Apache Kafka® to Iceberg: High-Throughput Ingestion
OnlineIMPORTANT PLEASE RSVP AT: https://www.meetup.com/trino-americas/events/312487299/
Join us for a focused, 45-minute online meetup which we’ll walk through our architecture, the key design decisions, and the lessons learned in building for scale. The result: a fully managed, easy-to-operate system benchmarked at 100 GB/s throughput, delivering cost-effective, near-real-time ingestion with exceptional query performance.
From Kafka to Iceberg: High-Throughput Ingestion at Starburst
Real-time analytics is critical for modern businesses — but bridging the gap between fast-moving Kafka streams and query-ready lakehouse tables remains a complex challenge. At Starburst, we encountered this firsthand while ingesting our internal telemetry data into Iceberg tables. Existing solutions fell short, plagued by issues such as lack of exactly-once guarantees, limited scalability, head-of-line blocking, small file proliferation, and high operational overhead and cost.
This prompted us to rethink streaming ingestion from the ground up. Our goal was a fully managed, highly available system that makes data actionable within minutes, optimized for both performance and usability. We designed and built a custom Kafka-to-Iceberg ingestion service that directly writes data in Iceberg format with strong guarantees, minimal latency, and continuous data maintenance for optimal query performance. Along the way, we developed novel techniques — such as Iceberg-aware commit coordination and adaptive Kafka consumer assignment — to overcome typical ingestion bottlenecks and deliver best-in-class price/performance.
Speaker: Lakshmikant (Pachu) Shrinivas, Staff Software Engineer at Starburst, hosted by Lester Martin, Developer Advocate at Starburst.1 attendee
Past events
49
