WeWork's use of Apache Kafka® On-Prem and rebalance protocol inside-out

Join us for an Apache Kafka® meetup on June 20th from 6:00pm, hosted by WeWork at a WeWork location in the city! The address, agenda and speaker info is below. See you there!

Agenda:

6:00pm - 6:30pm: Networking, Food and Drinks
6:30pm - 7:15pm: Boyang Chen, Confluent
7:15pm - 7:45pm: Alex Beltran, WeWork
7:45pm - 8:30pm: Additional Q&A and Networking

-----

First Talk

Speaker: Boyang Chen

Bio:
Boyang is an infrastructure engineer at Confluent, where he works in Kafka Streams team to build next generation streaming platform on top of Apache Kafka. Previously Boyang worked at Pinterest as a software engineer in ads infrastructure team. He worked on various ads realtime challenges and rebuilt the whole budgeting and pacing pipeline to a fast and robust stage with concrete revenue gain and business impact.
(Linkedin: https://www.linkedin.com/in/boyangc/)

Title: Rebalance Protocol inside-out: a Developer Perspective

Abstract:
Rebalance protocol is the coordination algorithm to process data in a dynamic scaling fashion for general Kafka clients, including Consumer, Connect and Streams. If you have ever been in the shoes of a Kafka application developer, you should have heard of this term and even been bitten by it some times. In fact, it is one of the known performance killers for large member group or state heavy application as of today.
In this talk, we will deep dive into this protocol, demo some troubleshooting experience and introduce two most recent improvements on top: static membership and incremental rebalancing. After the talk, you would gain a deeper understanding of the rebalance protocol, which shall boost your Kafka development velocity right away!

---

Second Talk

Speaker: Alex Beltran

Bio:
Alex Beltran has spent his career focusing on IoT devices in smart buildings. He has worked towards improving human comfort while reducing energy cost within buildings. Recently he has joined the We company to provide a platform to enable smart physical spaces.

Title: Apache Kafka® On-prem at WeWork

Abstract:
Our talk will focus on the adoption of Kafka connect and streaming for our On-premise IOT needs. One of the use cases of physical access needs interaction with devices from disparate protocols such as AMQP, MQTT. The family of open-source Kafka connectors and integrated real-time streaming helped us bootstrap quickly and get the system up and running. With the need for persistent on-prem storage, K-tables semantics has been handy. Our early end to end benchmark tests revealed we could process the anticipated load and meet the latency needs. The Strimzi Kubernetes operator for Kafka provided us the operation tools for easy deploy of the Kafka ecosystem on-prem. As we took this to a production deployment in our buildings it was crucial to invest in observability and integrations with vault for externalizing secrets for Kafka connect. We will share in detail the process of instrumenting Kafka streams with interceptors to provide meaningful insights on how this system is performing for our use case.

-----

KAFKA SUMMIT SF 2019: 30th September til the 1st October -

We are able to offer you a 25% discount on the standard priced ticket for Kafka Summit San Francisco (September 30th & October 1st). To redeem it, please go to bit.ly/KSummitMeetupInvite, click ‘register’, select ‘Conference Pass’ and enter the community promo code “KS19Meetup”.

Don't forget to join our Community Slack Team! https://launchpass.com/confluentcommunity

Want to speak or host? community@confluent.io

NOTE: Please do not sign up for this event if you are under 18.