Kafka Rebalance and Kafka- Binlog Integration for Speeding Up Production

We are happy to share with you a new meetup we are hosting together with Wix Engineering!

And this time...

We've got two great talks about about Kafka!

Join Wix’s very own Ariel Moskovich, for a look inside a project where he and his teammates work on achieving the desired balance of partitions between Brokers, while also reducing data movement and overhead in the cluster. And the one and only Barak Luzon, from Taboola, will show us how one can serve 500K http requests per second with a blazing fast response time - all that with caching that’s also fresh and does not create the extra load of fetching.

// Schedule:

17:00 - 17:50 - Kafka Rebalance - Notes From the Field / Ariel Moskovich

17:50 - 18:40 - Kafka-Binlog Integration for Speeding Up Production / Barak Luzon

//Please notice:

• The meetup will be online and the talk will be in English.
• Link to the event will only be visible to people who RSVP.
• The online meetup will be held on February 3, 17:00, IL Time Zone, GMT +2:00

// Kafka Rebalance - Notes From the Field - Ariel Moskovich

Apache Kafka is a popular distributed event streaming platform used for data pipelines, streaming analytics, data integration, and mission-critical applications. After working with Kafka for a while, I encountered issues of having to add new capacity and then balance it several times after that. Which can be quite amplified in high-scale environments with lots of Brokers, Topics, and throughput.

Join me for a look inside our project where we work on achieving the desired balance of partitions between Brokers while also reducing data movement and overhead in the cluster.

// Bio
Ariel Moskovich is a dedicated production Engineer with more than 20 years of experience in high scale and live production environments. He is currently working as a System Architect at Wix.com, the leading platform to create, design and manage websites. Before that he worked as chief infrastructure architect at AppsFlyer and as Senior production Engineer at the most challenging environments of Outbrain and Orbitz. Ariel is married to a wonderful wife, and he has three demanding, yet sweet kids.

// Kafka-Binlog Integration for Speeding Up Production - Barak Luzon

How can you serve up-to-date information in a high-scale production system?
Taboola provides content recommendations to billions of people. We have hundreds of frontend servers across 7 regions. They need to withstand a massive load of 500K http requests per second, while maintaining a fast response time.
In order to do so, we rely heavily on in-memory caching. By design, caching has a tradeoff between data freshness and load.

But what if I told you that you can have fresh data without creating the extra load of fetching it?
Our journey started with a "fast track" we created using Kafka and MySQL binlog,
and ended with huge performance improvements across thousands of services with blazing fast information updates.

Join me to hear about how we solved an impossible problem with a straightforward solution.

// Bio

“Trying is the first step towards failure”, said the great Homer Simpson and I would add that “Failing is the first step towards success”.

Barak Luzon has been around software since 2006, in various companies and positions, from the C4 system for intercepting rockets, through E-commerce and Ad-Tech.

He is always keen to learn new technologies and test them to see how far to the edge he can take them. He practices this passion by day at Taboola with his team of rockstars, while by night he spends time on his second passion - brewing his own beer.