Kafka and Stream Processing Meetup at LinkedIn

Hi Everybody!

The Linkedin SRE team in Bangalore is excited to host the latest edition of the Kafka and Stream processing meetup.
Given that a significant part of the community is well-versed with building and operating Kafka, our talks in this iteration of the meetup will focus on the learnings we've had at LinkedIn from administering Kafka at scale over the past few years. We're also flying in an engineer from our stream processing organisation in Sunnyvale, CA to talk about the various use cases we've had for Apache Samza.

The meetup will be hosted at the Linkedin office and we are excited to see you all there!!

Schedule and Event Details:

Please carry a govt. ID proof

Registration and seating: 9:45 AM to 10:15 AM

Welcoming attendees and introducing speakers: 10:15 AM to 10:30 AM

First talk: 10:30 AM to 11:15 AM
Monitoring Kafka: What you should and shouldn’t monitor – Ratish Ravindran (https://www.linkedin.com/in/ratishravindran/)
Is your Kafka cluster growing larger by the day? Is your list of monitored metrics growing longer? Is your Kafka setup getting harder to administer? If the answer to any of these questions is true, then this talk is for you!

Questions/Break: 11:15 AM to 11:30 AM

Second talk: 11:30 AM to 12:15 PM
Autonomous Workload Rebalancing in Kafka – Indrajeet Kumar (https://www.linkedin.com/in/indrajeet-kumar-a8412b25/)
Kafka at Linkedin processes over 3 Trillion messages a day with over 2000 Kafka brokers. At such a scale, maintaining balanced workload on kafka clusters as they go through irregular traffic patterns and hardware failures is a daunting task. SREs at Linkedin expend significant time and effort in handling these curveballs and making sure the hardware resources are utilized evenly - Intelligent automation is crucial to scale further. This talk outlines Linkedin’s approach to solving this problem with the help of Kafka Cruise Control.

Questions/Break: 12:15 PM to 12:30 PM

Third Talk: 12:30 PM to 1:15 PM
Stream Processing at LinkedIn with Apache Samza – Abhishek Shivanna (https://www.linkedin.com/in/ashivanna/)
Batch processing tools like MapReduce are good at processing and analyzing large amounts of data collected over a period of time - but sometimes in order to provide a rich experience to your users we need to process data as it arrives – in real-time, within seconds or lesser. Real-time stream processing with Apache Samza has been powering applications in production, both for internal analytics and for data products used on the live site for over 4 years. This talk will give you a brief introduction of stream processing with Samza and the kind of problems you can solve with it. We will also take a look at practical case-studies of large scale stream processing applications at LinkedIn.

Questions/Lunch - 1:15 PM onwards

Venue : LinkedIn Technology Information Private Limited, 7th floor, Tower A, Global Technology Park, Devarabisanahalli , Outer ring-road, Bangalore 560103

Date: Saturday 23rd June 2018

Best Regards,

LinkedIn SRE Team