Stream Processing with Apache Kafka & Apache Samza

Welcome:

Welcome to the upcoming Stream Processing Meetup hosted by LinkedIn in Sunnyvale.

This meetup focusses on Apache Kafka, Apache Samza and related streaming technologies.

Location:
Our new Corporate HQ in Sunnyvale. We will be on the 5th floor of 580 Mary.

Agenda:

6PM: Doors open

6-6:35PM: Networking & Welcome

6:35-7:10PM: Real-time Indexing of LinkedIn’s Economic Graph (Almog Gavra, LinkedIn)

In this presentation, we will cover the basics of LinkedIn’s Search Engine indexing pipeline, focusing on how we leverage Kafka and Samza to ingest over 10K events per second of real time updates. Furthermore, we will examine how we made the system both flexible and horizontally scalable; our pipeline accepts different input system streams and supports both full and partial document updates, but remains agnostic to the type of document (e.g. member profile, job posting or company page)

7:15-7:50PM: Samza at Redfin: Using Streaming to Help Home Buyers and Sellers (Brian Hanks, Redfin)

Redfin sends millions of notifications per day to our customers to help them buy and sell homes. In a hot market, customers who learn about new homes first have an advantage, and we want to be faster than any of our competitors. I'll talk about how we developed a streaming system based on Samza to provide a low latency, resilient, horizontally scalable, high throughput system to send notifications to our customers. I'll also speak about some of the challenges we have combining data from multiple sources, how we use some Samza features (such as local store) in unusual ways, some other ways Samza is being used at Redfin, and suggest some features that we'd like to see in Samza.

7:55-8:30PM: Kafka Controller Internals (Onur Karaman, LinkedIn)

The Kafka controller plays a critical role in the functioning of a Kafka cluster. It is responsible for broker coordination, topic creation, partition reassignments, and more. We will deep-dive into the controller's internals, protocols, best practices on controller operations, monitoring, as well as some recent enhancements.

RSVP:
Please RSVP only if you plan to attend in person. Our facility can host 300 guests.

Parking & Entrance:
You can park in the uncovered parking that is along the building or in the parking garage located behind the building. There is also street parking available for overflow.

NDA:
You will need to sign a standard NDA when you enter the lobby.

Food & Drink:
Food & drink will be provided.

Can’t join us live?:
Live Stream will be available here: https://primetime.bluejeans.com/a2m/live-event/hhzcgqqj

Want to talk at a future meetup?:
Please contact us via the “Contact” button in meetup.com.