Stream Processing with Apache Kafka & Apache Samza

Welcome:

Welcome to the upcoming Stream Processing Meetup hosted by LinkedIn in Sunnyvale. This meetup focuses on Apache Kafka, Apache Samza, and related streaming technologies.

Location: Unify Conference Room, LinkedIn Corporate HQ in Sunnyvale. We will be on the 1st floor of 950 W Maude Ave, Sunnyvale, CA 94085

Agenda:
6 PM: Doors open

6-6:35 PM: Networking

6:35-7:10 PM: Apache Samza 1.0: Recent Advances and our plans for future in Stream Processing
Prateek Maheshwari, LinkedIn

Apache Samza has reached a major milestone with its recent 1.0 release. In this talk, we step back and take stock of the major new features and enhancements in Samza 1.0. We also take a sneak peek at what's next on our roadmap. Both Stream Processing veterans and developers new to Stream Processing will discover useful new features to leverage for their applications.

7:15-7:50 PM: How and why we moved away from Kafka Mirror Maker to Brooklin- LinkedIn's story
Shun-ping Chiu, LinkedIn

For several years, LinkedIn has been using Kafka MirrorMaker as the mirroring solution for copying data between Kafka clusters across data centers. However, as LinkedIn data continued to grow, mirroring trillions of Kafka messages per day across data centers uncovered the scale limitations and operability challenges. To address such issues, we have developed a new mirroring solution, built on top of our stream ingestion service, Brooklin.
Brooklin Mirror Maker (BMM) aims to provide improved performance and stability while facilitating better management through finer control of data pipelines. In this talk, we will dive deeper into the challenges LinkedIn has faced with Kafka Mirror Maker, how BMM is designed to tackle these problems and our plans for iterating further on this.

7:55-8:30 PM: Puma - Stream Processing in Facebook
Speaker: Rithin Shetty, Facebook

In this talk, we’ll discuss ‘Puma’, a stream processing service at Facebook. Puma, developed internally at Facebook, is a mature stream processing system and has been in production for over 7 years. Users author their stream processing applications in a SQL like declarative language called Puma Query Language(PQL). Puma is used by hundreds of teams across Facebook for their stream processing needs. Due to its familiar SQL like syntax and support for rich testing environment, the application development is simple and fast. The user only needs to focus on the business logic while Puma takes care of the rest (provisioning jobs, handling variations in input traffic, load balancing, DR events, etc). Puma serves a wide range of use cases including accelerating batch pipelines, analyzing user behavior on Facebook, ingesting to various sinks, machine learning, etc. The talk will go into the overall Puma architecture and its main building blocks. It’ll also touch upon the SLA model and our learnings from running a stream processing service at scale.

8:30-9PM: Additional networking and Q&A

RSVP:
Please RSVP only if you plan to attend in person. Our facility can host 250 guests.

Parking:
You can park in the uncovered parking that is along the building or in the parking garage located next to the building.

NDA
You will need to sign a standard NDA when you enter the lobby.

Food & Drink:
Food & drink will be provided.

Can’t join us in person?:
Join us online - https://primetime.bluejeans.com/a2m/live-event/wgexhshj

Want to talk at a future meetup?
Please contact us via the “Contact” button in meetup.com.