Stream Processing Meetup @ LinkedIn - June 15 2016

Welcome to the June 15th 2016 Stream Processing Meetup hosted at LinkedIn in Mountain View.

This meetup will focus on Apache Kafka, Apache Samza and related streaming technologies.

Agenda:

6PM: Doors open

6-6:30PM: Networking

6:30-7:15 PM: Scalable Complex Event Processing on Samza @Uber, Shuyi Chen

The Marketplace data team at Uber has built a scalable complex event processing platform to solve many challenging real time data needs for various Uber products. This platform has been in production for some time and it has proven to be very flexible to solve many use cases. In this talk, we will share in detail the design and architecture of the platform, and how we employ Samza, Kafka, and Siddhi at scale.

Bio: Shuyi Chen is a Senior Software Engineer at Uber working on building scalable complex event processing platform for Marketplace. He has years of experience in storage infrastructure, data infrastructure, Android/iOS development in Google and Uber.

7:15- 8PM: Air Traffic Controller: Using Samza to Manage Communications with Members, Cameron Lee & Shubhanhu Nagar

Air Traffic Controller (ATC) is a system built on top of Samza which is responsible for managing many of the communication channels LinkedIn has with its members. Historically, LinkedIn has been known for sending too many emails to members which are not useful to them. The goal of ATC is to improve on that experience by providing some common functionality that multiple use cases can leverage, such as dynamic batching of messages, delivery time optimization, and channel selection. This discussion will include some of the functionality ATC provides, how Samza was leveraged to build ATC, and some challenges that we faced while building ATC.

Bio: Cameron Lee has been a software engineer at LinkedIn for 3+ years. Some of his main projects at LinkedIn include third party content ingestion and Air Traffic Controller (topic of presentation). Before LinkedIn, Cameron studied EECS at UC Berkeley.

Shubhanshu Nagar is a Staff Engineer at LinkedIn working on building a delightful communications experience for our members. Some of his main projects include ATC (Air Traffic Controller) and Login/Authentication Infrastructure. Before joining LinkedIn, he worked at Samsung Mobile for couple of years and is a Carnegie Mellon alumni.

8-8:45PM: Tuning Kafka for low latency guaranteed messaging, Jiangjie (Becket) Qin

Kafka is well known for high throughput ingestion. However, to get the best latency characteristics without compromising on throughput and durability, we need to tune Kafka. We will share our experiences to achieve the optimal combination of latency, throughput and durability for different scenarios.

Bio: Jiangjie (Becket) Qin is from Distributed Data System team at LinkedIn. He has been working on open source project Apache Kafka. Jiangjie graduated from CMU with Master's degree in Information Networking from INI in 2014. Before that, he worked in IBM for four years, where he managed IBM zSeries platform for banking clients.

RSVP:

Please RSVP only if you plan to attend in person. Our facility can host 200 guests.

Parking:

Anywhere that you see an open spot!

Security:

You will need to sign a standard NDA when you enter the lobby of 2025.

Food & Drink:

Food & drink will be provided. Please RSVP so we can plan the right amount!

Can’t join us in Mountain View?:

We will be live-streaming this event as well as posting recordings of the presentations. We will post the live-stream URL in this group within 1-hour of the event.

Want to talk at a future meetup?

Please contact us via the “Contact” button in meetup.com.