May Kafka Meetup

Join us for the May Apache Kafka Meetup at Yelp on Tuesday, May 17 from 6pm-8:30pm. If you're driving in, the closest parking garage (http://www.yelp.com/biz/minna-parking-lot-san-francisco) is located on Minna Street. If you're taking the BART, the closet stop (http://www.bart.gov/stations/mont/map) is Montgomery Street.

Please see the agenda and speaker information below. See you there!

Agenda

6:00pm: Doors Open

6:00pm - 6:30pm: Networking

6:30pm - 8:00pm: Presentations (See Below)

7:00pm - Doors Close

8:00pm - 8:30pm: Additional Q & A and Networking

First Talk

Speaker: Enrico Canzonieri, Yelp

Bio: Enrico works on the distributed systems team at Yelp, designing, building and maintaining NoSQL datastores and streaming infrastructure. He is one of the maintainers of Yelp’s Kafka deployment that moves terabytes of data and billions of messages every day. Enrico loves designing robust software solutions for stream processing that scale and building tools to make web developers’ interaction with the infrastructure as smooth as possible.

Title: Best Practices for a Kafka-based logging pipeline

Abstract: Kafka powers a variety of applications at Yelp, from the data pipeline to our logging infrastructure. The latter, our largest Kafka multi-regional deployment, is made up of clusters that move terabytes of data and billions of messages every day. Maintaining such a Kafka infrastructure can be challenging sometimes. In this talk we’ll share our experience operating Kafka clusters at scale, the best practices that we learnt along the way and the tools that help us every day keeping our Kafka stable and efficient.

Second Talk

Speaker: Usman Masood, Pipeline DB

Bio: Usman is the Chief Architect at PipelineDB and spearheads the design and development of PipelineDB and PipelineDB Enterprise. Prior to PipelineDB, Usman was at Locu where he hacked on backend infrastructure and lead their API and analytics efforts. Usman has a CS degree from MIT.

Title: SQL on Kafka

Abstract: Kafka is increasingly becoming the de facto standard messaging bus upon which organizations are building their stream-processing infrastructure. As a result, many of our users wanted the ability to seamlessly ingest data from Kafka.

We built an open-source extension called pipeline_kafka which lets users consume messages directly from Kafka and run continuous SQL queries on them in real-time. Recently, we also added the ability to push data back into Kafka.

In this talk we will discuss the technical details of pipeline_kafka, and how you can use it to glue PipelineDB and Kafka together and create sophisticated real-time data pipelines using only SQL.