Jay Kreps: Apache Kafka and the Rise of The Stream Data Platform

What happens if you take everything that is happening in your company – every click, every database change, every application log – and make it all available as a real-time stream of well-structured data? Welcome to Apache Kafka! Understanding streaming platforms is an in-demand skill in the big data landscape, and this exciting technology is being used by the top names in tech and finance, including Uber, Twitter, Netflix, LinkedIn, Yahoo, Cisco, and Goldman Sachs.

Presenter:
Jay Kreps led the development of Kafka. Last year, he started a new company called Confluent, which provides a stream data platform powered by Kafka, adding advanced enterprise-level security, usability, and reliability. He was also the initial developer on other open source projects, such as Apache Samza and Voldemort. Prior to Confluent, Jay was the lead architect for data infrastructure at LinkedIn.

Abstract:

1. How the design and implementation of Kafka was driven by the goal of acting as a real-time platform for event data.
2. The challenges there were involved in scaling Kafka to hundreds of billions of events per day at Linkedin, supporting thousands of engineers, applications, and data systems in a self-service fashion.
3. How real-time streams can become the source of ETL into Hadoop or a relational data warehouse, and how real-time data can supplement the role of batch-oriented analytics in Hadoop or a traditional data warehouse.
4. How applications and stream processing systems such as Storm, Spark, or Samza can make use of these feeds for sophisticated real-time data processing as events occur.

Schedule:
6:30-7:00 – Reception
7:00-8:00 – Presentation and Q+A with Jay Kreps
8:00-8:30 – Networking and follow-up conversations

Location:
Risk Management Solutions
7575 Gateway Blvd
Newark, CA 94560

• Construct conference room -

Food and drink will be provided by Risk Management Solutions.