5 Steps to Build streaming systems with Confluent's Neha Narkhede

6:00pm: Doors open
6:00pm - 6:30pm: Pizza, Drinks and Networking
6:30pm - 7:15pm: Neha Narkhede, Confluent
7:15pm - 7:45pm: Bill Bejeck, Confluent
7:45pm - 8:00pm - Additional Q&A & Networking

Speaker: Neha Narkhede
Neha Narkhede is co-founder and CTO at Confluent, a company backing the popular Apache Kafka messaging system. Prior to founding Confluent, Neha led streams infrastructure at LinkedIn, where she was responsible for LinkedIn’s streaming infrastructure built on top of Apache Kafka and Apache Samza. She is one of the initial authors of Apache Kafka and a committer and PMC member on the project.

Abstract:
Neha will cover a five-step adoption journey we observe organisations going through as they build streaming systems.

First, the organisation becomes aware of streaming as an architectural paradigm and builds a pilot system. After that pilot succeeds, streaming goes live with the first production system.

Third, the small production system spreads to an mission-critical application. Fourth, as the organisation gains competence with the new platform and comes to depend more and more on the advantages conferred by a streaming platform, that mission-critical application is expanded to a global use case, where streaming data is produced and consumed across geographies.

Finally, the organisation learns that a streaming platform is the proper core on which to layer all of its applications and services, and the “central nervous system” metaphor of comprehensive streaming adoption finally becomes true. Streaming adoption, having begun with an idea and a pilot project, has finally transformed a business and its entire information architecture.

Speaker: Bill Bejeck
Bill has been a software engineer for over 15 years. Currently, he is working at Confluent as an engineer on the streams team. He regularly contributed to Kafka Streams before starting with Confluent and continues to do so in his work today. Before Confluent, he worked on various ingest applications as a U.S. Government contractor using distributed software such as Apache Kafka, Spark and Hadoop. He has also written a book about Kafka Streams titled “Kafka Streams in Action” published in August 2018.

Title: Robust Operations of Kafka Streams

Abstract:
Apache Kafka added a powerful stream processing library in mid-2016, Kafka Streams, which runs on top of Apache Kafka. The community has embraced Kafka Streams with many early adopters, and the adoption rate continues to grow. Large to mid-size organizations have come to rely on Kafka Streams in their production environments. Kafka Streams has many advanced features to make applications more robust.

The point of this presentation is to show users of Kafka Streams some of the latest and greatest features, as well as some that may be advanced, that can make streams applications more resilient. The target audience for this talk are those users already comfortable writing Kafka Streams applications and want to go from writing their first proof-of-concept applications to writing robust applications that can withstand the rigor that running in a production environment demands.

Topics include:

Best practices on configuring a Kafka Streams application

Meeting production SLAs by minimizing failover and recovery times: configuring standby tasks and the pros and cons of having standby replicas for local state

Improving resiliency and 24×7 operability: the use of different configurable error handlers, callbacks and how they can be used to see what’s going on inside the application

Achieving efficient scalability: a thorough review of the relationship between the number of instances, threads and state stores and how they relate to each other

The talk will also present sample code so that attendees can view the concepts discussed in practice. Attendees of this talk will walk away with a deeper understanding of how Kafka Streams works, and how to make their Kafka Streams applications more robust and efficient.