Robust Stream Processing with Apache Flink - Jamie Grier (Data Artisans)

Robust Stream Processing with Apache Flink

In this hands on talk and demonstration I'll give a very short introduction to stream processing and then dive into writing code and demonstrating the features in Apache Flink that make truly robust stream processing possible. We'll focus on correctness and robustness in stream processing.

During this live demo we'll be developing a realtime analytics application and modifying it on the fly based on the topics we're working though. We'll exercise Flink's unique features, demonstrate fault-recovery, clearly explain and demonstrate why Event Time is such an important concept in robust stateful stream processing and talk about and demonstrate the features you need in a stream processor to do robust stateful stream processing in production.

We'll also use a realtime analytics dashboard to visualize the results we're computing in realtime. This will allow us to easily see the effects of the code we're developing as we go along.

Some of the topics covered will be:

• Apache Flink

• Stateful Stream Processing

• Event Time vs. Processing Time

• Fault tolerance

• State management in the face of faults

• Savepoints

• Data re-processing

Bio

Jamie Grier is Director of Applications Engineering at data Artisans where he’s extremely excited to be able to help others realize the potential of Apache Flink in their own projects. Jamie has been working on stream processing for the last decade at companies such as Twitter, Gnip and Boulder Imaging. This has spanned everything from ultra-high-performance video stream processing to realtime social media analytics.