Design Patterns of Streaming Platforms

*Note: The event is on 6th Fl.

Schedule:
6:00 - Doors & Food
6:30 - Talk 1
7:15 - Talk 2
7:45 - Wrap & Chat

*********
Talk 1: Real-time Processing At Scale with Flink

Speaker: Dean Shaw & Max McKittrick, Data Engineer @ Capital One

Abstract:
Potomac is a project at Capital One that collects clickstreams. In this talk, we will deep dive into the inner workings of Potomac's Flink Clusters. We will investigate topics such as: scaling, real-time deduplication, latency vs throughput, checkpointing and failure recovery, real-time vs micro-batch output, and more!

*********
Talk 2:

Speaker: Aravind Ramesh, Associate Staff Engineer @ DoubleVerify

Abstract:
DoubleVerify is a marketing measurement software company that works with some of the worlds largest brands to authenticate the quality of digital media, giving advertisers clarity and confidence in their digital investment. In this talk, we will deep dive into DV's internal streaming platform built on top of spark-streaming and discuss some useful design patterns. We will discuss topics such as autoscaling, failure recovery, application lifecycle management, and strategies to make your pipelines more testable and operable.