*Note: The event is on 6th Fl.
6:00 - Doors & Food
6:30 - Talk 1
7:15 - Talk 2
7:45 - Wrap & Chat
Talk 1: Real-time Processing At Scale with Flink
Speaker: Dean Shaw & Max McKittrick, Data Engineer @ Capital One
Potomac is a project at Capital One that collects clickstreams. In this talk, we will deep dive into the inner workings of Potomac's Flink Clusters. We will investigate topics such as: scaling, real-time deduplication, latency vs throughput, checkpointing and failure recovery, real-time vs micro-batch output, and more!
Speaker: Aravind Ramesh, Associate Staff Engineer @ DoubleVerify
DoubleVerify is a marketing measurement software company that works with some of the worlds largest brands to authenticate the quality of digital media, giving advertisers clarity and confidence in their digital investment. In this talk, we will deep dive into DV's internal streaming platform built on top of spark-streaming and discuss some useful design patterns. We will discuss topics such as autoscaling, failure recovery, application lifecycle management, and strategies to make your pipelines more testable and operable.