Name: Design Patterns of Streaming Platforms
Start: 2019-09-30T18:00:00-04:00
End: 2019-09-30T20:00:00-04:00
Location: 114 5th Ave

*Note: The event is on 6th Fl.

Schedule:
6:00 - Doors & Food
6:30 - Talk 1
7:15 - Talk 2
7:45 - Wrap & Chat

*********
Talk 1: Real-time Processing At Scale with Flink

Speaker: Dean Shaw & Max McKittrick, Data Engineer @ Capital One

Abstract:
Potomac is a project at Capital One that collects clickstreams. In this talk, we will deep dive into the inner workings of Potomac's Flink Clusters. We will investigate topics such as: scaling, real-time deduplication, latency vs throughput, checkpointing and failure recovery, real-time vs micro-batch output, and more!

*********
Talk 2:

Speaker: Aravind Ramesh, Associate Staff Engineer @ DoubleVerify

Abstract:
DoubleVerify is a marketing measurement software company that works with some of the worlds largest brands to authenticate the quality of digital media, giving advertisers clarity and confidence in their digital investment. In this talk, we will deep dive into DV's internal streaming platform built on top of spark-streaming and discuss some useful design patterns. We will discuss topics such as autoscaling, failure recovery, application lifecycle management, and strategies to make your pipelines more testable and operable.

Keira Zhou

NYC Data Engineering & Science (Data Council)

Technology

Software Development

Big Data

Data Analytics

Data Mining

Design Patterns of Streaming Platforms

114 5th Ave

Share

NYC Data Engineering & Science (Data Council)

Design Patterns of Streaming Platforms

NYC Data Engineering & Science (Data Council)

Details

Members are also interested in