Operationalizing data pipelines with StreamSets - Jowanza Joseph

Agenda

6:00 Food and socializing

6:30 Introductions, Sponsors, Presentation and Discussion

Abstract: Streaming platforms like Amazon Kinesis and Apache Kafka have become increasingly popular for managing event driven and big data applications. These frameworks allow for a wide array of new and exciting architectural paradigms but come with many operational difficulties and considerations. StreamSets is a data flow platform built to work with streaming workflows as well as an array of internet and storage protocols. This talk focuses on how to use Stream Sets to reduce the complexity of building a distributed streaming platform. I’ll walk through the genius of using StreamSets, its architecture and the problems it helps us solve at One Click Retail. I will also demo a streaming analytics pipeline built with StreamSets and Amazon Kinesis.

Bio: Jowanza Joseph is a senior data engineer at One Click Retail. Jowanza’s work focuses on building a distributed data platform on top of Apache Mesos. He’s currently working on a book about Apache Spark to be published the first quarter of 2018.

7:30 Hard stop and cleanup; all help welcomed :)