Spark Structured Streaming : Introduction and Internals

Structured Streaming is a new stream processing engine built on Spark SQL, which enables developers to express queries using powerful high-level APIs including DataFrames, Dataset and SQL.

In this meetup, we'll walk through the basics of Structured Streaming, its programming model and processing the data in Kafka with Structured Streaming. We'll use the techniques including Window operations, Watermarking and Triggers. The concepts will be illustrated using code examples.

Presenter: Revin Chalil. Big Data Development Engineer. Expedia.