IoT with Spark & Scala Spark Datasets

We have 2 talks for this meetup.

Agenda:
6:00 socialize, food and drink
6:30 IoT with Spark
7:15 Scala Spark Datasets

All of the Things - Exploring IOT with Spark, Kafka, and Kudu (Brandon Kvarda)

This talk will examine the IOT/maker ecosystem to get a view into what sort of things are being made by average people and companies alike. Data will be collected and analyzed in realtime from the public API on Particle.io, which contains a messaging platform for wi-fi and cellular microcontrollers.

Strongly typed Scala operations for working with Spark Datasets (John Nestor)

The addition of Datasets to Spark was a big step forward for Scala developers by providing static typing rather than the dynamic typing of DataFrames. Sadly, many operations on Datasets still require dynamic types where checking occurs at run-time. This talk describes a project is to add a new set of transforms on Scala Datasets that are fully static typed so checking of existence of field names and the types of fields can occur entirely at compile-time. The transforms operate on Datasets whose element type is a Scala case class. Each transform takes one or more Datasets and transforms them to produce a new Dataset. Code is available on GitHub (https://github.com/nestorpersist/dataset-transform)

Thank you Context Relevant for hosting this meetup and sponsoring food, Cloudera for sponsoring drink!