Spark-a-thon -- Developing Spark Extensions (using Scala)

Time to learn how to develop extensions to Apache Spark (http://spark.apache.org/) using the best programming language Scala (http://www.scala-lang.org/).

Bring your laptop, JDK, sbt, and IntelliJ IDEA. Clone the Spark repo (https://github.com/apache/spark) or stay with Spark 1.6.2 (https://spark.apache.org/news/spark-1-6-2-released.html) if you like, but you should rather use Spark 2.0.0 RC2 (http://people.apache.org/~pwendell/spark-releases/spark-2.0.0-rc2-bin/) instead. The real fun is on master, though.

It is about extending the platform itself (not about writing Scala apps for Spark) so writing custom schedulers or streaming sinks is expected.

Use comments below to build the agenda first. What would you like to have covered?

Official language: Polish and Scala.