Skip to content

Spark Summit East 2015 Warmup meetup

Photo of François Le Lay
Hosted By
François Le L.
Spark Summit East 2015 Warmup meetup

Details

We will be hosting a warmup session ahead of the summit with two presentations confirmed. Also, we would like to facilitate networking among attendees so all ideas are actually welcome !

Extending SparkR: SparkSQL, DataFrames, and MLLib in the SparkR project (Chris Freeman / Alteryx)

Given the ever-increasing popularity of both Apache Spark and the R language in the data science world, it's only natural to want to combine the two into a framework that exposes the power of Spark in a setting that R users will find familiar. Over the last year, the contributors to the SparkR project have been working on doing just that. SparkR is an open-source R package that provides a light-weight front-end to Spark and enables running R programs at scale. Earlier versions of SparkR supported basic Spark functionality like the RDD API and enabled writing distributed R programs. This talk will introduce the more recent work being done to extend SparkR to include support for SparkSQL, the new DataFrame API introduced in Spark 1.3, and the MLLib machine learning library. We'll be discussing the development process, demoing some of the new functionality, and giving you some insight into what the future holds for SparkR. In addition, we're looking forward to feedback and questions from the audience as we continue to develop and improve the new functionality.

Easier Spark Monitoring (Ryan Williams / Hammer Lab)

Using Spark is great when things go well, but can be perplexing when things fail. Ryan will present some tools for viewing graphs of Spark metrics that make monitoring and debugging issues easier.
Bio: Ryan is a software developer at Hammer Lab (http://www.hammerlab.org/) at Mt. Sinai working on tools for genomic analysis built on top of Spark.

BONUS : a Spark Summit coupon

Haven't purchased your Spark Summit East 2015 (http://spark-summit.org/east/2015) tickets yet ? Thanks to DataBricks members of Spark-NYC Meetup can get 20% off registration by using code "LeLay20".

See you soon !

Photo of Spark-NYC group
Spark-NYC
See more events
Spotify
45 W 18th St, 7th floor · New York, NY