Spark 1.4 Overview & Spark Committers Q&A - Spark Summit Meetup

This is a past event

549 people went


Come to the official Spark meetup for Spark Summit 2015 to hear from Spark release manager Patrick Wendell about release 1.4 of Spark. In addition we will host a Q&A session with Spark Committers from Databricks, Cloudera, and AMPLab including Michael Armbrust, Joseph Bradley, Tathagata Das, Yin Huai, Xiangrui Meng, Josh Rosen, Sandy Ryza, and Shivaram Venkataraman. This meetup is on the evening of the first day of the Spark Summit. You do not need to attend Spark Summit to register for this meetup.

The talk and Q&A session will be filmed and the video will be published on the Apache Spark channel ( on YouTube.

For Spark Summit, we have a great set of talks from NASA, CIA, Netflix, Baidu, Airbnb, Microsoft, and more. Use the discount code “SFmeetup” to get 15% off registration.

Beer, soda and water will be provided.

RSVP is required for this event. Names will be checked at the door.

Attendees are welcome to attend the Sparks Summit Networking Event after the meet-up concludes.

Agenda for the evening:
6:30 Mingling
7:00 Patrick Wendell, Talk: Deep Dive into Spark 1.4
7:15 Spark Committer Q&A
8:00 Mingling
9:00 End

Speaker: Patrick Wendell

Title: Deep Dive into Spark 1.4


This talk will announce the Spark 1.4 release and review the new features shipping in this release. It will also give some broader context on the major initiatives in the Spark roadmap. Spark 1.4 significantly extends Spark's new DataFrame API's along with the first pieces of Project Tungsten, an initiative targeting performance optimizations throughout Spark's internal engine. Spark adds new support for visualization of execution plans, telemetry over streaming data, and SQL/dataframe query plans. In Spark's MLlib, the newer pipeline API graduates from alpha in this release. 1.4 also adds a large number of built-in transformers to MLlib along with math functions in Dataframes. Spark also adds an R API based on the SparkR project.

Speaker Bio:

Patrick Wendell is a co-founder of Databricks as well as a Spark Committer and PMC member. In the Spark project, Patrick has acted as release manager for several Spark releases. Patrick also maintains subsystems of Spark's core engine. Before helping start Databricks, Patrick obtained an M.S. in Computer Science at UC Berkeley. His research focused on low latency scheduling for large scale analytics workloads. He holds a B.S.E in Computer Science from Princeton University