addressalign-toparrow-leftarrow-rightbackbellblockcalendarcameraccwcheckchevron-downchevron-leftchevron-rightchevron-small-downchevron-small-leftchevron-small-rightchevron-small-upchevron-upcircle-with-checkcircle-with-crosscircle-with-pluscontroller-playcrossdots-three-verticaleditemptyheartexporteye-with-lineeyefacebookfolderfullheartglobegmailgooglegroupshelp-with-circleimageimagesinstagramFill 1light-bulblinklocation-pinm-swarmSearchmailmessagesminusmoremuplabelShape 3 + Rectangle 1ShapeoutlookpersonJoin Group on CardStartprice-ribbonprintShapeShapeShapeShapeImported LayersImported LayersImported Layersshieldstartickettrashtriangle-downtriangle-uptwitteruserwarningyahoo

What's New in Spark 0.6 and Shark 0.2

  • Nov 5, 2012 · 6:30 PM

This meetup will cover the new features and improvements in the recent releases of Spark 0.6 and Shark 0.2. These releases make available a significant amount of work that has gone into the projects over the past few months, including multiple new usability features, such as simplified deployment modes, and substantial performance optimizations (over 2x in some operations). The major new features will be presented by members of the Spark and Shark teams. In addition, we'll talk briefly about what's coming next for each project, and how we are structuring the development process to make the roadmap more visible.

As a preview, some of the new features are:

For Spark 0.6:

  • Simpler deployment, through a new pure-Java standalone cluster mode
  • Java API, exposing all of Spark's features cleanly in Java
  • New, custom communication and storage layer
  • Expanded documentation, including new tuning and config guides

And for Shark 0.2:

  • Hive 0.9 support
  • Hive Thrift server, allowing multiple clients to share one Shark instance and cache
  • Optimized joins and group-bys
  • ADD FILE and ADD JAR

Airbnb graciously offered to host the meetup. Doors will open at 6:30, and talks will start at 7.

Join or login to comment.

  • Matei Z.

    Thanks everyone who came by. I've posted the slides at http://files.meetup.com/3138542/meetup-nov-5-2012.pptx. We may also get a video soon since the meetup was recorded.

    1 · November 7, 2012

  • Kalpit S.

    Matei and the rest of AMPLAB guys did a good job of presenting the enhancements. They were clear and concise.

    November 6, 2012

  • Johann S.

    Great to see progress on the projects! Presentations covered a lot of ground, provided clear examples, and balanced detail with introductory context to address current users as well as those attending to learn about Spark and Shark.

    November 6, 2012

  • Frank O.

    Webcast? slides?

    November 4, 2012

People in this
Meetup are also in:

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy