addressalign-toparrow-leftarrow-rightbackbellblockcalendarcameraccwcheckchevron-downchevron-leftchevron-rightchevron-small-downchevron-small-leftchevron-small-rightchevron-small-upchevron-upcircle-with-checkcircle-with-crosscircle-with-pluscrossdots-three-verticaleditemptyheartexporteye-with-lineeyefacebookfolderfullheartglobegmailgooglegroupshelp-with-circleimageimagesinstagramFill 1linklocation-pinm-swarmSearchmailmessagesminusmoremuplabelShape 3 + Rectangle 1ShapeoutlookpersonJoin Group on CardStartprice-ribbonShapeShapeShapeShapeImported LayersImported LayersImported Layersshieldstartickettrashtriangle-downtriangle-uptwitteruserwarningyahoo

Spark 0.8 release and Spark at Bizo

  • Sep 30, 2013 · 6:30 PM

This event will feature two talks: we will first announce the Spark 0.8 release, followed by a use case talk from Bizo. We'd like to thank Tagged for hosting the event.


Spark 0.8 release

Spark 0.8 is the biggest Spark release yet, as well as our first under Apache. With 67 developers and 20 companies contributing, this release adds a slew of new features. To make debugging and productionizing Spark jobs easier, we have a new monitoring UI and metrics infrastructure. To expand Spark's out-of-the-box capabilities, 0.8 adds MLlib, a standard library of high-quality machine learning algorithms. For Python users, PySpark has been greatly expanded to bring it near feature-parity with Scala, and now supports IPython and Windows. And for deployability, Spark 0.8 includes much-improved support for YARN, new EC2 scripts, and simpler packaging. This talk will give a tour of these and other new features.

Spark at Bizo

Bizo allows marketers target display campaigns to specific business demographic audiences, eg. people in finance or medicine; basically we help marketers get in front of the "right people". Part of the tooling we provide to customers is a reporting platform that has all kinds of fun shiny funnel charts stuffed with pretty metrics (seriously marketers love funnels even more than developers love free pizza). Recently we had to build some new reports that allow users to compare behavior of their website visitors based on whether or not visitors have been exposed to one of our display ads. This was a perfect opportunity for us to test out using Spark in production as it involved processing a fairly large amount of log data from multiple sources on a nightly basis. 

This talk will walk through how we're using Spark in production today on Amazon's EMR service. I'll cover how we've setup our installation & deployment, how we structure our Spark jobs for easy unit testing, plus talk about how we put together a successful Spark hackday to get other engineering sub-teams at Bizo excited about using Spark. Finally I'll cover some common-pitfalls & caveats we've encountered - especially with regards of translating some of our older Hive jobs to Spark & how we go about debugging failed Spark jobs. 

Doors open at 6:30, with talks starting at 7.

Join or login to comment.

  • Chang L.

    Thanks to the Spark team and Tagged for an excellent presentation. Glad to hear how people are using Spark.

    October 3, 2013

  • Matei Z.

    Hey folks, for those interested, the Spark 0.8 release slides are now up here: http://files.meetup.com/3138542/meetup-sept-30-2013.pdf

    2 · September 30, 2013

    • Jay C.

      Big thanks

      October 1, 2013

    • Burt P.

      Excellent! Thank you Matei.

      October 1, 2013

  • A former member
    A former member

    Last minute can't attend. Enjoy the talk!

    September 30, 2013

  • Pablo D.

    I am not able to make it this time. I'll change my RSVP. Sorry

    September 30, 2013

  • Ted E

    Seriously, guys? Are you all a bunch of kids?

    1 · September 30, 2013

    • A former member
      A former member

      It gets better. Tweet 'free parking' if found and GeoTweetSearch will allow other drivers to find. How many times have you left a parking space with time on the meter?

      September 30, 2013

  • Ishan C.

    space shuttle landing strip?

    September 30, 2013

  • Adam J. B.

    Heliport?

    September 29, 2013

    • A former member
      A former member

      There is a Heliport 1.2 miles from the office. Commodore Heliport
      San Francisco, CA[masked]http://goo.gl/maps/zh...­

      1 · September 29, 2013

  • Shashidhar D.

    Will there be any space for bicycles?

    September 29, 2013

    • A former member
      A former member

      Yes, we have plenty of bike racks on the wall inside.

      September 29, 2013

  • Jay C.

    Can I watch the talk on-line

    2 · September 25, 2013

  • Timothy St. C.

    webcast?

    September 25, 2013

People in this
Meetup are also in:

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy