First Spark User Meetup

  • January 31, 2012 · 6:30 PM

On January 31st, we’re hosting the first Spark User Meetup at Klout in San Francisco. Spark is a cluster programming framework that provides in-memory computing for iterative and interactive analytics and a high-level programming interface in the Scala language. This meetup will be a chance to learn Spark from the developers, hear about other peoples’ experiences with Spark, network, and hear about future development plans.

At the first meeting, we’re planning to have a Spark tutorial by Matei Zaharia followed by a presentation from Karthik Thiyagarajan of Quantifind on their experience using Spark to replace Pig for predictive analytics (details below).

If you want to follow along, we strongly recommend you sign up for Amazon EC2 and make sure you can launch instances. This way you’ll be able to bring up your own Mesos/Spark cluster on EC2 and play with a Wikipedia dataset in real time!

 

Time: Tuesday January 31. Pizza and beer at 6:30, talks at 7 PM.

Location: Klout, 77 Stillman St, San Francisco, CA 94107

 

About Quantifind:

Quantifind is a small start up into predictive analytics. We are building a platform that automatically identifies relevant signals in both structured and unstructured content, contextualizes them based on what matters most to the customer, and derives insights about past and future events in a way that is directly relevant and actionable.

The talk highlights our experience using Spark in various ways ranging from a distributed batch processing framework powering our analytics pipeline to an interactive computing infrastructure serving some of our internal exploratory tools.

Over the course of moving our analytics pipeline from Pig to Spark, we realized that Spark's inherent characteristics of low latency and interactivity can be leveraged as an agile way to create new computing services. So we experimented with creating web services on top of Spark which answered queries in real time by performing operations on a cached RDD. Today, Spark acts as a sharded in memory infrastructure for many of the services we use internally helping us explore our data and prototype algorithms in an agile manner.

 

Future Meetups: The meetup will rotate among locations in San Francisco, Silicon Valley and Berkeley.

Join or login to comment.

  • A former member
    A former member

    Great start. Large turnout, and informative speaker.

    February 1, 2012

  • A former member
    A former member

    The examples were really great to see the value in Spark.

    February 1, 2012

  • A former member
    A former member

    I am glad that I got enough exposure at the Meetup to generate ideas on how I can use Spark for my current projects that I am working on.

    February 1, 2012

  • Andy K.

    Lots of interesting folks!

    February 1, 2012

  • Matei Z.

    February 1, 2012

  • Matei Z.

    Thanks everyone for coming! I've posted my slides at http://files.meetup.com/3138542/user-meetup-1.pptx, and will post Karthik's soon too.

    Don't forget to fill in our poll too: tinyurl.com/firstsparkmeetup

    January 31, 2012

  • A former member
    A former member

    kloutlaw!

    January 31, 2012

  • A former member
    A former member

    Can we use the quest network? If so what's the password?

    January 31, 2012

People in this
Meetup are also in:

Sometimes the best Meetup Group is the one you start

Get started Learn more
Katie

I'm surprised by the level of growth I've seen since becoming an organizer, it's given me more confidence in my abilities.

Katie, started NYC ICO

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy