Scalding: Powerful and Concise MapReduce Programming

Scala is a functional programming language on the JVM. Hadoop uses a functional programming model to represent large-scale distributed computation. Scala is thus a very natural match for Hadoop. Twitter uses Scalding for data analysis and machine learning, particularly in cases where we need more than sql-like queries on the logs, for instance fitting models and matrix processing. It scales beautifully from simple, grep-like jobs all the way up to jobs with hundreds of map-reduce pairs.

We are pleased to have Dr. Oscar Boykin and Dr. Argyris Zymnis from Twitter to give us some insight on Scalding DSL and provide some example jobs for common use cases. They will also be including a hands-on session where participants can try scalding on their local machine to run a realistic job.

MORE ABOUT THE PRESENTERS...
Presenters Dr. Oscar Boykin and Dr. Argyris Zymnis are the main scalding committers. Both are at Twitter and work on the Revenue Quality Engineering team. Both are often bearded but generally discourage use of the term "neckbeard".

Join or login to comment.

  • Max W.

    Hey guys, the video from this meetup is now online: http://mrkn.co/f0w1c­

    May 10, 2012

  • Vlad P.

    It's a great framework. Demonstrates the power of scala, the convenience of mapreduce, and the amazing professionalism of Twitter Engineers.

    April 20, 2012

  • Andrew H.

    Always love to hear engineers from twitter speak.

    April 20, 2012

  • A former member
    A former member

    Surprisingly easy for me, a data generalist of sorts, to understand. I appreciated the down-to-earth approach Oscar and Argyris took. Someone with no map-reduce knowledge should have been able to take away something from it. There were a sufficient number of examples and a good amount of explanation given by Oscar.

    April 20, 2012

  • Florent B.

    Interesting subject and very good speakers (skilled + entertaining)

    April 19, 2012

Our Sponsors

People in this
Meetup are also in:

Sometimes the best Meetup Group is the one you start

Get started Learn more
Katie

I'm surprised by the level of growth I've seen since becoming an organizer, it's given me more confidence in my abilities.

Katie, started NYC ICO

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy