addressalign-toparrow-leftarrow-rightbackbellblockcalendarcameraccwcheckchevron-downchevron-leftchevron-rightchevron-small-downchevron-small-leftchevron-small-rightchevron-small-upchevron-upcircle-with-checkcircle-with-crosscircle-with-pluscontroller-playcrossdots-three-verticaleditemptyheartexporteye-with-lineeyefacebookfolderfullheartglobegmailgooglegroupshelp-with-circleimageimagesinstagramFill 1light-bulblinklocation-pinm-swarmSearchmailmessagesminusmoremuplabelShape 3 + Rectangle 1ShapeoutlookpersonJoin Group on CardStartprice-ribbonprintShapeShapeShapeShapeImported LayersImported LayersImported Layersshieldstartickettrashtriangle-downtriangle-uptwitteruserwarningyahoo

Mad-Railers/Big Data Madison - MapReduce Edition!

Matthew Rathbone will cover the general format of a MapReduce job, as well as how to build a job in ruby, and run it in Amazon EMR. In fact, he might even run one live.

MapReduce is how Google manages to search millions or billions of sources with snappy response times.

Proposed agenda:
- General structure of a distributed map reduce framework
    - What is a mapper?
    - What is a partitioner?
    - What is a reducer?
    - How does data flow from one to the other?
- How do I write this in ruby (or python, or bash, or even lisp)?
- LIVE DEMO 1
- Quick chat about more sophisticated use cases
- Man that's slow, what higher level frameworks are there? [hive, pig, scoobi]
- LIVE DEMO 2

This will be a joint meetup with Mad-Railers. Enjoy!

Join or login to comment.

  • A former member
    A former member

    pretty cool

    May 22, 2012

  • A former member
    A former member

    Exactly what I was hoping for - a broad overview of the applications of Map Reduce with a taste of the implementation Also free pizza.

    May 22, 2012

  • A former member
    A former member

    I'll preface this by saying that I'm not a programmer, I mainly came to possibly network to find a front\backend guy. I have to say having Matt from 4s speak was pretty amazing. He had a fantastic and deep knowledge of what he talked about and seemed to be very passionate about his work. Even as someone who had no idea what he was talking about, it was obvious he was an expert in his field.

    There was plenty (too much lol) of pizza & drinks. Was a good quick meeting. Looking forward to finding the right person for our project and attending more informative meets.

    May 22, 2012

  • Thomas C. M.

    It was good seeing Hadoop and Rails together. The presentation was packed with information that'll take a while to digest, and the side conversations I had with people were cool. It's always interesting to hear what other people are working on and how they're applying technology to support business processes.

    May 22, 2012

  • Louie P.

    Matthew had a great presentation. Short but detailed. Even had the "gotchas"

    May 22, 2012

  • Matthew R.

    Looking forward to the talk tomorrow, hope to see everyone there!

    May 20, 2012

30 went

Our Sponsors

  • Cloudera

    Cloudera is the general sponsor of Big Data Madison, and buys food.

  • Hortonworks

    Hortonworks is general sponsor for a round of after-meetup drinks.

  • O'Reilly Media

    Discounts on various conferences and products.

  • Zendesk

    Zendesk covers the meetup.com and other ancillary fees.

People in this
Meetup are also in:

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy