Hadoop @foursquare


At foursquare, we're generating, collecting, and analyzing billions of log events each week. Combined with data exported from mongodb, such as foursquare's 2 billion check-ins, we use Hadoop to build recommendations, power internal reporting, and drive product features.

We've recently introduced a number of new components to our Hadoop stack in order to create a reliable, scalable, and production ready pipeline. These include Oozie for workflow management, Thrift for data serialization, Pig for adhoc analysis, and Scoobi for writing MapReduce jobs in Scala. We'll share some experiences, lessons learned, and how we're making it easier for developers and non-developers to use Hadoop at foursquare.

Join or login to comment.

  • Stephen S.

    Excellent speakers!

    June 21, 2012

  • saurabh

    Enjoyed the talk!!! very informative and candid!!! I captured few highlights of the talk and put in the blog http://sauragar.wordpress.com/2012/06/20/hadoop-foursquare/. It did not fully justified the talk, but give some info to people who has not attended the talk.

    June 20, 2012

  • Andrew R K.

    Slides should be available sometime soon. Unfortunately, there were some technical difficulties which prevented a recording from being made :( Sorry!

    June 20, 2012

  • Matthew Z.

    Great talk and good company, i'd love to share the presentation with some coworkers if possible.

    June 20, 2012

  • A former member
    A former member

    Excellent meetup! Thanks to Blake and Joe for the talks!

    June 20, 2012

  • Hiroyuki W.

    Excellent presentation on Hadoop and Hadoop related technologies @foursquare

    June 20, 2012

  • Richard G.

    Very frank and revealing talk on past and present challenges/solutions managing big data at Foursquare

    June 20, 2012

  • Tony B.

    Good presentation by both speakers; gained good insight on how Foursquare is attempting to solve data growth issues, in some cases however reinventing the wheel/.

    June 20, 2012

  • A former member
    A former member

    Good tech overview of Foursquare's hadoop effort, this type of meetup is better than sales presentations, but both have their place. Thanks to the guys at Foursquare who presented and hosted.

    June 20, 2012

  • Artem E.

    This was a great session, keep them coming and provide slides

    June 20, 2012

  • Patrick D.

    The meetup was informative.

    June 20, 2012

  • Adam S.

    A++ meetup, would attend again. Excellent talk, touching on both high level and deep technical details of Hadoop data pipeline at a hot startup.

    June 20, 2012

  • A former member
    A former member

    Loved how open the speakers were about their processes and thoughts.

    June 20, 2012

  • Fred G.

    Intense tech!

    June 20, 2012

  • rutvik

    yes please record this...i would love to be at this one but might not be able to make it.

    June 18, 2012

  • Pete S.

    We're making plans to record the meetup for those that aren't able to attend. I'll keep the group posted.

    June 14, 2012

  • John Mark M.

    I would love to attend but am on business travel. Will this be recorded?

    June 14, 2012

Our Sponsors

People in this
Meetup are also in:

Create a Meetup Group and meet new people

Get started Learn more
Henry

I decided to start Reno Motorcycle Riders Group because I wanted to be part of a group of people who enjoyed my passion... I was excited and nervous. Our group has grown by leaps and bounds. I never thought it would be this big.

Henry, started Reno Motorcycle Riders

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy