SHUG 1. Up-to-date introduction to Apache Hadoop and Big Data at Spotify

We are pleased to invite you to the first meeting of Stockholm Hadoop User Group. The opening meeting will be held on Thursday, January 24 at 19:00 at Spotify Office (Birger Jarlsgatan 61, 11tr, Stockholm).


Topic: Up-to-date introduction to Apache Hadoop and Big Data at Spotify

Speakers: Wouter de Bie (Spotify), Adam Kawa (Spotify)

Abstract:

In this talk, we will present Apache Hadoop - one of the most popular tools for big data processing, that has been successfully deployed in production by many companies for several years. While Hadoop is considered to be a reliable, scalable and cost-effective solution, it is still constantly being improved by a large community of developers. As a result, the recently released version 2.0 offers several revolutionary features including YARN (often called MapReduce 2.0), HDFS Federation and a highly-available NameNode, which makes the Hadoop cluster much more powerful and reliable. We will describe the advantages of these components and explain how they improve the “more traditional” version of Apache Hadoop.

In the second half of the presentation, we will give overview of the Big Data infrastructure at Spotify. We will discuss real-world problems that we have been solved using open-source technologies, as well as challenges that we would like to attack in the future.

Wouter de Bie is the Team Lead Analytics and Data Infrastructure at Spotify. He heads up a team of data engineers and is responsible for the data infrastructure and architecture.

Adam Kawa is a software developer at Spotify whose daily job includes the implementation of MapReduce jobs and administration of the Hadoop cluster. Prior to Spotify, he worked with Hadoop at Warsaw University and IBM. From time to time, he blogs at HakunaMapData.com.

Additional information:

Last, but not least, free pizza and beverages will be available for the participants during the meetup. If you are willing to come, please click the “attend button” on this page. This will help us to estimate the number of pizzas and drinks based on declared attendance.

The event is scheduled for a little more than 2 hours in total. The presentation will take about 1h 15min + about 15 minutes for Q&A. After that there will be about 45 minutes of socializing.

Note: please RSVP to this meetup, since we need to put everybody on a guest list for entering the Spotify office. The event will be held in the cafeteria of the Spotify office, so don’t go to the normal entrance but to the 11th floor.

The door will be open at 18:30. Because of fire regulations, we need to keep a list of everybody in the building, so please make sure that you get your name ticked off the list at the entrance or (in case of a +1), make sure that the person at the door puts your name on the list.

See you at the meeting!

SHUG Team

Join or login to comment.

  • Adam K.

    Hi,
    We have already created a small questionnaire that hopefully will give better understanding of where to go ;)
    Please fill it in - it will take a minute or two: http://bit.ly/VsktsU
    Thank you!

    February 1, 2013

  • Wouter de B.

    Since it was the first meetup we did, it was hard to figure out what the audience would be. Adam and I have talked about setting up a small questionnaire that we'll send out to get a better understanding of what people would like to hear and find appropriate speakers for those topics.

    2 · January 25, 2013

    • Wouter de B.

      Haha, no stepping on toes :) I really appreciate the feedback! Price of course depends on your use case, but the fact that Hadoop is open source doesn't require you to buy a license. Hardware and possibly support by a 3rd party vendor will still cost money. In our case (with our data volumes), going the open source route is probably cheaper overall.

      January 25, 2013

    • Matzulu E.

      I realized that the evaluation I wrote came out a bit harsh. I want to add that I'm really glad you put this together. Thanks guys!

      January 25, 2013

  • Antonio Da S.

    Nice presentation! A bit too heavy on the tech side (although I liked it, maybe it's not for everyone?) and I would've liked it if there were more user cases.

    January 25, 2013

  • Wouter de B.

    Hi all!

    I've just uploaded the slides from yesterday at http://www.meetup.com/stockholm-hug/files/

    Take care all!

    // Wouter

    1 · January 25, 2013

  • Guillermo C.

    Thank you very much for the meetup yesterday, it was very good! Adam and Wouter, do you think you could share your slides? I think it would be good to have them, if possible :-).

    Thanks again!

    January 25, 2013

  • Daniel L.

    Thanks for a great meetup. I think it was perfect as a first step and looking forward to the next one.

    January 25, 2013

  • Jonas F.

    Nice meetup yesterday.

    It is quite obviously a complex topic, would this meetup benefit from being split into tracks? For example one track for developers and another for ops or whatever makes sense for the topic.

    January 25, 2013

  • A former member
    A former member

    It was very good and now I'm looking forward to more advanced topics! :)

    January 25, 2013

  • Matzulu E.

    Personally I would have liked more of the use-case, and perhaps more baked in to the theoretical part. However, that is my opinion and I guess it's different for the people in the audience that haven't worked a lot with Hadoop

    January 25, 2013

  • Lars A.

    Thank you Spotify for hosting a nice meetup, and for two very interesting presentations!

    January 25, 2013

  • Antonio Da S.

    Good info and presentation!

    January 24, 2013

  • Theodore V.

    Great meetup, and the Spotify people are great, looking forward to the next one!

    January 24, 2013

  • Rinman

    Didn't have any pre-knowledge about Hadoop, besides very general articles and blog posts, and this first meetup served as a perfect introduction, fast paced and packed with information.

    January 24, 2013

    • Daniel E.

      I can't more then agree here! Really good!

      January 24, 2013

  • Wouter de B.

    Hi all!

    Thanks for attending todays meetup! I hope it was interesting for all of you and I'm looking forward to the next installment. Again, if anybody would like to help out with organizing the next edition or wants to present, please let us know.

    Thanks again!

    Wouter

    January 24, 2013

  • George E. K.

    Thorough and understandable presentation! The second part with the business side of Spotify was also very good! Congratulations!

    January 24, 2013

  • Wouter de B.

    Hi all!

    Adam and myself are preparing for our first ever Stockholm Hadoop User Group meetup! We'll start at 19:00 and the door will be open from 18:30. If all deliveries go well, we should have pizza, beer and soda then, so grab a slice, grab a drink and grab a seat when you arrive!

    1 · January 22, 2013

  • Adam K.

    Hi!
    We will provide more information about the date this week!
    Please, stay tuned ;)

    1 · January 7, 2013

  • Carl B.

    Still no date for this? :)

    January 7, 2013

  • Magnus E.

    Sounds really good!

    November 21, 2012

  • Wouter de B.

    Since (mid-)December is a busy period in Sweden, I would suggest moving this to the beginning of January. Spotify could host this meeting at our office on Birger Jarlsgatan.

    November 2, 2012

    • Guillermo C.

      Hej! By my side, I also think it's better to postpone it to the beginning of January, as in December most of the people is very busy or unavailable.

      November 2, 2012

Our Sponsors

People in this
Meetup are also in:

Create a Meetup Group and meet new people

Get started Learn more
Bill

I started the group because there wasn't any other type of group like this. I've met some great folks in the group who have become close friends and have also met some amazing business owners.

Bill, started New York City Gay Craft Beer Lovers

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy