addressalign-toparrow-leftarrow-rightbackbellblockcalendarcameraccwcheckchevron-downchevron-leftchevron-rightchevron-small-downchevron-small-leftchevron-small-rightchevron-small-upchevron-upcircle-with-checkcircle-with-crosscircle-with-pluscrossdots-three-verticaleditemptyheartexporteye-with-lineeyefacebookfolderfullheartglobegmailgooglegroupshelp-with-circleimageimagesinstagramlinklocation-pinm-swarmSearchmailmessagesminusmoremuplabelShape 3 + Rectangle 1ShapeoutlookpersonJoin Group on CardStartprice-ribbonShapeShapeShapeShapeImported LayersImported LayersImported Layersshieldstartickettrashtriangle-downtriangle-uptwitteruserwarningyahoo

Storm at Spotify

This time we will visit Spotify HQ in New York City to learn about how they use Storm.

In this talk, Spotify engineer Neville Li will share their experience building real-time features with Storm and Kafka, including recommendation, social, data visualization and ads targeting. We will cover topics such as architecture, production integration, and best practices.


The commercial music streaming service Spotify was launched in 2008 and since then is has registered over 24 million active users of which 6 million are paying users. They have 3.7 million Facebook fans. It has over 20 million songs online and every day[masked] new songs are added to the database. Users created over 1 billion playlists and over $ 500 million has been paid out to rights holders since the launch of Spotify. It may be clear that without big data techniques and tools used, Spotify would not be able to exist.

Spotify is a data-driven company, meaning that data is used in almost any part of the organization. The numbers confirm this: Spotify users create 600 Gigabyte of data per day and 150 Gigabyte of data per day via different services. Every day 4 Terabyte of data is generated in Hadoop, a 700-node cluster running over 2.000 jobs per day. They currently have 28 Petabytes of storage, spread out over 4 data centres across the world. This is the first time they will be talking about their deployment and use cases for Storm.

Neville Li (@sinisa_lyh) is a Software Engineer at Spotify, where he has been crunching data since 2011 and has introduced Storm, Scalding, and Spark to Spotify growing data ecosystem.



 

 

 

 

 

As always, we will have book raffle sponsored by O'Reilly.

Food and drinks will be provided by Spotify.

Agenda:

6:30 - Arrive to Spotify, meet other members  
6:45 - Books giveaway 
7:00 - Storm at Spotify
8:00 - Q&A  
8:15 - Open Discussion, Networking

Location:

Spotify  
45 West 18th St  
7th floor  
New York, NY

Our Sponsors:

DataTorrent

DataTorrent is the most powerful real-time computation platform.

NoSQL Weekly

A free weekly newsletter featuring curated news, articles, new releases, jobs etc related to NoSQL.

 

 


Join or login to comment.

  • Digant M.

    Great talk.

    March 26, 2014

  • Neville L.

    Thanks all for coming today. Slides here: http://www.slideshare.net/sinisalyh/storm-at-spotify

    March 25, 2014

    • A former member
      A former member

      hi Neville .. for some reason this link doesn't seem to work for me

      March 26, 2014

  • Eric P.

    Excellent presentation. Thanks!

    March 26, 2014

  • A former member
    A former member

    great talk!

    March 26, 2014

  • Robert D.

    great talk. learned a few new things

    March 25, 2014

  • Arman

    Thanks for the presentation @Neville, it was fun to learn the ways you use Storm and the other technologies in your environment.

    March 25, 2014

  • A former member
    A former member

    Unfortunately I can't make it, opening up a spot. Please post video / slides if possible. Thanks!

    March 25, 2014

  • Sam W.

    I'm looking forward to attend this meetup, please another spot opening!

    March 25, 2014

    • Sam W.

      It shows 25 spot left, how do I rsvp?

      March 25, 2014

  • Jonathan D.

    I couldn't get through on the waitlist and would really like to attend this meetup.

    March 25, 2014

  • Nitin k.

    It would be great if this presentation is recorded. There are people who have last minute cancellation but really want to learn and benefit a lot from this....
    Also, would be great if slides can be put up... Pl. let us know….

    March 25, 2014

    • Eugene D.

      Yes. Presentation will be recorded by Hakka Lab and posted online later on.

      March 25, 2014

    • Tim B.

      This meeting is indeed being recorded. There was a meeting announcement from Eugene Dvorkin:

      "I hope to see everybody tomorrow at Spotify. For those of you who can't make it or are on the waiting list I have good news: Hakka Lab will record the presentation and post it online later, the same way they did for our Zookeeper talk. This will give all of us the opportunity to learn how Spotify uses Storm Framework.

      Please update your RSVP if you can't attend in person - we have a long waiting list.

      And last, please bring your business cards, I have several books from O'Reilly to raffle."

      March 25, 2014

  • Colby R.

    Whats the capacity @ Spotify?

    March 24, 2014

  • A former member
    A former member

    Hope I get in

    March 17, 2014

  • Roman K.

    Glad to join

    March 12, 2014

  • Rotem D.

    Hello

    March 4, 2014

Our Sponsors

People in this
Meetup are also in:

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy