addressalign-toparrow-leftarrow-rightbackbellblockcalendarcameraccwcheckchevron-downchevron-leftchevron-rightchevron-small-downchevron-small-leftchevron-small-rightchevron-small-upchevron-upcircle-with-checkcircle-with-crosscircle-with-pluscrossdots-three-verticaleditemptyheartexporteye-with-lineeyefacebookfolderfullheartglobegmailgooglegroupsimageimagesinstagramlinklocation-pinm-swarmSearchmailmessagesminusmoremuplabelShape 3 + Rectangle 1outlookpersonJoin Group on CardStartprice-ribbonImported LayersImported LayersImported Layersshieldstartickettrashtriangle-downtriangle-uptwitteruseryahoo

SHUG 12. Improving developer productivity (+) Assembling Lambda Architectures

We are happy to invite you to the next meeting of Stockholm Hadoop User Group! Please find the details below.

Title: Improving developer productivity in the Big Data world: how to iterate faster on your Hadoop jobs.

Speaker: David Whiting, Data Engineer at Spotify


The demand for processed data is increasing exponentially at Spotify and we've found our developer infrastructure to be at least as much of a barrier to that scaling as having the hardware available. This talk will tell the story of what problems we've faced in transitioning from the Hadoop team being a small annexe of analytics to having dozens of developers throughout the organisation writing code to run on the cluster; and how we're working to solve these by building better developer infrastructure to enable developers to iterate faster and get the data they need when they need it.


David Whiting has been working with Data Infrastructure at Spotify for the past 16 months; currently part of the DataEx team dedicated to improving the development experience for engineers working with data. He mostly works with Hadoop, but can occasionally be found dabbling in data warehousing, SQL query optimisation and front-end web apps; as well as telling everybody else they're not doing enough testing and that everything is better with static typing. As well as generating music data, he also generates music under the guise of Demoscene Time Machine ( ), takes part in the occasional triathlon and has some very unusual dance moves.



Assembling Lambda Architectures using YARN

Speaker: Aaron Wiebe, Principal Architect at Hortonworks


One of the most common uses for Hadoop is within a Lambda Architecture.  As time has progressed, the vision of a Lambda architecture using Hadoop and YARN has evolved.  In this session we'll review Lambda Architecture, its common variants using streaming technologies such as Storm, and how Lambda architectures will be implemented using YARN in the future.


Aaron is a Principal Architect at Hortonworks, and is primarily focused on working with our key customers in building, deploying, and managing the Hortonworks Data Platform at scale.  Bringing four years of Hadoop experience, Aaron focuses both technical and organizational impacts to the Hadoop solution, and works with the Hortonworks product team to drive features and functionality that our customers are seeking.  Prior to joining Hortonworks, Aaron was responsible for the creation and implementation of Big Data at BlackBerry.  In this role, over 45PB of Hadoop capacity was deployed, supporting all BlackBerry services in all geographies - an extended ecosystem of roughly 4000 servers.


Additional information

RSVP to the meetup 

Please RSVP to this meetup, since we need to put everybody on a guest list for entering the Spotify office. The event will be held in the cafeteria of the Spotify office, so don’t go to the normal entrance but to the 11th floor.

Pizza and drinks

Thanks to Spotify, pizza and beverages will be available for the participants during the meetup. This is another reason to RSVP to this meetup, if you are willing to come - it will help us to estimate the number of pizzas and drinks based on declared attendance.

The entrance

The door will be open between 17:45 and 18:15* Because of fire regulations, we need to keep a list of everybody in the building, so please make sure that you get your name ticked off the list at the entrance or (in case of a +1), make sure that the person at the door puts your name on the list.

*Unfortunately we can not leave the door open all the time (the company security policy), nor have a person that will be constantly watching for guests coming late. If you need to come later, please let us know in the comments below, so that somebody will come to the door to open it a given time.

See you at soon!


Join or login to comment.

  • kiarash r.

    is the talk gong to be recorded?

    June 25, 2014

    • Adam K.

      Unfortunately, we will not record...

      June 25, 2014

Our Sponsors

People in this
Meetup are also in:

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy