Netflix's move to AWS

Over the last year, Netflix has migrated its website and streaming service from a conventional datacenter implementation to the Amazon public cloud. Along the way, they re-wrote most of their code base, built a completely new data source backend based primarily on SimpleDB, and re-tooled their processes with high levels of automation. As a result, despite high and accelerating growth rates in Netflix subscriber counts, the growth rate of Netflix’ datacenter footprint has been halted, and all capacity expansion is now leveraging AWS. Since data is modified in the datacenter and in the cloud, bidirectional replication has been implemented, and Netflix has had to learn “roman riding” (look it up) with one foot in each environment.

In this talk, Netflix’ Cloud Architect Adrian Cockcroft will discuss the datacenter anti-patterns that motivated a new code architecture, data architecture and deployment model. The Netflix cloud architecture takes advantage of almost every feature of AWS, and is optimized for running in a highly automated environment with ephemeral instances, non-deterministic performance, and agile deployment processes.

***

About the Speaker:
Adrian Cockcroft, Cloud Architect at Netflix

Over the last three years Adrian has managed and architected development of personalization algorithms for the Netflix web site and TV based devices, refactoring code into middle tier services, and development of a new personalization platform optimized for the cloud. He recently moved to the Cloud Systems group, with a focus on performance monitoring and analysis tools. Adrian is probably best known as the author of several books while a Distinguished Engineer at Sun Microsystems: Sun Performance and Tuning; Resource Management; and Capacity Planning for Web Services. He was also a founding member of eBay Research Labs, where he contributed to the Skype Java API and built mobile application prototypes. ***

A big thanks to the following sponsors for making this event happen:

Join or login to comment.

  • A former member
    A former member

    Very interesting presentation. Could have been slightly shorter, but I understood that this was a Beta. thanks for the cool evening.

    October 18, 2010

  • A former member
    A former member

    This was a quality meetup event. Thank you for hosting it! There was initial delay because of the equipment setup issues but once things got rolling, it was great!

    October 17, 2010

  • A former member
    A former member

    Thanks the video was very informative

    October 16, 2010

  • A former member
    A former member

    Everyone I met was interesting!

    October 16, 2010

  • Chris W.

    Good idea to stream it, given the number of RSVPs. I watched from home and it worked well. Also good idea to take questions via Twitter. The only problem was not being able to see the slides.

    October 16, 2010

  • Sebastian

    Big thanks to Patrick for editing and uploading the video!

    October 15, 2010

  • Patrick M.

    The video is now available. Go to http://blip.tv/file/4252897

    October 15, 2010

  • Mike C.

    Very interesting talk, thanks.

    October 15, 2010

  • A former member
    A former member

    In the presentation I was looking answers for Why? How ? and Future direction of Netflix. Mr.Adrian not only answered them but also provided pointers that will be helpful to both cloud computing novice and experts. Time management and content was excellent. I thank him for providing an insight on the factors to consider before jumping from data center to cloud computing. Presentation was invaluable to me and also thanks for the time taken to share.
    Due to technical difficulties in streaming, no fault of anyone, he was delayed by 30 minutes. Sebastian, good job for organizing and coming up with the idea of streaming. Waiting for the video and presentation.

    October 15, 2010

  • Abhi

    Good talk and in general good move.

    October 15, 2010

  • A former member
    A former member

    Watch the Ustream and read the Tweets. Thanks for broadcast.

    October 15, 2010

  • Luc S.

    followed via stream...

    October 15, 2010

  • A former member
    A former member

    Great talk, extremely informative from a technical point of view, and well delivered.

    Nevertheless, I think Netflix experience does not easily translate to much of the
    audience, which is startup companies. The main reason is that Netflix is a higly
    profitable company whose IT budget is a very small fraction of the overall budget.
    This is exactly opposite to most IT startups. Therefore Netflix reasons for moving
    to Cloud do not really apply to many IT startups.

    October 15, 2010

  • Joe P.

    It's more than a little ironic that in a meeting about how to scale video streaming to millions of users, the video streaming was such poor quality. Since there was some doubt as to whether the space would hold all the people, watching remotely was encouraged. Number of people allowed to indicate they were intending to attend should have been capped at the size of the venue or streaming et. al. should have been tested in advance.

    October 15, 2010

  • David A.

    Speaker was very very good. He knew his stuff and didn't get bogged down in small details.

    October 15, 2010

  • Jim

    Good talk.

    October 15, 2010

  • Ray N.

    AWEsome!!!

    October 15, 2010

  • Marty P.

    The talk was very interesting and I felt like I got a lot out of it.

    I was watching the live feed, which was rough to start but got better once the dojo's network calmed down a bit. The organizers did a good job responding to feedback.

    I especially appreciated the photos of the slides that some people tweeted. These were readable while those in the video were not. If folks in the audience did this more regularly, it would be a boon to those viewing remotely.

    Great meetup. Thanks a lot.

    October 15, 2010

  • Miha A.

    It is unbelievable how a technical presentation, about a conceptual breaktrough - run a 15 million subscriber service with de-facto no physical data center - can gather an audience of nearly 600, as if this was rock star event. People follow breathlessly how Netflix did it

    October 15, 2010

  • Steve S.

    Streaming didn't work for me -- lots of drop-outs.

    October 15, 2010

  • Miha A.

    Did any Hackerdojo meetup event have more than 562 attendees? Or this is an absolute record?

    October 15, 2010

  • Naren C.

    Thank you Adrian and Organizers. Loved the content. On my wish list - stream all hacker dojo events + twitter for real-time chatter. Improve the streaming experience thought (maybe, Netflix can help :-))

    October 15, 2010

  • A former member
    A former member

    thank you cloudflower for hosting. it would have been awesome if the video and audio were better from hackerdojo, maybe next time.

    October 15, 2010

  • Gaurang K.

    Hello guys -- i could attend only the first half of the talk. Please please upload the HD recording when you can. Appreciate all your help in arranging a nice seminar -- and ofcourse to Adrian for giving so much useful info.

    October 15, 2010

  • Adrian C.

    I will watch twitter for questions tagged #netflixcloud

    October 14, 2010

  • Shailen S.

    Thanks for the stream link Sebastian. Will the online audience be able to ask interactive questions?

    October 14, 2010

  • Sebastian

    We'll have HD footage. Any volunteers to do the editing?

    October 14, 2010

  • Sebastian

    We'll have HD footage. Any volunteers to do the editing?

    October 14, 2010

  • Bernard G.

    Will the video also be captured for later viewing?

    October 14, 2010

  • A former member
    A former member

    If the live stream works out, we'll be showing it from the CrowdFlower office in San Francisco at 455 Valencia - feel free to stop by and watch if you're in SF and want to avoid the drive down south.

    October 14, 2010

  • Sebastian

    *Update* The session will be streamed live at http://www.ustream.tv/channel/silicon-valley-cloud-computing-group - pizza not included ;-)

    October 14, 2010

  • Shailen S.

    Does anyone have an update on whether this will be streamed live?

    October 14, 2010

  • Andrey A.

    What are the URLs for the live stream and for the recording?

    October 13, 2010

  • Sebastian

    We'll be recording and streaming the event, but even with that, we're definitely going to be packed!

    October 13, 2010

  • Adrian C.

    This talk is a "beta version" of a talk to be given at QConSF on Nov 3rd. This version will be more devops focused on the AWS components, the QConSF talk will go into more detail on the software architecture.

    October 13, 2010

  • Moazam R.

    Seriously, as Stephen mentioned, does the Dojo has space for 450+ (now 512!) people?

    Or is this going to be broadcast/streamed/recorded ?

    October 13, 2010

  • Stephen O.

    Does the Dojo have enough room for 450 people?

    October 11, 2010

  • Adrian C.

    Teaser presentation summary for this Cloud Meetup: Netflix in the Cloud http://slidesha.re/9KD8c5

    October 7, 2010

  • Ankush K.

    It would be great if some body would like to carpool.
    I am a student and live in San Jose downtown.

    September 30, 2010

  • sumeet r.

    I agree with Shuo, it would be great if Adrian could share the slides before the session - we (YouSendIt) have been thinking about migrating to AWS, and would love to come prepared

    September 30, 2010

  • Shuo Y.

    If possible, I'd suggest the speaker to share the slides before his talk, so that we can think deeply and ask questions. In this way, the talk will be much more interactive and helpful :-)

    September 30, 2010

  • Sanjib P.

    Would appreciate if anyone would like to carpool, I donot drive and live in San Ramon East Bay.

    September 30, 2010

Our Sponsors

  • CloudStack

    CloudStack (cloudstack.org) is an open source cloud computing platform.

  • Scalr

    Cloud management software

  • Ubuntu

    Ubuntu is the open source cloud OS, supported by Canonical.

  • Pivotal

    Redefining PaaS for enterprise, leveraging big fast data.

  • Rackspace

    The open cloud company and OpenStack co-founder

  • HP

    To learn more about HP’s cloud solutions, visit hp.com/go/cloudtalk

  • AWS

    Pay-as-you-use cloud computing services for everyone

  • General Catalyst

    Funds breakthrough ventures and backs exceptional entrepreneurs.

  • Aruba Networks

    Aruba provides next-gen network solutions for the mobile enterprise.

  • Riverbed Stingray

    Riverbed Stingray: Fast, reliable, secure applications in the cloud

People in this
Meetup are also in:

Create a Meetup Group and meet new people

Get started Learn more
Bill

I started the group because there wasn't any other type of group like this. I've met some great folks in the group who have become close friends and have also met some amazing business owners.

Bill, started New York City Gay Craft Beer Lovers

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy