Dave Lester and Florian Leibert on Mesos!

Dave Lester of Twitter and Florian Leibert of Airbnb are coming from SF to talk about Apache Mesos.

Mesos provides cluster scheduling for multiple kinds of resources: CPU, memory, I/O, rack locality, etc. It leverages isolation based on Linux containers (cgroups), and is scalable to 10,000s of nodes. For example, it allows the same cluster to run Hadoop, Spark, MPI, in addition to real-time workloads in Storm, and request/response in web apps. Overall, it obviates the need for using VMs. More long-term, Mesos provides OS-level building blocks for developing new distributed frameworks.

Imagine -- you've got petabytes of data you're running over in one cluster. You've got another cluster running Memcache, Heroku, orsome web apps or whatever. You're serving stuff, so when you crunch everything, you've got to go over the wire to get your data to the other cluster. Would make more sense if you ran a slightly bigger cluster that ran both, so you don't have to pay the tax sending everything back and forth over the wire? Come learn about Mesos.

Speaker Bios

Dave Lester

Dave is currently an Open Source Advocate at Twitter and MIMS 2013 graduate, passionate about the open web and fostering strong communities. While in school, Dave worked as an Integration Engineer Contractor for the Mozilla Foundation, and interned at Common Crawl.

Prior to returning to school, Dave lived in Washington, DC and was an Assistant Director at the Maryland Institute for Technology in the Humanities (MITH) at the University of Maryland where among other duties he managed software development. A few years earlier he co-founded an unconference called THATCamp.

Much of my recent work is online, which you can find on his website, blog, or github.

Florian Leibert

Florian Leibert was an early engineer at Twitter where he helped build critical infrastructure for doing analytics and search. He was primarily responsible for Twitter’s user search product. After a few years at Twitter, Flo joined Airbnb and built out the data infrastructure team. Flo is an early leader in the Apache Hadoop community and has extensive experience using it at scale. He has published a number of patents and papers in the area of information retrieval and large-scale distributed systems. Flo was also the main author of Chronos, a fault tolerant job dependency scheduler and job orchestration framework built on top of mesos.