Past Meetup

Mesos - A Platform for Fine-Grained Resource Sharing in the Data Center

This Meetup is past

27 people went

Location image of event venue

Details

Abstract

We present Mesos, a platform for sharing commodity clusters between multiple diverse cluster computing frameworks, such as Hadoop and MPI. Sharing improves cluster utilization and avoids per-framework data replication. Mesos shares resources in a fine-grained manner, allowing frameworks to achieve data locality by taking turns reading data stored on each machine. To support the sophisticated schedulers of today's frameworks, Mesos introduces a distributed two-level scheduling mechanism called resource offers. Mesos decides how many resources to offer each framework, while frameworks decide which resources to accept and which computations to run on them. Our results show that Mesos can achieve near-optimal data locality when sharing the cluster among diverse frameworks, can scale to 50,000 (emulated) nodes, and is resilient to failures.

Presenter:

Ankur Chauhan

Links:

https://www.cs.berkeley.edu/~alig/papers/mesos.pdf

http://mesos.apache.org

List of Mesos Frameworks (http://mesos.apache.org/documentation/latest/mesos-frameworks/)