Apache Spark is a next-generation tool that makes ground-breaking improvements over MapReduce on Hadoop. It lets developers be more productive and use their resources more efficiently. But it's less known that the authors of Mesos introduced it in their paper as a framework "to validate their hypothesis".
We built a distribution of Spark at Typesafe, exploiting the synergy with Mesos. We will show how the combination is ready for multi-tenant, heterogeneous cluster environments. We'll see how they fit into a larger picture, along side YARN or containers. And we'll have a look at enterprise use cases that drove some of our choice. Finally, we'll also mention the bumps on the road, and improvements we would like to see to make the Spark and Mesos combination even more versatile and powerful.
