Workload Distribution

Pizza, beer, and mingling.

"Mesos - Cluster Management @ Twitter"
Benjamin Hindman (https://www.meetup.com/SF-Bay-Area-Large-Scale-Production-Engineering/events/97563092/www.linkedin.com/pub/benjamin-hindman/9/b21/880) - Tech Lead, Twitter

"Reliable JVM Concurrency & Distribution with Akka"
Brendan McAdams (http://www.linkedin.com/in/bwmcadams) - Consultant/Instructor, Typesafe

Scaling an application requires distributing the work across multiple servers to get more done. For a web tier, this is pretty well understood, and we use load balancers (hardware or software) to spray requests across the tier.

For other types of work, a load balancer may not be enough. Your application may require acknowledgement of completion, or may require the ability to chain together many events. Stateful processing may also have other needs, such as assuring data co-locality in order to avoid sending large messages around.

This month's theme is "Workload Distribution." I'd like to hear about the kinds of tasks folks are distributing throughout their applications, and how they are doing it. There are a growing number of middleware components for managing this; a few examples include Apache Mesos (http://incubator.apache.org/mesos/), Gearman (http://gearman.org/), and Akka (http://akka.io/).

Do you have an application that is using this technique to distribute work across a set of hosts? What is the workload, and how do you distribute it? Can you tell us what led you to use the framework you chose? What do you like or dislike about it? What is the programming model like? How easy/hard is it to scale the system by adding more nodes? How do you tell when the system needs more resources?

For our March event, I'd like to see talks about Workload Distribution, who's doing it, and how they're doing it.

I'm looking for 2-4 20-25 minute talks. If you can give a talk, please contact me, Chris Westin, through meetup.

As well as the evening's theme talks, we can fit in 2-3 five minute lightning talks at the beginning of the evening; any topic that would be interesting to the #lspe audience is welcome. If you're interested in giving a lightning talk, contact me, Chris Westin, through meetup.

Note that the March event we will be in San Francisco, where Twitter have offered to host us. We know there are folks in SF that follow us and would like to attend, our events, so this is your chance! Twitter is in suite 900, you will need to register at reception, and show ID; I'm told this will go faster if you've RSVPed. Please allow time for that. Also note that the venue is a little smaller than our usual, so please update your RSVP if your situation changes and you cannot attend.