Concurrency for RaspberryPi using 0MQ

This month we have guest speaker Dave Jones talking about how he used 0MQ (zero-M-Q) to significantly improve the concurrency of a network of RaspberryPis.

Dave is a long-time member of Python North West and has given many talks for us in the past, which have always been interesting, informative, and entertaining. He's also heavily involved with RaspberryPi, developing several Python libraries for it. The issue he was trying to solve which required the use of 0MQ was one of effectiveness. A network of 20 RPis all linked together was being used to pre-build RaspberryPi Python package binaries with the goal of decreasing the amount of time any individual user had to wait for their own Pi to build the library when it was being installed - particularly problematic when using virtual environments where the same package might regularly need to be re-installed from source. The 20 Pi network was only performing twice as well as 1 Pi on its own, and the number of packages to pre-build was huge, so speed was a factor.

Dave's talk will cover the solution to the concurrency issue, the potential pitfalls - like achieving concurrency in an existing code-base without totally gutting everything for alternate async versions, and how to avoid the horrors of locking and race conditions when using threads. The answer to which, unsurprisingly given the title of the talk, turns out to be to use queues managed by 0MQ.

As always, the talk will be followed by a drink in the pub, hope to see you there.