January's Code Share: Concurrency

The Concurrency Revolution

At the end of 2004, in his article "The Free Lunch is Over" (http://www.gotw.ca/publications/concurrency-ddj.htm), Herb Strutter wrote:

"Concurrency is the next major revolution in how we write software... concurrency is of the same order as OO both in the (expected) scale of the revolution and in the complexity and learning curve of the technology." The world of hardware has progressed rapidly, and we enter 2012 with dual core (http://www.t3.com/features/best-phones-2011-dual-core-smartphones-to-buy) phones in our pockets. Is our software keeping up with the pace?

Before our software can be concurrent we programmers must learn to think concurrently. Strutter continues:

"Probably the greatest cost of concurrency is that concurrency really is hard... Everybody who learns concurrency and thinks they understand it, ends up finding mysterious races they thought weren’t possible, and discovers that they didn’t actually understand it yet after all" What is the best way to tackle this difficult problem? What affect does the need for concurrecny have on our code? Should we turn to functional programming (http://www.ibm.com/developerworks/java/library/j-fp/index.html) and languages such as Clojure (http://clojure.org/) and Scala (http://www.scala-lang.org/). In his MSDN Blog (http://blogs.msdn.com/b/brandonwerner/archive/2008/09/16/the-rise-of-functional-programming-f-scala-haskell-and-the-failing-of-lisp.aspx) Brandon Werner observes:

In the living world of mutable objects and variables, breaking out two transactions to work concurrently that operate on the same living data is a bad idea. Add structural programming's solution to this problem, optimistic and pessimistic locking, and you have dead-locks in short order. Functional programming has been a natural place to explore parallel processing and new ways of doing atomic transactions because of the reasons above. An alternative approach is Mechanical Sympathy (http://mechanical-sympathy.blogspot.com/), gaining an understanding of what is happening within the machine allows us to understand the pitfalls and identify the opportunities. In his article (http://martinfowler.com/articles/lmax.html) about the LMAX Disruptor architecture (http://code.google.com/p/disruptor/) Martin fowler notes that this is essential for successful performance testing:

Mechanical sympathy is important to developing the right tests. Testing a low level concurrency component is meaningless unless you take into account the caching behaviour of the CPU. Let's get together and look at the approaches we are taking for concurrency. What does it look like in the code we are writing? Do we have to sacrifice readability? Is it necessary to learn a new language? Is an understanding of the Java Memory Model (http://java.sun.com/docs/books/jls/third_edition/html/memory.html) essential?

Coding Challange: Test Driving Concurrency

Testing for concurrency problems is difficult because the fault may only occur under specific circumstances.

The challange for this month is to demonstrate some test driven development for concurrency problems.

Can you write a test case that succinctly demonstrates the following concurrency problems?

Race Conditions Deadlock Starvation Nondeterminism You may find the sample third chapter from "High-Performance Java Platform Computing" useful: Race Conditions and Mutual Exclusion (http://java.sun.com/developer/Books/performance2/).

Here is my attempt. Clearly we can be do better. Is jUnit the right tool? Do other languages make it easier?

import org.junit.Test; public class TestDrivingConcurrency { @Test() public void singleThreading() { new BuildTestString(0).start(); while(results[0] == null) {} org.junit.Assert.assertEquals("0123456789", results[0]); } @Test() public void multiThreading() { new BuildTestString(1).start(); new BuildTestString(2).start(); new BuildTestString(3).start(); new BuildTestString(4).start(); while( results[1] == null || results[2] == null || results[3] == null || results[4] == null) {} org.junit.Assert.assertEquals("0123456789012345678901234567890123456789", results[1]+results[2]+results[3]+results[4]); } static int i; static String result; static String[] results = new String[5]; class BuildTestString extends Thread { int resultIndex; public BuildTestString(int resultIndex) { this.resultIndex = resultIndex; } public void run() { for(i = 0, result = "";i<=9;i++) { try { sleep(100); } catch (InterruptedException e) { org.junit.Assert.fail(e.getMessage()); } result+=i; } results[resultIndex] = result; } } } It gives the following failure:

org.junit.ComparisonFailure: expected:<01234567890123456789[01234567890123456789]> but was:<01234567890123456789[10012345678910111201234567891011]> at org.junit.Assert.assertEquals(Assert.java:123) at org.junit.Assert.assertEquals(Assert.java:145) at TestDrivingConcurrency.multiThreading(TestDrivingConcurrency.java:27) What's Going To Happen?

On Monday 9th January, a couple of days before the share, we'll send out an email to everybody who has signed up. If you have any code to contribute please send it in a reply to that email.

After the event we will be heading to the Half Moon, 213-223 Mile End Road, Mile End, Greater London, E1 4AA - http://www.jdwetherspoon.co.uk/home/pubs/the-half-moon - for drinks/networking.

This is a joint event with the Graduate Developer Group - https://www.meetup.com/grad-dc/events/44932902/ - providing an opportunity for both experienced developers and newcomers to the profession to share points of view.

Please Note -

At previous events LJC members have found this venue/particular room hard to find.

This event is being held in the Francis Bancroft Building - Room Number - FB 1.13.

Please see this map for further information (the building is numbered 26 on this map) http://www.qmul.ac.uk/docs/about/26065.pdf

The actual signs at the building will say 'Bancroft' building only rather than Francis Bancroft.

Directions -

From Mile End Station you will need to turn left out of the station and then continue for 5 mins approx. up the Mile End Road and then cross over the road to the QM campus and go into the East Gate. This is the first entrance to QM you come to when coming from the direction of Mile End Tube. Continue straight on up Westfield Way for a couple of minutes and then bear left at the Curve cafe. The Bancroft Building is on your right (number 26 on the map).