The whole point is that statistics is that you cannot in interpret them any way you like. The numbers either support your claim or they do not. Regardless of who is making the claim, anybody can disprove it.
Consider the recent case of Reinhart, Rogof and Herndon:
This week, economists have been astonished to find that a famous academic paper often used to make the case for austerity cuts contains major errors. Another surprise is that the mistakes, by two eminent Harvard professors, were spotted by a student doing his homework.
In the case of the Chaos report we are free to criticise constructively because the data is available. You can see, for example, that it is based on a survey of project managers and therefore measures perception, not objective reality. Data and method are presented for you to pick apart.
How do you critique this claim?
Today it is understood that productivity of agile teams is 25 to 50 per cent higher than teams that follow traditional waterfall software development model. But it will alarm you if I say that there are studies done by Dr. Jeff Sutherland which show that some agile teams are 400 to 800 per cent more productive than their traditional counterparts! So, what are these teams doing different that makes them hyper-productive? Here are 5 steps which an ardent agile follower would agree with: -
25% to 50% higher productivity? Where does that come from?
Then a claim of teams achieving 400% to 800%. That's a shocking claims. How is it backed up? With a link Sutherlands Wikipedia entry, an entry that makes no reference to these studies.
What follows? A simple description of Scrum.
On 18 June[masked]:06, Stephen Masters <[address removed]>
Of course, such statistics can be interpreted any way you like. And those Chaos reports tend to take a lot of liberties in interpreting the data, to make their point.