Chaos! (Or Maybe Not)
Several months ago, a colleague asked me if I could clarify some of the buzz he'd heard about the Standish Group's CHAOS report statistics on IT project success and failure. This wasn't the first time I'd heard grousing - from both the research community and the IT community - about this set of statistics, long considered to be the gold standard of project management research (at least for IT). But, it was the first time that, in looking for discussion about it online, I found that people were beginning to view the CHAOS Report critically. It turns out that, for many people in the project management discipline, the story told by the Standish numbers does not mirror the reality they perceive around them in their own workplaces.
As a project management researcher, I find the statistics baffling myself. At PM Solutions, we have seen, over the course of ten years of survey-based and action research, that things are improving on the project front. That is, when companies focus on improving project management, they are rewarded with measurable business results. This has been borne out by every study we have done, from the first (2000) Value of Project Management study to the just-completed State of the PMO 2010. Yet the 2009 Standish numbers showed a marked increase in failures. Could IT projects really be so different from all others?
My prejudice is that they are not, and at least one expert agrees with me. Christopher Sauer of Oxford University did a study some years ago suggesting how IT projects could learn from construction projects, and saying that the two were really not as different as it seemed. The difference, his research argued, was that the organizations that supported construction projects were more project-oriented ... that the way the sponsoring organization viewed IT projects might set them up to fail. Subsequently, Chris published on his research website PM Perspectives news of a study of IT projects that found results very different from Standish:the study found that two-thirds of IT projects succeeded ... with some out-performing their targets.
A significant difference in this study was the definition of success, which incorporated project benefits and business value to define the "star performers" in the study.
Indeed, the way one defines "challenged," "failed" or "successful" will skew the results one direction or the other. And, of course, these definitions can change from one company to another ... and certainly between IT and other types of projects.
For me, the crucial point is that, until companies have established good measurement baselines so that they know how projects were performing to begin with, they won't know to what precise extent the application of discipline, methodology, training or structure yields concrete benefits. This knowledge allows companies to make informed decisions about where to invest their energies and money. It also allows them to know a failure when they see one.
Read up on this controversy at the links below:
"Crisis" in IT ... or Not?
Then let us know: What's your experience? Do one-third of projects truly "fail"?
About the Author
Jeannette Cabanis-Brewin
Jeannette Cabanis-Brewin is editor-in-chief for PM Solutions Research, and the author, co-author and editor of over twenty books on project management, including the 2007 PMI Literature Award winner, The AMA Handbook of Project Management, Second Edition.
View Posts by Jeannette Cabanis-Brewin
Jeannette Cabanis-Brewin
Tags:
(12 votes. Average: 2.5 out of 5)
4 Comments on Chaos! (Or Maybe Not)
Popular Posts
- The Project Management Maturity Model … Now with Agile/Adaptive Assessment!
- PM Solutions Celebrates 25 Years of Project Management Excellence
- Managing Project Managers Within and Outside the PMO
Popular Categories
- Agile or Adaptive PM (5)
- Culture & Change Management (75)
- Project & Program Management (82)
- Portfolio Management (50)
- Project Management Office (PMO) (107)
- Project Management Research (50)
- Resource Optimization (26)
search blog:
Sign up for PM Solutions'
Insights eNewsletter
Delivered every other month in addition to periodic research, white papers, and news alerts.
Jim Sloane says:
I think most people remember the 1998 chaos theory and think of its results, here and now in 2009, and will still in 2010 and beyond.
Their (standish Group) latest version shows progress, as we have done better and better jobs of managing projects. THis makes sense to me.
JIm
Posted on April 30, 2010 at 4:08 pm
Jeannette Cabanis-Brewin says:
Quite true: over the whole span of the Standish research, there’s been improvement ... but between the 2009 report and the one previous to it, there was an uptick in the failure, and this is what people are reacting to/perplexed by, I think.
Posted on May 3, 2010 at 2:23 pm
Glen B Alleman says:
Starting in 2005, there was discussion with the difficulties in the Standish report
http://herdingcats.typepad.com/my_weblog/2005/07/finally_a_chall.html
The most recent discussion comes from a variety of sources
http://herdingcats.typepad.com/my_weblog/2010/02/standish-report.html
What has come now is the understanding that the basic statistics of the Standish approach are seriously flawed. The result is a forecast of failure rates that is not credible.
In the end Standish has committed a simple error, they were asleep in the high school stats class.
Posted on May 8, 2010 at 10:52 pm
Jeannette Cabanis-Brewin says:
Thanks Glen for reminding me of the Herding Cats site. I used to read it quite often.
Posted on May 10, 2010 at 9:26 pm