Why the 70% Failure Rate of Change Projects Is Probably Right
There are some folks in the change management community and profession who rail against those who are willing to quote a 70% failure rate. They believe they’re uneducated, and they don’t understand that the numbers aren’t supported by science. This is my response: I understand enough about science and statistics to say with reasonable confidence that the number is within the ballpark of correct – given a set of constraints.
Let’s tear this down to explain what the 70% failure rate for change projects means and how, in one critical definition, a 70% failure rate is generous.
What is a Change Project?
Here’s the first issue we have. Change projects can be anything from a change that has no employee impact – such as upgrading from one email server to the next version – to the kinds of continuous quality improvement exercises that Deming taught us to do and the kinds of change that are transformative to the organization. To be clear, I’m confining my responses here largely to transformative change – the kind that the 70% estimate was initially intended to represent.
While the other kinds of change projects are quite valid and correct, they weren’t what the original proponents of the 70% number had in mind.
On Time, On Budget, and On Value
The starting point for our evaluation of change project success must be overall success of the project when measured as to whether the project was delivered complete, on time, and on budget. The research by McKinsey and the University of Oxford suggests that largescale software projects are 45% over budget, 7% over time, and deliver 56% less than predicted. (See Delivering large-scale IT projects on time, on budget, and on value.) The publicly available data doesn’t explain which projects each apply to, but if we apply an even distribution, we get roughly 78% of projects with some sort of failure to meet all three criteria (108% total minus overlap of ~29%).
One could argue all projects are not like IT projects – and I’d easily agree. I’d say the kinds of large-scale IT projects that were being studied had some degree of organizational transformational aspect to them. If I had to venture a guess, I’d say that organizational transformation is harder than IT projects. But if we use just these criteria that have a research study behind it, we’re over the 70% failure rate that is used as an estimate.
A Lack of Research
The primary argument leveled against the 70% failure rate has been that it wasn’t based on hard research. Anyone who knows anything about statistics should know this instantly. Statistics almost never land on round numbers. For instance, Edgar Dale’s cone of experience has been widely misinterpreted over the years. The idea is that some people learn one way or another. Percentages were applied to the cone at some point, and they stuck – despite the fact that Dale himself didn’t add them, and their round numbers stand out as having limited possibility to be based on research.
In short, if the percentage ends in a zero, it’s an estimate – not a statistical proof. (See Superforecasting for more.)
However, I can, of course, look for other studies for other kinds of projects with varying results. The one I quoted above was literally the first result I got back when I queried for project failure rates – in short, Google thinks it’s representative of the kinds of results people are looking for.
Establishing Criteria
What’s wrong with this? How is it that the numbers can be so bad? The answer is in the standard that I used for success. I said everything had to go right. The reality of organizational change is that nothing goes right all the time; there are always issues. If I relax the criteria to just budget – or just the projected features are present in the software – well, the numbers get better.
The truth is that we’re simply talking about the criteria that project managers use for success. Take the Sydney Opera House. They spent 17x the initial planned budget – but few people would consider it a failure. The result is, then, that the criteria being used for success may – or may not – be the right criteria or at least the criteria that others are using.
Conflict of Observation
The second major issue that is raised when disagreeing with the estimated 70% failure rate is that it doesn’t match consultants’ observations. Because it doesn’t match their success rate, it must not be right. The problem is that the consultants have fallen into the trap of sampling bias. That is, their clients are the ones who are committed to the change enough to pay for the consultant. They’re forward thinking enough to realize that they need help. These are the kinds of organizations that are most likely to succeed.
So even if the consultants were completely ineffective – as some are – they wouldn’t see a 70% failure rate. Their clients have self-selected themselves into a premium position of caring and therefore doing better than the average.
Read the Journal Articles
The most tragically humorous aspect of the arguments that 70% of change initiatives don’t actually fail is that the authors of such arguments seem to have failed to completely read the entire journal article they’re citing to refute the 70% number. The article “Do 70 Per Cent of All Organizational Change Initiatives Really Fail?” by Mark Hughes points to the problem of identifying what we mean by failure – and what we mean by change.
Hughes’ point isn’t that the number isn’t as high as 70% – his point is simply that the number itself is unsupported by research. It could be that 90% of change initiatives fail when measured by project management terms and when they’re transformative. Hughes simply doesn’t know. Neither do I, but in the spirit of Douglas Hubbard’s work, How to Measure Anything, I can make a reasonable statement that the initial estimate of 70% is reasonable – given a tight definition of what success is.
So the next time someone tells you that 70% of change initiatives fail, you can answer that there’s research that says many large scale projects fail at a similar rate with the tight criteria of on time, on budget, and on value. There’s no reason to believe change management projects are any different.
