Leadership and the New Science: Discovering Order in a Chaotic World
It was during a conversation with a friend that Margaret Wheatley’s work first came up. In speaking of the non-linear and chaotic effects of change, he pointed specifically to Wheatley’s work in Leadership and the New Science: Discovering Order in a Chaotic World. The heart of the work is the deepening understanding of science that relies on probabilities and chaos to create the predictability that we expect. The work was published in 2006, and since its publication, we’ve learned more about the world we live in and how our belief in formulaic certainty remains a pervasive illusion.
Managing Change
I built the Confident Change Management course with a keen awareness that 70% of change projects fail. In fact, 70% of all large projects fail. They fail to complete their work on time or on budget, or they don’t accomplish the results that were hoped for. Wheatley expressed her confusion that we’d speak of change management – but that change seemed to be overwhelming us, making us feel less capable and more confused.
The answer seems to be found in the realization that change is more complicated and nuanced than we’d like to believe. The good news in the discovery of the chaos that underlies our beliefs of certainty is that the chaos itself has a certain order to it.
The Relationship to Leadership
As we move into deeper understanding, we recognize that everything is about relationships. Leadership is, as Burns and Rost said, about these relationships. (See Leadership and Leadership for the Twenty-First Century respectively.) These relationships hold electrons around the nucleus of an atom and the planets in orbit around the Sun. Even through subatomic particles, we find that relationships rule. Instead of individual parts with their own functions, we find that different parts operate as a part of the whole through their relationships.
These relationships weave together at higher levels of organization, as Richard Dawkins suggests in The Selfish Gene. Others’ work, like that in The Evolution of Cooperation, SuperCooperators, and Does Altruism Exist?, explains how cooperation is a better strategy for ingroups and competition is a better strategy for outgroups. Collectively, these strategies lead to greater success. However, the dynamics of these benefits change based on relationships. The more cooperative the relationship, the greater the results.
Change Happens when Necessary
There’s a venture capitalist way of thinking about opportunities as either vitamins or antibiotics. The point is that people buy antibiotics because they must – because there’s an immediate and critical motivating factor that causes them to take immediate action. Vitamins are, of course, a good idea, but they’re only a good idea when people have the resources for them and when they remember. The former are better candidates for investment (though this is only one dimension), because the demand system is built into the model.
In organizational leadership, organizations only change when they need to change to preserve at least some of themselves. Organizations have come and gone because they didn’t change fast enough. Organizations that did change may have changed so much that they’re very different organizations than when they started – but at least they’re still alive. Success stories for radical organizational transformation are less prevalent, but it happens.
When we look at the successes and failures for organizational transformation, we can hone in on timing. Organizations like Kodak had everything they needed to be successful. They invented digital photography, but they failed to see the need because their core photochemical business was so profitable. Organizations like IBM made it because of leadership that recognized the end was coming before it arrived.
Thus, even in the “necessity” category, some organizations won’t recognize the necessity quick enough and won’t pivot fast enough (because of capability or willpower). As a result, the organization may die. Resilient organizations are those that can detect when a change is necessary sooner – and can react quicker.
Probabilities Not Predictions
Jonathan Haidt proposes that the reason for consciousness is the ability to predict. (See The Righteous Mind.) The ability to predict future dangers justifies the massive cost that cognition requires. Our brains consume 20-30% of our glucose (energy) while accounting for only 2-3% of our body mass. They’re very expensive to operate even when they’re designed for energy savings. (See Thinking, Fast and Slow.) It’s easy to see how our evolutionary history would lead us to crave the ability to predict in every area of our life.
However, as we reach a subatomic (quantum) level, we see the language shift from prediction to probability. Instead of A+B=C, we get A+B will yield C 37.6% of the time. Gone are the assurances of certainty, replaced by the bookie’s statistics and probabilities.
Some would argue that we can achieve certainty. After all, we were taught that combining chemicals in a beaker would result in a new thing. Despite the inexperience of the high school class doing the work, the results largely came out as expected. The real difference between our high school chemistry and the lives we lead is scale. When you’re scaled to billions and billions of atoms, the result may not be what’s expected in every case – but the number of cases where it didn’t are rounding errors, undetectable, incomplete reactions. The other variable is time to completion. Normally, these reactions are complete in seconds or minutes – not the months or seasons that most leadership challenges take.
We don’t want to know what the average result is for a set of circumstances like ours. We want to know what will happen to our change, our initiative, our pet project. We’re stuck with probabilities, and that’s not satisfying.
It Feels
Some things are hard to measure. Some things that are measurable don’t have a measurement framework that make them useful. Gary Klein explains in Sources of Power about the fire captains that knew how fires work and how to direct firefighters safely into battle a blaze, but they couldn’t describe how they knew these things. Klein called it recognition-primed decisions (RPD). More broadly, in the field of knowledge management, it’s called tacit knowledge, and some of it is difficult, if not impossible, to articulate. (See Lost Knowledge.)
There’s a lot of interest, and some promising results, for artificial intelligence in the detection of medical diseases. (More specifically, it’s the machine learning branch of artificial intelligence.) The computer can consistently evaluate more criteria than a human can. They’re getting better detection rates. The problem – and one that we’re likely to wrestle with for a while – is that we can’t exactly explain what the models are looking for. There’s no clear set of articulatable criteria that the system is using. Even when we build systems to solve some problems, we can’t articulate why it works the way it does. Sometimes, intuition and “feel” is all we have.
However, the work on prediction tells us that our “feelings” are notoriously biased. They say that they’re too easy to lead us astray. So, while we must consider how we feel about something, we must simultaneously be suspicious. (For more, see The Signal and the Noise, Superforecasting, and Noise.)
Information as Nourishment
The Information Diet calls for conscious consumption of information. It uses a new metaphor. Instead of information being power, it’s nourishment. (Although it doesn’t express this quite as clearly as Wheatley.) Instead of looking at the collection of information to consolidate power, it’s a way to nourish and grow. The model exposes the downside of information. It’s possible to become overwhelmed by information. Like overeating, too much information can make it harder for us to do anything. The Age of Overwhelm focuses on the impacts of being overwhelmed and the need for better strategies for managing information.
Information managers, information architects, and librarians have long known that our current strategies for information management are failing us under the weight of an overwhelming amount of information. They’ve been working to improve the tools we have – but the rate isn’t keeping up. (See also The Organized Mind.)
I Crave Companions
Wheatley closes with a comment that is worthy of quoting: “I crave companions, not competitors.” We need more people who are willing to support us and to journey with us through Leadership and the New Science.