The Wisdom of Crowds
I was recently reading a blog post titled Mild Super Power on the Dilbert Blog. It made me think about how there are sometimes more factors at play than may be immediately obvious. While I don’t subscribe to Scott Adam’s claim that he can avoid speeding tickets because he understands the economics of placing police cars at various points – mainly because economists assume that people are rational and we all know that people aren’t. I’d also suggest that there are other factors – such as the desire to get some sleep on the job – which may play a factor in where cars are positioned to catch speeders.
One of Scott’s statements which rings very true is “My reason for majoring in economics in college was to understand how the world works, so I would be more equipped to navigate in it.” I think this is a singular point of clarity – choosing to use your educational opportunities to help you better understand (and therefore navigate) the world around you. I didn’t choose to major in economics – in fact, it was a topic that until quite recently that I cared very little about. Recently however, the emergent behaviors of groups – that is complex (or different) patterns that emerge when many individual actors follow simple behavior rules – has become quite interesting.
The Wisdom of Crowds holds a similar mystery to me. It illuminates how crowds can be very (hyper) intelligent and why sometimes they’re not. Traditional thinking (in the US at least, I can’t speak for other areas of the world) is that you get together a group of exceptionally smart people and you put them in a room and eventually you’ll get something good out of them. However, this necessarily is restrictive. There are only so many people that are included. There’s always some bar of “exceptionally smart” vs. “really smart”.
The fundamental premise for the book is that there can be a lot of value to allowing larger groups of people work on the same problem. That the individual errors that are contained in their solutions tend to cancel each other out. If you read Stumbling on Happiness you’ll know that we all have a very poor ability to predict the future because we tend to leave things out of our mental images. I suspect that we have similar foibles when it comes to our ability to solve problems.
Fundamentally the author (James Surowiecki) suggests that there are three types of problems cognition (a problem of knowledge), coordination (getting everything to fit together), and cooperation (getting everyone to get along.) He illustrates how groups solve the different types of problems differently.
One of the interesting tangents in the book is a discussion – in a few places – about Tacit knowledge. That is knowledge gained by doing. I help the church I attend by supporting the technology around the delivery of Sunday services. I’ve learned tons about the subtleties of music and of sound than I could have ever learned via reading a book. (And I learn really well by reading.) I’ve learned by having to run the sound board. This knowledge isn’t easily capture and shared with others.
One of the considerations for problem solving large coordination and cooperation problems is the influence of tacit knowledge – you have to have “been there and done that.” When you get the people together to solve problems those who’ve actually done it have the greatest chance of coming up with solutions that will work.
I’m reminded of a story about the infamous Skunk Works at Lockheed Martin. That’s where Kelly Johnson and his team came up with some of the most sophisticated and radical aircraft designs of the time including the U-2, F-117, and my personal favorite plane the SR-71 (Blackbird). The story is that when they were building one of the prototype aircrafts one of the machinists came up with a problem because one of the parts wouldn’t fit quite right. An engineer walked out on the manufacturing floor, bent a piece of cardboard and told the machinist to “make it like that” – and asked the machinist to return the cardboard to the engineer because he’d have to draw the part later. What’s striking to me about this is that the two different disciplines (engineering and machining) met at the point where the problem surfaced and they both exercised their tacit knowledge about the situation to solve the problem – without a hierarchy of structure, without bureaucracy, just taking the best of what (arguably the smallest) crowd knew.
If you find the ability of a crowd to create a solution that no one person could come up with – or don’t believe that a pool of different people with different perspectives and experiences could be smarter than 99% of everyone out there – you need to read The Wisdom of Crowds.