The Wisdom of Crowds

Perhaps we as Knights of Moleskine, Spirit and Ale are wiser than we think! Or, is that we think wiser because we're Knights of Moleskine, Spirit and Ale? That is what James Surowiecki thinks. He belongs to the New Yorker crowd. Sir Dayvd of Oxfordshire recommends this book. I haven't read it yet; however, checking out the website, the Q&A section did peak my interest. Here that is for your review and future wisdom. Cheerio, Sir Hook is Getting Wiser of Warrick.

Q & A with James Surowiecki

How did you discover the wisdom of crowds? The idea really came out of my writing on how markets work. Markets are made up of diverse people with different levels of information and intelligence, and yet when you put all those people together and they start buying and selling, they come up with generally intelligent decisions. Sometimes, though, they come up with remarkably stupid decisions—as they did during the stock-market bubble in the late 1990s. I was interested in what explained the successes and the failures of markets, and as I got further into it I realized that it wasn't just markets that were smart. In fact, crowds of all sorts were often remarkably wise. Could you define "the crowd?" A "crowd," in the sense that I use the word in the book, is really any group of people who can act collectively to make decisions and solve problems. So, on the one hand, big organizations—like a company or a government agency—count as crowds. And so do small groups, like a team of scientists working on a problem. But just as interested—maybe even more interested—in groups that aren't really aware themselves as groups, like bettors on a horse race or investors in the stock market. They make up crowds, too, because they're collectively producing a solution to a complicated problem: the bets of people betting on a horse race determine what the odds on the race will be, and the choices of investors determine stock prices. Under what circumstances is the crowd smarter? There are four key qualities that make a crowd smart. It needs to be diverse, so that people are bringing different pieces of information to the table. It needs to be decentralized, so that no one at the top is dictating the crowd's answer. It needs a way of summarizing people's opinions into one collective verdict. And the people in the crowd need to be independent, so that they pay attention mostly to their own information, and not worrying about what everyone around them thinks. And what circumstances can lead the crowd to make less-than-stellar decisions? Essentially, any time most of the people in a group are biased in the same direction, it's probably not going to make good decisions. So when diverse opinions are either frozen out or squelched when they're voiced, groups tend to be dumb. And when people start paying too much attention to what others in the group think, that usually spells disaster, too. For instance, that's how we get stock-market bubbles, which are a classic example of group stupidity: instead of worrying about how much a company is really worth, investors start worrying about how much other people will think the company is worth. The paradox of the wisdom of crowds is that the best group decisions come from lots of independent individual decisions. What kind of problems are crowds good at solving and what kind are they not good at solving? Crowds are best when there's a right answer to a problem or a question. (I call these "cognition" problems.) If you have, for instance, a factual question, the best way to get a consistently good answer is to ask a group. They're also surprisingly good, though, at solving other kinds of problems. For instance, in smart crowds, people cooperate and work together even when it's more rational for them to let others do the work. And in smart crowds, people are also able to coordinate their behavior—for instance, buyers and sellers are able to find each other and trade at a reasonable price—without anyone being in charge. Groups aren't good at what you might call problems of skill—for instance, don't ask a group to perform surgery or fly a plane. Why are we not better off finding an expert to make all the hard decisions? Experts, no matter how smart, only have limited amounts of information. They also, like all of us, have biases. It's very rare that one person can know more than a large group of people, and almost never does that same person know more about a whole series of questions. The other problem in finding an expert is that it's actually hard to identify true experts. In fact, if a group is smart enough to find a real expert, it's more than smart enough not to need one. Can you explain how a betting pool can help predict the future? Well, predicting the future is what bettors try to do every day, when they try to figure out what horse will win a race or what football team will win on Sunday. What horse-racing odds or a point spread represent, then, is the group's collective judgment about the future. And what we know from many studies is that that collective judgment is often remarkably accurate. Now, we have to be careful here. In the case of a horse race, for instance, what the group is good at predicting is the likelihood of each horse winning. The potential benefits of this are pretty obvious. If you're a company, say, that's trying to decide which product you should put out, what you want to know is the likelihood of success of your different options. A betting pool—or a market, or some other way of tapping into the wisdom of crowds—is the best way for you to get that information. Can you give an example of a current company that is tapping into the "wisdom of crowds?" There's a division of Eli Lilly called e.Lilly, which has been experimenting with using internal stock markets and hypothetical drug candidates to predict whether new drugs will gain FDA approval. That's an essential thing for drug companies to know, because their whole business depends on them not only picking winners—that is, good, safe drugs—but also killing losers before they've invested too much money in them. You've explained how tapping into the crowd's collective wisdom can help a corporation, but how can it help other entities, like a government, or perhaps more importantly, an individual? Well, the same principles that make collective wisdom useful to a company make it just as useful to the government. For instance, in the book I talk about the Columbia disaster, showing how NASA's failure to deal with the shuttle's problems stemmed, in part, from a failure to tap into knowledge and information that the people in the organization actually had. And in a broader sense, I think the book suggests that the more diverse and free the flow of information in a society is, the better the decisions that society will reach. As far as individuals go, I think there are two consequences. First, we can look to collective decisions—as long as the groups are diverse, etc.—to give us good predictions. But the collective decisions will only be smart if each of us tries to be as independent as possible. So instead of just taking the advice of your smart friend, you should try to make your own choice. In doing so, you'll make the group smarter. When you talk about using the crowd to make a decision, are you talking about consensus? No, and this is one of the most important points in the book. The wisdom of crowds isn't about consensus. It really emerges from disagreement and even conflict. It's what you might call the average opinion of the group, but it's not an opinion that every one in the group can agree on. So that means you can't find collective wisdom via compromise. What would Charles MacKay—the author of Extraordinary Popular Delusions and the Madness of Crowds—think of your book? He would probably think I'm deluded. Mackay thought crowds were doomed to excess and foolishness, and that only individuals could produce intelligent decisions. On the other hand, a good chunk of my book is about how crowds can, as it were, go mad, and what allows them to succumb to delusions. Mackay would like those chapters. What do you most hope people will learn from reading THE WISDOM OF CROWDS? I think the most important lesson is not to rely on the wisdom of one or two experts or leaders when making difficult decisions. That doesn't mean that expertise is irrelevant, or that we don't need smart people. It just means that together all of us know more than any one of us does.