Friday, June 1, 2007

Book Review: The Wisdom of Crowds. Why the Many are smarter then the few by James Surowiecki

Under the right circumstances groups are remarkably intelligent and are often smarter than the smartest people in them. Groups do not need to be dominated by exceptionally intelligent people in order to be smart. Even if most of the people within a group are not especially well informed or rational, it can still reach a collectively wise decision.

When our imperfect judgements are aggregated in the right way, our collective intelligence is often excellent.

The argument of this book is that chasing the expert is a mistake and a costly one. We should stop hunting and ask the crowd (which of course, included the geniuses as well as everyone else). And the chances are, it knows.

If you put together a big enough and diverse enough group of people and ask them to ‘make decisions affecting matters of general interest’, that groups decisions will, over time, be ‘intellectually superior to the isolated individual’, no matter how smart or well-informed he is.

4 conditions that characterize wise crowds
  1. Diversity of opinion (each person should have some private information even if its just an eccentric interpretation of the known facts)
  2. Independence (peoples opinions are not determined by the opinions of those around them)
  3. Decentralization (people are able to specialize and draw on local knowledge)
  4. Aggregation (some mechanism exists for turning private judgments into a collective decision)

If a group satisfies the above conditions, its judgment is likely to be accurate. Why?
If you ask a large enough group of diverse, independent people to make a prediction or estimate a probability and then average those estimates the errors each of them makes in coming up with an answer will cancel themselves out

You could say its as if we’ve been programmed to be collectively smart.

Grouping only smart people together doesn’t work that well because the smart people tend to resemble each other in what they can do.

The odds of a homogeneous group of people reaching a good decision are slim at best.

Groups are better at deciding between possible solutions to a problem than they are at coming up with them. Invention may still be an individual enterprise (largely!!!) but selecting among inventions is a collective one.

One key to successful group decisions is getting people to pay much less attention to what everyone else is saying.

The Linux way


Unlike Windows which is owned by Microsoft and worked on only by Microsoft employees, Linux is owned by no one. When a problem arises with the way Linux works, it only gets fixed if someone, on his own, offers a good solution. There are no bosses ordering people around, no organizational charts dictating peoples responsibilities. Instead, people work on what they’re interested in and ignore the rest. This seems like – in fact it is – a rather haphazard way to solve problems. But so far at least, it has been remarkably effective, making Linux the single most important challenger to Microsoft.

What decentralization offers Linux is diversity. In a traditional corporate model, top management hires the best employees it can, pays them to work full-time, generally gives them some direction about what problems to work on and hopes for the best. That is not a bad model. It has the great virtue of making it easy to mobilize people to work on a particular problem and it also allows companies to get very good at doing the things they know how to do. But it also necessarily limits the number of possible solutions that a corporation can come up with, both because of mathematical reality (a company has only so many workers and they have only so much time) and because of the reality of organizational and bureaucratic politics. Linux, practically speaking doesn’t worry much about either. Surprisingly there seems to be a huge supply of programmers willing to contribute their efforts to make the system better. That guarantees that the field of possible solutions will be immense. There’s enough variety among programmers and there are enough, that no matter what the bug is, someone is going to come up with a fix for it. And there’s enough diversity that someone will recognize bugs when they appear. “Given enough eyeballs all bugs are shallow.”

The group’s collective solution may well be smarter than even the smartest persons solution.

In small groups, diversity of opinion is the single best guarantee that the group will reap benefits from face-to-face discussion. The presence of a minority viewpoint all by itself makes a groups decisions more nuanced and its decision-making process more rigorous. This is true even when the minority viewpoint turns out to be ill conceived. The confrontation with a dissenting view, logically enough forces the majority to interoate its own positions more seriously.

Group dynamics

All evidence suggests that the order in which people speak has a profound effect on the course of a discussion. Earlier comments are more influential and they tend to provide a framework within which the discussion occurs. Once that framework is in place, its difficult for a dissenter to break it down. This wouldn’t be a problem if the people who spoke earliest were also more likely to know what they were talking about. But the truth is that, especially when it comes to problems where there is no obvious right answer, there’s no guarantee that the most–informed speaker will also be the most influential. On juries, for instance, two-thirds of all foremen – who lead and structure discussions – are men, and during deliberations men talk far more than women do, even though no one has ever suggested that men as a gender have better insight into questions of guilt and innocence. In groups where the members know each other status tends to shape speaking patterns, with higher-status people talking more and more often than lower-status people. Even when higher-status people don’t really know what they’re talking about, they’re more likely to speak.

In small groups, ideas often do not succeed simply on their own merits. Even when its virtues might seem self-evident, an idea needs a champion in order to be adopted by the group as a whole. And when advocates are chosen on the basis of status or talkativeness, rather than perceptiveness or keenness of insight, then the groups chance of making a smart decision shrinks.

If you tend to talk a lot in a group, people will tend to think of you as influential almost by default. Talkative people are not necessarily well liked by other members of the group, but they are listened to. People at the center of the group tend to become more important over the course of a discussion.






If small groups are included in the decision-making process, then they should be allowed to make decisions. If an organization sets up teams and then uses them for purely advisory purposes, it loses the true advantage that the team has: namely, collective wisdom.

The idea of the wisdom of the crowds is not that a group will always give you the right answer but that on average it will consistently come up with a better answer than any individual could provide.

If groups on the whole are relatively intelligent (as we know they are) then there’s a good chance that the stock price is actually right. The problem is that once everyone starts piggybacking on the wisdom of the group, then no one is doing anything to add to the wisdom of the group.

Groups are only smart when there is a balance between the information that everyone in the group shares and the information that each of the members of the group holds privately. It’s the combination of all those pieces of independent information, some of them right, some of them wrong, that keeps the group wise.



In a crowd of people, there are some people who will never riot, and some people who are ready to riot at almost any time – these are the instigators. But most people are somewhere in the middle. A groups willingness to riot depends on what other people in the crowd are doing. As more people riot, more people decide that they are willing to riot too (Think of what Andy Serwer wrote about the stock market boom: “the more stocks go up the more of us get into the market.”)

What matters is what combination of people are in the crowd. If there are a few instigators and lots of people who will act only if a sizeable percentage of the crowd acts, its likely nothing will happen. For a crowd to explode, you need instigators, “radicals” – people with low thresholds for violence – and a mass of people who can be swayed. Once a crowd crosses the threshold into violence, its behavior is shaped by its most violent members. If the image of collective wisdom that informs much of this book is the average judgment of the group as a whole, a mob is not wise. Its judgment is extreme.

The analogy to a stockmarket bubble is obvious: the more investors who refuse to buy stocks just because the other people are buying them, the less likely it will be that a bubble will become inflated.

No comments: