This column appears in The Times today.
THE REPUTATION of exit polls was perceptibly if unfairly damaged by the US presidential election. But, as a writer in Fortune magazine points out, another predictor was unambiguously accurate. This was the electronic predictions market: the various websites allowing punters to place bets on the electoral outcome.
It was no fluke. An exchange in which many people try to outguess each other, each having a financial stake in the outcome, is difficult to beat. It exemplifies a phenomenon referred to on this page yesterday by Peter Riddell as the “wisdom of crowds”, after a book of that title by James Surowiecki, a New Yorker columnist.
Large groups are often castigated as irrational, conformist, pliable or atavistic. “How can 59,017,382 people be so DUMB?” ran a recent headline in the Daily Mirror. Knowledge, in its view, comes to those platonic guardians — the Mirror presumably included — who reflect hard and well enough on a problem.
A liberal society does not work like this. Knowledge is a process, not an outcome. The process is a critical exchange of views among the uninterested and the indolent as well as the educated and informed, yielding a collective wisdom. The evidence is that it works. This is because some problems have no predetermined set of solutions waiting to be discovered: continual experiment, rather than rational consideration of all the options and the selection of an ideal, is necessary. Experimenting is messy. It generates numerous possible choices, most of which are swiftly dismissed.
Take the stock market. It contains informed investors but also innumerable traders on sentiment and superstition. Even many professionals rely on an irrational forecasting technique known as “chartism”, the study of patterns in price charts (as opposed to tealeaves or the constellation of planets, which would make quite as much sense). Stock markets have periodic bubbles and crashes. Yet economies with liquid financial markets invariably turn out to be better at allocating scarce resources to productive uses than bureaucratic command economies.
Collective wisdom was invoked last year in the well-publicised (and swiftly aborted) Pentagon plan to create a futures market that would allow speculators to bet on the likelihood of terrorist attacks. The plan was unworkable. A market where a few people possess highly accurate information which is unavailable to other traders is easily manipulated. Terrorists could have spread false information by betting on incorrect outcomes.
But ideas that get killed are a path to wisdom. Their rejection ensures that the search for understanding flows into new tributaries. A liberal culture, informed by the collective wisdom, is continually seeking, whereas theocratic absolutists already know. That gives us the advantage.
