Forecasts are useless and behavioural economists know that people see patterns that aren’t there
An old horse-racing tipster scam takes the following elegant form: send predictions about the winner in a 10-horse race to 10,000 people, with 10 different predictions, each sent to 1,000 people. After the race, focus on the 1,000 who received a successful prediction and send each of them a prediction of the winner in another 10-horse race; again, 10 different predictions, equally spread. After the second race, 100 people will have received two successive winning predictions and will be unaware of the 9,900 who have not. As a final flourish, forecast another 10-horse race and you will have 10 people, each of whom has received 10 three successive correct forecasts against substantial odds. Then simply write to each of them and ask for a few thousand pounds in exchange for your next three tips.
Punters can be forgiven, I feel, for falling for such nonsense – because it is at least cleverly constructed nonsense. But a recent working paper, written by two behavioural economists, Nattavudh Powdthavee and Yohanes Riyanto, makes me wonder whether such classic scams are overkill. They conducted a laboratory experiment (actually, two: one in Thailand and one in Singapore, both with undergraduate students as subjects), which duplicated the old fraud. The twist was that the mechanics of the trick were entirely transparent. The tips were given in sealed, numbered envelopes – each set of envelopes unique to each student.
Instead of horse-racing, the students were shown coin-flips and given a number of good reasons to believe that the coin-flips were random: the coins came from the participants, not the experimenters; the coins were changed every couple of flips; participants, rather than the experimenters, would perform the actual flips. The students were told that each numbered envelope contained a forecast of the next coin flip.
The students were given tokens to gamble with and invited to bet on each coin-flip, with the stake to double or to disappear. The students were also invited to pay a fixed price to look inside each envelope ahead of time. After each coin-flip, the students could open the prediction for free and see whether it was correct or not.
You can appreciate that the forecasts here are transparently useless. With almost 400 students, some were bound to witness a string of correct forecasts by chance. The question is, would the students who randomly received correct predictions through sheer fluke actually start to pay for future predictions? And how long would it take for them to start buying?
The researchers answer these questions pithily: “Yes, and not long.” After witnessing a single correct forecast, students were more likely to pay to see a second forecast; this effect becomes large and statistically very significant after a second correct forecast. After witnessing four correct coin-toss forecasts, more than 40 per cent of students were willing to pay to see the fifth, although the chance of four correct predictions is a not-exactly-stunning one in 16.
In some senses this should be no surprise. Behavioural economists and psychologists have known for some time that people see patterns that just aren’t there. Powdthavee and Riyanto also speculate that this is a particular feature of Thai culture.
But what gives pause for thought is the obvious uselessness of the tips. A horse-racing tipster will boast of insider knowledge and hint that racing results are pre-arranged for the convenience of the cognoscenti. Nobody believes that there is much “insider knowledge” about the next toss of the coin.
Of course, the cultivated readers of the FT would not make the same crass errors as the young students did. But, just in case, next time you see an investment manager touting impressive returns on a couple of funds, ask yourself how many other funds the company manages.
Also published at ft.com.