Tim Harford The Undercover Economist

Articles published in June, 2018

A simple way for computers to improve our economic forecasts

I am not one of those clever people who claims to have seen the 2008 financial crisis coming, but by this time 10 years ago I could see that the fallout was going to be bad. Banking crises are always damaging, and this was a big one. The depth of the recession and the long-lasting hit to productivity came as no surprise to me. I knew it would happen.

Or did I? This is the story I tell myself, but if I am honest I do not really know. I did not keep a diary, and so must rely on my memory — which, it turns out, is not a reliable servant. In 1972, the psychologists Baruch Fischhoff and Ruth Beyth conducted a survey in which they asked for predictions about Richard Nixon’s imminent presidential visit to China and Russia. How likely was it that Nixon and Mao Zedong would meet? What were the chances that the US would grant diplomatic recognition to China?

Professors Fischhoff and Beyth wanted to know how people would later remember their forecasts. Since their subjects had taken the unusual step of writing down a specific probability for each of 15 outcomes, one might have hoped for accuracy. But no — the subjects flattered themselves hopelessly. The Fischhoff-Beyth paper was titled, “I knew it would happen”.

This is a reminder of what a difficult task we face when we try to make big-picture macroeconomic and geopolitical forecasts. To start with, the world is a complicated place, which makes predictions challenging. For many of the subjects that interest us, there is a substantial delay between the forecast and the outcome, and this delayed feedback makes it harder to learn from our successes and failures. Even worse, as Profs Fischhoff and Beyth discovered, we systematically misremember what we once believed. Small wonder that forecasters turn to computers for help.

We have also known for a long time — since work in the 1950s by the late psychologist Paul Meehl — that simple statistical rules often outperform expert intuition. Meehl’s initial work focused on clinical cases — for example, faced with a patient suffering chest pains, could a two or three-point checklist beat the judgment of an expert doctor? The experts did not fare well.

However, Meehl’s rules, like more modern machine learning systems, require data to work. It is all very well for Amazon to forecast what impact a price drop may have on the demand for a book — and some of the most successful hedge funds use algorithmically-driven strategies — but trying to forecast the chance of Italy leaving the eurozone, or Donald Trump’s impeachment, is not as simple. Faced with an unprecedented situation, machines are no better than we are. And they may be worse.

Much of what we know about forecasting in a complex world, we know from the research of the psychologist Philip Tetlock. In the 1980s, Prof Tetlock began to build on the Fischhoff-Beyth research by soliciting specific and often long-term forecasts from a wide variety of forecasters — initially hundreds. The early results, described in Prof Tetlock’s book Expert Political Judgement (UK) (US), were not encouraging. Yet his idea of evaluating large numbers of forecasters over an extended period of time has blossomed, and some successful forecasters have emerged – as described in Tetlock’s later book, Superforecasting (UK) (US).

The latest step in this research is a “Hybrid Forecasting Tournament”, sponsored by the US Intelligence Advanced Research Projects Activity, designed to explore ways in which humans and machine learning systems can co-operate to produce better forecasts. We await the results.

If the computers do produce some insight, it may be because they can tap into data that we could hardly have imagined using before. Satellite imaging can now track the growth of crops or the stockpiling of commodities such as oil. Computers can guess at human sentiment by analysing web searches for terms such as “job seekers allowance”, mentions of “recession” in news stories, and positive emotions in tweets.

And there are stranger correlations, too. A study by economists Kasey Buckles, Daniel Hungerman and Steven Lugauer showed that a few quarters before an economic downturn in the US, the rate of conceptions also falls. Conceptions themselves may be deducible by computers tracking sales of pregnancy tests and folic acid.

Back in 1991, a psychologist named Harold Zullow published research suggesting that the emotional content of songs in the Billboard Hot 100 chart could predict recessions. Hits containing “pessimistic rumination” (“I heard it through the grapevine / Not much longer would you be mine”) tended to predict an economic downturn. His successor is a young economist named Hisam Sabouni, who reckons that a computer-aided analysis of Spotify streaming gives him an edge in forecasting stock market movements and consumer sentiment.

Will any of this prove useful for forecasting significant economic and political events? Perhaps. But for now, here is an easy way to use a computer to help you forecast: open up a spreadsheet, note down what you believe today, and regularly revisit and reflect. The simplest forecasting tip of all is to keep score.

 

 
Written for and first published in the Financial Times on 1 June 2018.

My book “Messy: How To Be Creative and Resilient in a Tidy-Minded World” is now available in paperback both in the US and the UK – or through your local bookshop.

Free email updates

(You can unsubscribe at any time)

Six unconventional introductions to economics

My list of five of the best introductions to economics wasn’t exactly the usual suspects, but I wanted to stray a little further off the obvious territory and recommend six books you might want to read to give you an unusual introduction to economics.

A couple of years after the financial crisis I came across Charles Perrow’s Normal Accidents (UK) (US). Perrow is a sociologist who became fascinated by particular kinds of system, ones which were “complex” (meaning that consequences of error are unpredictable) and “tightly coupled” (meaning that the consequences unfold quickly and irreversibly). His case studies include terrible accidents such as the Challenger disaster and Chernobyl – hauntingly described – but I increasingly came to realise that economic and financial systems could and should be studied with the same eye.  (For the same reason, I’d also recommend anything by James Reason. (UK) (US).)

Yoram Bauman and Grady Klein’s Cartoon Introduction To Economics (UK) (US) is perfectly conventional in many ways – except that it’s a cartoon, and also pretty funny, as you might expect from Bauman, a stand-up comedian. Good stuff.

Cory Doctorow’s For The Win (UK) (US) made me question whether I shouldn’t be trying to write about economics through fiction. My conclusion so far has been “no”, partly because Doctorow already does it so well. For The Win describes a a struggle between the young protagonists who work inside multiplayer computer games, and Big Business trying to run a cartel. Learn about globalisation, unionisation, virtual gold mining, and enjoy the thrill of the chase too.

James Owen Weatherall’s Physics of Wall Street (UK) (US) is a fine tour of how physicists and mathematicians from Bachelier to Mandelbrot to Jim Simons have tried to understand how markets work – and profit from their understanding. In the dock: economists, for not getting it. Harsh, I feel – but a very interesting and readable book anyway.

If you want to read an account with novelistic qualities that’s a true story, pick up Michael Lewis’s The Big Short (UK) (US). You’ve probably read it anyway, but read it again. He’s a superb writer and he really understands Wall Street.

Finally, I found Ben Goldacre’s Bad Science (UK) (US) simply revolutionised the way I thought about numbers, evidence, and the newspapers. Like Lewis’s book this will be familiar to many of you, but it bears reading again and again.

And one more suggestion; my freewheeling history of technology tells the story of particular inventions or ideas, and uses each one to teach us a lesson about how the economy works. In the US it’s Fifty Inventions that Shaped the Modern Economy and in the UK, Fifty Things That Made the Modern EconomyEnjoy!

Free email updates

(You can unsubscribe at any time)

25th of June, 2018MarginaliaResourcesComments off

The best way to solve problems is to wait for a century or two

In 1928, an anonymous donor resolved to clear the UK’s national debt and gave £500,000 with that end in mind. It was a tidy sum — almost £30m at today’s prices — but not nearly enough to pay off the debt. So it sat in trust, accumulating interest, for nearly a century.

The trust now contains £400m, and we have decided we are no longer willing to wait. The British government has gone to court to get the money now, a move that eloquently captures the payday-loan mood it is displaying in its Brexit negotiations. No gain is too small, no price too great, as long as the bill comes later.

What might we achieve if only we were willing to play the truly long game?

That anonymous trust fund suggests an instructive thought experiment. Let us assume that it grows 3 per cent a year faster than the UK economy — not inconsistent with what Thomas Piketty has measured in the long run. At that rate, the trust fund will double as a proportion of gross domestic product every 25 years. In just three centuries, it will have grown 4,000-fold relative to the economy as a whole.

As long as the debt stays roughly in proportion to national income — not an outrageous assumption — then the trust fund would be sufficient to pay off the debt a mere four centuries after the original bequest.

Perhaps that is too optimistic. No matter. If four centuries are not enough, why not five? It is surprising how many problems will simply solve themselves if we wait long enough.

This analysis is glib, I admit. Over such a long time horizon there is always a risk that bad luck strikes and the trust fund is wiped out entirely. If the fund falls to zero at any point, all the compound interest in the world is useless after that.

A wise investor may be able to avoid such an outcome: in 1956, John L Kelly, a mathematician at Bell Labs, derived a formula we now know as the Kelly criterion. It was designed to allow an investor or gambler with a known edge to maximise her compound rate of return, while avoiding the risk of bankruptcy.

Yet even Kelly’s criterion only works if the risks are correctly understood. Kelly himself survived a plane crash as a Navy pilot, only to die of a brain haemorrhage at the age of 41. The world is full of risks. Can anyone guarantee that over the next 300 years both the UK trust fund and country will survive asteroid strikes, thermonuclear war or a deliberately engineered pandemic?

Perhaps we are getting ahead of ourselves. The imminent threat to the trust fund is the British government itself, which has decided that a tiny advantage is worth seizing now, since the costs will fall to someone else. (You may supply your own analogy at this point.) All democratically elected governments struggle to see past the next election, but this one struggles to see past next Tuesday. In fairness, it often feels as if the next election may come sooner than that.

And it is hard to take a truly long-term perspective, whether contemplating the future of human life or the prospect of cheesecake. The Astronomer Royal Sir Martin Rees wrote a book titled Our Final Century, warning of the existential threats arising from complex, interconnected modern systems. The book was renamed Our Final Hour in the US, perhaps because a century seemed like too much time to kill.

Economists and moral philosophers argue among themselves over how to account for the interests of future generations. The answer is far from obvious. It turns out to be crucial in pondering a rational response to slow-burning disasters such as climate change — assuming that anyone cares about a rational response, which seems a forlorn hope.

The Chinese are sometimes admired as fabulously long-term thinkers, although sometimes I wonder whether that is merely the mythologising of westerners. (Zhou Enlai impressed many in 1972 when as Chinese premier he said that it was “too early to tell” about the consequences of the French Revolution. He was under the impression that the question was about the student uprising in Paris in 1968.)

No, those who genuinely show patience are rare. There is Warren Buffett, of course — his favourite holding period, “forever”, has served him well. And the Long Now Foundation, based in San Francisco and founded in the year “01996”, which supports ideas such as a modern Rosetta stone designed to preserve languages through time and ­catastrophe.

I am pleased that a few souls are willing to take the long view. Perhaps the champion is Anders Sandberg, a researcher at Oxford university’s Future of Humanity Institute. Dr Sandberg points out that since computation requires far less energy at ultra-cold temperatures, an advanced civilisation could get much more done with the resources available if it first waited a few trillion years for the entire universe to approach absolute zero.

This resolves the famous Fermi paradox: since the universe is so big, why haven’t aliens appeared from somewhere? The answer: they’re quietly having a trillion-year siesta, waiting for the cool of the twilight of the cosmos.

Written for and first published in the Financial Times on 25 May 2018.

My book “Messy: How To Be Creative and Resilient in a Tidy-Minded World” is now available in paperback both in the US and the UK – or through your local bookshop.

Free email updates

(You can unsubscribe at any time)

Bread, Brexit, and the power of the third option

Written for and first published in the Financial Times on 18 May 2018.

Imagine that you sell bread-making machines. Your task is complicated by the fact that most people have only a hazy grasp of what a bread-making machine does, let alone the joys and sorrows of owning one.

Nevertheless, there is a simple trick that will help these machines to fly off your shelves: next to what seems to be a perfectly adequate $150 bread-maker, place a $250 bread-maker with a long list of bewildering extra functions. Customers will think to themselves: “I don’t need all that nonsense. The cheaper, simpler bread maker is the better option.” Some of them will buy it, even though they would not have otherwise.

Itamar Simonson, a marketing professor at Stanford University, attests that the kitchenware company Williams-Sonoma doubled their sales of bread-makers in the early 1990s using this sort of technique. Mr Simonson, along with Amos Tversky, one of the fathers of behavioural economics, demonstrated similar preference reversals in a laboratory setting.

Mr Simonson and Tversky showed that when people are wavering between two options, you can change what they choose by offering a third, unattractive option. A $1,000 camera might seem extravagant unless there’s a $5,000 camera sitting next to it. The grande sized cup at Starbucks seems restrained when put next to the venti, a Brobdingnagian vat of flavoured warm milk.

All this brings us to Brexit. What we voters feel about different flavours of Brexit (hard, soft, train-crash) depends in part on facts, in part on propaganda, and in part on our prejudices. But it also depends on the comparisons that come readily to mind.

That means that the re-appearance of the European Economic Area is an intriguing development in the debate. The House of Lords recently voted to keep the UK in the EEA, and therefore the single market, after leaving the EU. This “Norway option” seems a popular enough plan: a BMG opinion poll in January found 52 per cent of people in favour of staying in the single market, and only 14 per cent of people against. In these polarised times that is as decisive a margin as one might expect for anything. Nevertheless, both prime minister Theresa May and the leader of the opposition, Jeremy Corbyn, have rejected the single market option, making it unlikely.

This might seem illogical. Why not go for a popular compromise that respects both the close vote and the fact that the Leave campaign won the referendum? But, remembering the tale of the bread-maker, it makes perfect sense that Mr Corbyn and Mrs May, both Euro-sceptics, should fear the Norway option being placed in front of voters.

To most voters, the EU is like a bread-maker: we don’t really understand what it does and we don’t know what to think about it. The Norway option clarifies matters in a way that does not help Leavers. It is very much like being in the EU, except just a little bit worse. If it becomes a salient possibility, it makes staying in the EU look rather attractive by comparison.

A hard Brexit will probably go quite badly for the UK, but it does have the merit of being a very different path to remaining in the EU. A Norway-option Brexit might well work out smoothly, but it is almost guaranteed to underperform the option of not leaving at all. No wonder Brexiters — so cavalier about having their cake and eating it before the vote — are now determined to ensure that the Norway option is taboo. They realise that if the British public decides that staying in the single market is a plausible plan, they might eventually reach the conclusion that staying in the EU itself would be even wiser.

This sort of preference reversal can occur in other circumstances, too. A hard Brexit offers temptations to many voters: control over immigration; an independent trade policy; no more membership fees to Brussels. It also offers obvious risks: leaving the largest single market in the world; damage to the political settlement in Northern Ireland; setbacks to scientific and diplomatic collaboration. Staying in the EU merely offers business as usual.

Do we tend to find a mix of stark risks and clear rewards appealing? That depends on whether the costs or the opportunities seem more salient. During the referendum campaign, the opportunities opened by Leavers seemed expansive, while the costs (“lower GDP by 2030!”) were vague and dull. During the negotiation process, it is the opportunities that are starting to seem vague while the costs are becoming vivid, at least to the small number of people who are paying attention.

None of this makes it likely that Brexit will be reversed. The simple fact that Leave won the referendum is likely to be proof against all sorts of psychological subtleties. Yet these seem to be nerve-racking times for the Brexiters.

It was always clear that asking an absurdly simple question about an absurdly complicated decision was unlikely to work out well. There is one ironic consolation: however befuddled our referendum decision might have been, the divided cabinet is now doing its best to make us, the great British public, seem like philosopher kings by comparison.

My book “Messy: How To Be Creative and Resilient in a Tidy-Minded World” is now available in paperback both in the US and the UK – or through your local bookshop.

Free email updates

(You can unsubscribe at any time)

Why it’s important to gather the evidence – but easy to forget it even then

Written for and first published in the Financial Times on 11 May 2018.

It is hard to know which is more frustrating: last week’s announcement that over the past nine years, 450,000 British women were accidentally not invited for breast cancer screening; or the widespread indifference of a howling media to the evidence that such screening is of doubtful benefit anyway. Mammograms lengthen the lives of some women and shorten the lives of others: they allow the early detection and treatment of dangerous tumours, but they also produce many false positives, leading to the unnecessary and risky treatment of tumours that would never have caused any problems.

The best evidence we have, after weighing up several high-quality clinical trials, is that the harms and the benefits are finely balanced. When UK women are offered breast screening, they are sent a leaflet explaining the advantages and the risks so that they can make an informed choice. That choice should not have been denied to them by an administrative blunder. Still, we should be grateful that the error did not disrupt cervical cancer screening instead, which has convincingly been shown to save lives.

We should draw two lessons from the affair, beyond the obvious, which is that British institutions need to get a grip. The first lesson is that it pays to collect the best evidence that we can. The second is that having the best evidence is seldom enough.

Still, the evidence is a start. The world is full of sensible-seeming ideas that disappoint — along with some odd-seeming ideas that turn out to work. Among the latter is the idea that lemon juice prevents and cures scurvy, a disease so debilitating that ships could lose half their crews. In 1747, James Lind, a Scottish doctor, conducted one of the most celebrated early clinical trials proving the efficacy of lemon juice. This is not what common sense might have suggested. The mechanism was obscure: a chemical in lemons — later dubbed “vitamin C” — makes the difference between life and death in tiny doses.

Randomised trials have finally become entrenched in medicine as the obvious way to assess what works — as, just as importantly, have reviews that systematically assemble, evaluate and summarise all the available trials in one place. This did not happen easily, since few senior doctors enjoy being embarrassed by an unexpected trial result.

Such trials have also become an important way to assess ideas in education, criminal justice and economic development. Their use is far more patchy and more controversial but is still yielding dividends.

A new book, Randomistas (UK) (US), by Andrew Leigh, an Australian economist turned politician, gives plenty of examples. One — notorious in geek circles — is “Scared Straight”, a programme designed to deter juvenile offenders by taking them to prison to be bullied in short bursts by terrifying inmates. Scared Straight was so fashionable in the late 1970s that a documentary film about the policy won an Oscar; randomised trials showed it to be counterproductive.

That is often the way. Three decades ago the sociologist Peter Rossi quipped that the more rigorously a social programme evaluation was designed, the more likely it was to show a net benefit of zero. Unfortunately, Mr Rossi may well have been right — but showing which ideas do not work is one of the most important roles of high quality trials.

And not every idea fails. While the evaluations of Scared Straight showed that it made matters worse, a randomised trial of police protocols for domestic violence in Minneapolis in 1981 demonstrated that the police needed to be tougher on domestic abusers, arresting them rather than having a quiet word, if they wanted to prevent future assaults.

Randomised trials of cash transfers to entrepreneurs in developing countries have shown some excellent results, including a spectacular trial in which some Nigerian entrepreneurs with high-quality business plans were randomly chosen to receive $50,000 to realise their ideas.

This research is useless, however, if the people making the decisions are not aware of it. The academic’s cliché, “more research is needed”, is not necessarily true. Often all the necessary research has been done, but it has not been assembled and systematically reviewed. Or — as in the case of breast screening — it has been systematically reviewed, but not enough people have noticed.

Lind’s trial of lemon juice is instructive here. As early as 1601, James Lancaster of the East India Company had demonstrated in an informal trial that lemon juice was proof against scurvy. It took two centuries for the Royal Navy to make it part of sailors’ rations — and longer still for other navies to catch on. Yet as voyages grew shorter, and still lacking a convincing theory for why lemon juice vanquished scurvy, we simply forgot. In 1911, 300 years after Lancaster’s demonstration, Robert Scott’s expedition to the South Pole — including a Royal Navy surgeon — did not know how to prevent scurvy. They suffered grievously as a result. Knowledge can be gained; it can also be ignored, or forgotten.

 

My book “Messy: How To Be Creative and Resilient in a Tidy-Minded World” is now available in paperback both in the US and the UK – or through your local bookshop.

Free email updates

(You can unsubscribe at any time)

The best economics podcasts in 2018

My favourites are:

  1. NPR’s Planet Money remains an outstanding show, with stories, humour, very clear explanations and high production values. But – horrors! – they’ve poached the amazing Cardiff Garcia from the FT, and Cardiff is co-presenting (with Stacey Vanek Smith)…
  2. NPR’s The Indicator which is basically a shorter, chattier version of the same thing. Available daily. Works very well.
  3. A new entry is Tyler Cowen’s Conversations with Tyler. This is so wide-ranging that it barely qualifies as an economics podcast, but it’s a joy to listen to. Tyler’s questioning style is unique and he has a remarkable range of people on the podcast – Martina Navratilova, Charles Mann, Garry Kasparov, Agnes Callard, Matt Levine…
  4. Freakonomics Radio remains a favourite. Stephen Dubner asks questions that others don’t think to ask, slips between serious and silly topics and generally gets a top-notch line-up of interviewees.
  5. Slate Money, presented by Felix Salmon, who is great but still interrupts his (changing crew of) co-hosts a little too too much. Always very smart and sometimes very well informed too.
  6. Behavioural economics enthusiasts should try The Hidden Brain with Shankar Vedantam. Guests have included Daniel Pink, Phil Tetlock, Alison Gopnik, Richard Thaler, Daniel Kahneman… even me.
  7. Try the Trade Talks podcast with Chad Bown and Soumaya Keynes for a nerdy (but witty) dive into the details of how trade negotiations and agreements work. Ordinarily I would suggest that this might be a little too geeky, but this is a fast-moving subject at the moment, and Bown and Keynes have a light touch, too.
  8. Russ Robert’s EconTalk offers long, searching conversations between Russ and a wide variety of guests, often with interesting books or essays to discuss. The sound quality can be patchy and the tone of the interviews varies a lot depending on whether the subject is the evidence for education, or the importance of meditation. But it’s a very good source of smart ideas.

 

 

Excellent not-quite-economics podcasts include:

  1. Revisionist History with Malcolm Gladwell
  2. Start Up
  3. Reply All
  4. Radiolab
  5. TED Radio Hour
  6. 99% Invisible
  7. Akimbo
  8. WorkLife
  9. The Digital Human
  10. Stephen Fry’s Great Leap Years

 

 

My employers at the FT have some very fine podcasts at the moment. I particularly recommend:

  1. FT Banking Weekly
  2. FT Money
  3. FT Brexit Unspun

 

 

Then there are MY PODCASTS:

  1.  More or Less, a weekly guide to the numbers that surround us.
  2. Pop Up Economics, mostly by me but also featuring guests including Gillian Tett and Malcolm Gladwell. 13 episodes, currently dormant but enjoy the archive.
  3. 50 Things That Made the Modern Economy – although there are actually 52 episodes. Series 1 is complete, but subscribe and watch this space.

 

That should be plenty to be going on with.

And if you want to make your own podcast, have a listen to some of the recommendations above and grab yourself a microphone (UK) (US). “Out On The Wire” (UK) (US) is a superb guide to how some of the top shows are made.

Previously: Best economics podcasts 2016, Best economics podcasts 2011.

 
My book “Messy: How To Be Creative and Resilient in a Tidy-Minded World” is now available in paperback both in the US and the UK – or through your local bookshop.

Free email updates

(You can unsubscribe at any time)

5th of June, 2018MarginaliaRadioResourcesComments off

Cheap innovations are often better than magical ones

Written for and first published in the Financial Times on 4 May 2018

If you type “technology indis…” into Google, you are instantly directed to a webpage discussing Arthur C Clarke’s third law: “any sufficiently advanced technology is indistinguishable from magic”. The science fiction writer’s aphorism was published in 1962, at a time when a demonstration of Google’s autocompleting search engine would indeed have seemed like sorcery.

There are plenty of other examples: electricity, the aeroplane and the telephone would all have seemed miraculous and inexplicable to earlier generations. Each of them exemplified what a technological breakthrough is supposed to look like, deservedly winning attention as they appeared.

We need to be careful, however, not to overlook much simpler technological advances. The lightbulb is a safer and more controllable source of artificial light than the candle or the oil lamp, but what really makes it transformative is its price — the cost of illumination has fallen approximately 400-fold in the past two centuries.

Supercomputers and space travel get all the press. Merely being cheap doesn’t. But being cheap can change the world. Consider barbed wire (cheap fencing), the shipping container (cheap logistics), or the digital spreadsheet (cheap arithmetic). Ikea gave us cheap furniture, and the same principles of simple modular assembly are giving us cheaper solar panels, too.

My favourite example is paper: the Gutenberg press radically reduced the cost of producing writing, but it was of little use without an accompanying fall in the cost of a writing surface. Compared to papyrus, parchment or silk, one of paper’s most important properties was that it cost very little.

With all this in mind, what are today’s technological advances that we may be overlooking or misunderstanding because they are cheap rather than magical? The obvious answer: sensors. We are surrounded by inexpensive sensors — in our phones, increasingly in our cars — continually taking in information about the world.

A new book suggests a different, albeit related, answer. Prediction Machines (UK) (US) by Ajay Agrawal, Joshua Gans and Avi Goldfarb argues that we’re starting to enjoy the benefits of a new, low-cost service: predictions. Much of what we call “artificial intelligence”, say the authors, is best understood as a dirt-cheap prediction.

Predictions are everywhere. Google predicts that when I type “technology indis…” I am looking for information about Clarke’s third law; Amazon makes a prediction about what I might buy next, given what I have bought already, or searched for, or placed on my wishlist. A prediction may literally be a forecast about the future, or more generally it may be an attempt to fill in some blanks on the basis of limited information.

Not all such predictions are very good, but not all of them need to be. The tiny keyboards on our smartphones turn out to be quite serviceable when combined with modestly accurate predictions — from suggesting an entire one-phrase email reply (“I agree with you”) to subtly expanding the “H” and shrinking the surrounding keys on a touchscreen if the phone thinks that “H” is the more likely target for a fat-thumbed typist.

Errors in predictive text tend to be trivial and easy to correct, so a high error rate does not matter much. Clumsy text predictors can be released into the world so that they may learn. A high error rate in a self-driving car is not so easy to forgive.

As Mr Agrawal and colleagues point out, sufficiently accurate predictions allow radically different business models. If a supermarket becomes good enough at predicting what I want to buy — perhaps conspiring with my fridge — then it can start shipping things to me without my asking, taking the bet that I will be pleased to see most of them when they arrive.

Since good predictions reduce uncertainty, we may also see less demand for things that help us deal with uncertainty. If that conspiratorial fridge can arrange just-in-time delivery of meal ingredients by predicting my requirements, it can be much smaller as a result.

Another example is the airport lounge, a place designed to help busy people deal with the fact that in an uncertain world it is sensible to set off early for the airport. Route-planners, flight-trackers and other cheap prediction algorithms may allow many more people to trim their margin for error, arriving at the last moment and skipping the lounge.

Then there is health insurance; if a computer becomes able to predict with high accuracy whether you will or will not get cancer, then it is not clear that there is enough uncertainty left to insure.

All this seems a useful way to look at the fast-changing world of machine learning — more useful than pondering Clarke’s most famous creation, the murderous computer HAL 9000. Some automated predictions are already marvellously good, but many are changing the world not because they are omniscient, but because they’re good enough — and cheap.

 

More on all this in my recent book, “Fifty Things That Made The Modern Economy” – now out! Grab yourself a copy in the US (slightly different title) or in the UK or through your local bookshop.

Free email updates

(You can unsubscribe at any time)

Elsewhere

  • 1 Twitter
  • 2 Flickr
  • 3 RSS
  • 4 YouTube
  • 5 Podcasts
  • 6 Facebook

Books

  • Fifty Inventions That Shaped the Modern Economy
  • Messy
  • The Undercover Economist Strikes Back
  • Adapt
  • Dear Undercover Economist
  • The Logic of Life
  • The Undercover Economist

Search by Keyword

Tim’s Tweets

Free Email Updates

Enter your email address to receive notifications of new articles by email (you can unsubscribe at any time).

Join 169,207 other subscribers

Do NOT follow this link or you will be banned from the site!