Tim Harford The Undercover Economist

Articles published in April, 2011

A spoonful of medicine…

Should economics be more like medicine? I don’t mean that economists should be more like doctors – I’ve met a few doctors – but that economists should learn from the relationship that medical practice has with medical evidence.

Medicine, like economics, deals with complex systems that are still not well understood; like economics, it has its share of quacks; but unlike economics, medicine has swallowed many of its ethical qualms about running controlled experiments in difficult circumstances.

Randomised controlled trials are now catching on in economics, especially development economics. (Such a trial would involve, for instance, approving loans to a randomly chosen subset of loan applicants.) Two new books, Poor Economics and More Than Good Intentions, each by leading practitioners of randomised trials, explain the consequent discoveries.

Clearly such trials have their limits, but I’m a big fan of the approach. However, it would be a great shame if economists learned nothing more from doctors than to use randomisation.

One lesson that has emerged all too slowly from medical practice is the need for trial registries, in which researchers give notice that a clinical trial is about to begin, noting exactly what they will do.

Trial registries sound like a pernickety piece of bureaucracy. In fact, they could hardly be more important. When analysing any statistical finding, researchers must allow that sometimes remarkable patterns emerge by chance. Imagine that there are 20 researchers, each investigating whether mint humbugs cure cancer. Purely by happenstance, we’d expect one of the researchers to find evidence that they do. She’ll approach a medical journal and get her fascinating results published. The other 19 researchers may not bother at all – or, realising that their research is destined to be published in The Journal of Uninteresting Results, they will drag their heels.

In short, published research is systematically biased in favour of striking results that may be coincidence. The trial registry matters because later researchers into the anti-carcinogenic properties of humbugs can take all the non-results into account.

Dean Karlan, a Yale economist, co-author of More Than Good Intentions, and founder of Innovations for Poverty Action, which co-ordinates and evaluates development projects in poor countries, argues that trial registries are harder to design in social science than in medicine. Researchers cannot control a project as tightly as clinicians can – they may find that the project they are evaluating is changed halfway through.

High-quality empirical research is not just a matter of using tools such as randomised trials and trial registries – it’s about the entire research culture. A simple example: if academic careers are in thrall to the number of articles published in the top journals, and if the top journals are not interested in publishing boring-sounding replications of earlier research, then these replications will not be attempted. Yet replication is a foundation of experimental science.

Professor Jonathan Shepherd, a clinician at Cardiff University, points out that the culture of evidence permeates medicine. Doctors are trained in university hospitals; their professors are themselves practising doctors, and their research agenda is driven by their needs as medical practitioners.

Meanwhile their pupils, thoroughly indoctrinated as to the value of medical evidence, read about new research in the British Medical Journal every week. In short, in medicine, academic evidence and everyday practice are intertwined. No doubt this symbiotic relationship is less than perfect in the real world; nevertheless it is something economists would do well to emulate.

Also published at ft.com.

Keynes vs Hayek Round 2

I know a lot of you have been waiting for this – the “Fight of the Century” between Keynes and Hayek, and congratulations to John Papola and Russ Roberts for transforming the way economics is communicated:

Personally I prefer the first video, “Fear the Boom and Bust”:

Funnier, and the Hayekian sympathies of its creators are conveyed with a bit more subtlety. Nevertheless the world of economics communication is a better place with these rapping titans in it.

Your thoughts?

Why we’re all far too sure of ourselves

At least since President Truman asked for a “one-handed economist” – who presumably would be unable to say, “on the other hand” – politicians have demanded the appearance of certainty where certainty cannot exist.

Economists and other academics tend to respond to this demand if they want to be heard in the corridors of power. They do so in a wide variety of ways: at a recent Leverhulme lecture at the Institute for Fiscal Studies in London, the economic statistician Charles Manski laid out a typology of unreasonable “certitudes”.

A memorable example is the “conventional certitude”, in which a spuriously precise number becomes the focus for all debate. In the US, the Congressional Budget Office estimates how much each piece of putative legislation will alter the budget deficit over the following decade. CBO estimates are about a hundred times too precise: they are reported to the nearest billion dollars, when a range of several hundred billion would be more reasonable for major legislation.

But they are then adopted as gospel by almost all politicians, analysts and media outlets. In the UK, the Treasury’s – now the Office for Budget Responsibility’s – forecasts are treated with a little more scepticism, but only a little. The central forecast of the budget deficit tends to be universally accepted. It just seems simpler that way.

Being open about uncertainty is not just a case of reporting some kind of statistically-derived “margin of error”. There are many ways for a conclusion to look statistically robust but be wrong. What is needed is to be clear about the underpinning assumptions and open-minded about what would happen if the assumptions were mistaken.

It is not clear why we enjoy certitude so much – certitude being the subjective experience of feeling certain. In contrast – as Kathryn Schulz observes in her wonderful book Being Wrong – there is simply no psychological experience of “being wrong” at all, only the lurching realisation of having been wrong until a moment ago.

Manski argues that analysts should be far more open about the extent of their doubts, and that politicians can and should be able to cope. I am more pessimistic. Politicians are creatures of certitude: they join a tribe of like-minded people, convinced that the tribe on the other side is wicked and stupid. The media love certitude, too. Newspaper editors hate headlines with “may” or “might” in them.

For these reasons, the scholar who is honest about her doubts will find her work ignored in favour of some clever-sounding chap who just seems to know so much more about how the world works. (How else could he be so certain?) Brilliant scholars with strong, clear views, such as Milton Friedman, John Maynard Keynes and Paul Krugman, enjoy larger followings than brilliant scholars who deal in doubts and complications, such as Elinor Ostrom and Thomas Schelling.

Manski might seem quixotic in his request that serious policy analysis be presented with more humility, given that neither politicians nor the media have much appetite even for overly-certain serious policy analysis.

But there is a serious cost to excess certainty. Whenever an analyst or academic presents a number or a conclusion with too much precision, they reduce the demand for better evidence. Why run a pilot, set up a proper survey, if the answer is already known to three decimal places?

The fact is that our political system simply does not take evidence seriously. If I had to suggest one single reason for that, it’s our love of certitude. Evidence is the way to reduce honest doubts. Stuffed on a fattening diet of certitude, who has room for doubt? And if we have no doubts, who needs evidence?

Also published at ft.com.

Wed with a bang not a whimper

We can’t all have a wedding that costs tens of millions – or even £5bn, the amount Prince William and Kate Middleton’s nuptials will reportedly (and implausibly) knock off Britain’s struggling economy. But even if most of us are untroubled by bills for carriage rides from the palace and the hiring of Westminster Abbey, the rising cost of getting hitched appears to be a global issue.

Reports suggest that tying the knot is now a $30,000 affair in the US, with price tags also rising to £20,000 in the UK; not far short of income per capita in each country. Celebrity weddings are now so expensive they almost rival the cost of celebrity divorces. And nor is this just a rich-world phenomenon. Abhijit Banerjee and Esther Duflo, authors of Poor Economics, note that even households in poverty in north-west India typically spend 10 per cent of their income on weddings and other festivals.

What could be the return on such lavish spending? It becomes easier to comprehend when you realise that most of us do pay a good deal less. Headlines about the rocketing costs of solemn vows are often gathered from surveys of readers in fancy wedding magazines, exactly the kind of people who might arrange a fancy wedding. Indeed, if they weren’t tempted by a blow-out for the big day before they subscribed, they soon will be.

But the mooted £20,000 tag is not even typical for a flashy wedding-magazine-reader’s wedding. Instead it is the average, where a few major extravagances inflate the figure well beyond what most couples would recognise. Just 10 £100,000 weddings in a survey of 1,000 people notch up average costs by £1,000.

If Will and Kate’s wedding did cost £5bn (which it will not), it would raise the average for everyone else by over £30,000. But the cost of the median wedding – that which is more expensive than half of all weddings, but less expensive than the other half – is certain to be much lower. (I am grateful to colleagues on BBC Radio 4’s More or Less for pointing this out.)

Fine. Let’s say the true cost of a typical British wedding is a generously proportioned £10,000. Carve off enough for a honeymoon, a couple of thousand more for the basics – a dress, a suit, a couple of rings, and a venue for the ceremony – and suddenly you’re looking at spending an average of 50 quid a head on food, wine, and entertainment for a hundred guests. This is hardly absurd.

Indeed, it may be a bargain. Andrew Oswald, a professor at Warwick University and a specialist in the economics of happiness, thinks that a happy marriage brings as much life satisfaction as an extra £70,000 a year. Even without this, get your gift registry right and you could even make a profit. And if you don’t, you’ll still be paid back in the long run. Zsa Zsa Gabor aside, most of us can expect to be guests at a wedding rather more often than we are hosts.

The writer is married. He regards his wife as cheap at twice the price.

Also published at ft.com.

More or Less

Tomorrow’s “More or Less” presents the results of our alternative to the census, asks “What is GDP?” – and should it be replaced? – and meets the man Michael Gove says is the most important figure in British education. Tune in on Friday at 1.30pm BST on Radio 4 – or Sunday 8pm – or subscribe to our podcast.

21st of April, 2011RadioComments off

Don’t blame the (mostly) efficient markets hypothesis

I’m going to defend the poor old efficient markets hypothesis. Somebody has to. It’s been getting quite a pounding since the credit crunch began. David Wighton of The Times commented in January 2009, “The theory was officially declared dead at the World Economic Forum in Davos. There were no mourners.” In June of that year, Roger Lowenstein wrote in The Washington Post, “The upside of the current Great Recession is that it could drive a stake through the heart of the academic nostrum known as the efficient-market hypothesis.” More recently, Matthew Bishop and Michael Green, authors of The Road from Ruin, have argued that the EMH was partly responsible for the crisis.

It’s probably worth pausing for a moment to recall what the EMH actually means. It’s not a Reaganite claim about the superiority of free markets over government intervention; it’s a far narrower and more technical claim about the price of liquid assets such as shares or corporate bonds. It is, nevertheless, hugely important.

The EMH has several forms. The weakest says that not only is past performance no guarantee of future performance, but nothing about the way a share’s price has bounced around in the past tells you anything about how it will move in the future. The strongest says that the market price is the correct price: that all privately and publicly available information that might be relevant to the value of a share is already reflected in today’s price. The weak form tells you not to listen to stock pickers who point to recently soaring shares. The strong form tells you not to bother doing any research into shares, because it cannot possibly do you any good.

In its strong form, the EMH cannot always be true. (How would the market become so efficient, since no rational participants would bother with research?) Perhaps it is never true, although as my colleague John Kay has pointed out, the difference between the EMH usually being true and always being true may be difference enough to explain the likes of Warren Buffett.

But did the EMH lead to the crisis? Not directly, for sure. The first thing the EMH would tell you is to be suspicious of bond salesmen who claim that structured subprime vehicles can offer high rewards and almost no risk. I think it is telling that according to Michael Lewis’s book The Big Short, some savvy investors who wanted to bet against subprime mortgages hesitated to do so, for fear that they had missed a trick. They instinctively took the EMH seriously, and only bet heavily against subprime after they had met the subprime enthusiasts and concluded they really were as foolish in person as their strategies suggested. The EMH encourages scepticism, not gullibility, about sure-thing investments.

It is more defensible to suggest that the EMH worked wickedness indirectly, through the attitude of regulators. Matthew Bishop tells me that he sees three ways in which the EMH was responsible for the crisis. First, it seduced Alan Greenspan into believing either that bubbles never happened, or that if they did there was no hope that the Federal Reserve could spot them and intervene. Second, the EMH motivated “mark-to-market” accounting rules, which put banks in an impossible situation when prices for their assets evaporated. Third, the EMH encouraged the view that executives could not manipulate the share prices of their companies, so it was perfectly reasonable to use stock options for executive pay. These are cogent points. Regulators, then, should be wary of the EMH.

Yet I remain convinced that the efficient markets hypothesis should be a lodestar for ordinary investors. It suggests the following strategy: choose a range of shares or low-cost index trackers and invest in them gradually without trying to be too clever. If only a few more bankers had taken such advice seriously.

Also published at ft.com.

Penn and Teller on risk

Emphatically not safe for work, but something of a modern classic, recommended to me by one of the UK’s most eminent medical statisticians…

15th of April, 2011VideoComments off

Why banks are going to auction

In 1873, Walter Bagehot famously argued that in a banking crisis, the Bank of England should be willing to lend “freely and readily” in exchange for good collateral. This view seemed quaint five years ago, when international capital markets were willing to pay cash for the most illiquid-seeming assets, but it quickly became relevant during the credit crunch.

Yet it is a challenge to support banks that are temporarily embarrassed by a lack of liquidity, without bailing out bankrupt banks or relieving them of all responsibility for their own liquidity management.

Paul Klemperer – the Oxford University theorist behind the auctions for “3G” mobile phone licences 11 years ago, and less gloriously my thesis supervisor – has been working with the Bank of England to design an auction to solve the problem. The design exploits a long tradition in auctions: that of proxy bidding.

In a standard auction, a proxy bid of, say, £260 for a case of champagne means that the buyer is happy to outbid all-comers until the bidding reaches £260.

The Klemperer auction design allows bidders to submit multiple proxy bids for different kinds of object. Imagine an auction in which both vintage and non-vintage champagne is available, and several cases of each are on offer. A package of bids might include one of £250 for a case of non-vintage champagne, plus a “paired” bid of either £300 for vintage champagne or £200 for non-vintage, plus a third, bottom-feeding, bid of £150 for a case of vintage or £100 for non-vintage, useful if prices are low. Depending on how prices in the auction work out, this package of bids might win up to three cases of champagne, and the quality would vary depending on which kind of champagne was more in demand.

Although it may need a computer to figure out exactly who gets what, this basic idea is not too troubling, and the multiple proxy bids enable a bidder to specify, quite intuitively, how much booze they’d like to buy, and what sort of price-differential would tempt them to favour one type of champagne over another.

This is roughly what the Bank of England is now doing to supply liquidity to banks. The banks bid different interest rates to borrow money from the Bank, and offer either higher- or lower-quality collateral, depending on the interest rate premium required for the lower-quality stuff.

(It would be possible to run two separate auctions or combine the two auctions into one, with some fixed premium for low-quality collateral. But the first alternative dilutes the competitiveness of the auctions, while the second will malfunction if the Bank misjudges the level of the premium.)

The Bank has been running these auctions every month since last summer, and seems pleased with the innovation. The auctions may seem fiddly, but “all the complexity lies on our side”, says Chris Salmon, an executive director and chief cashier at the Bank of England. While they haven’t yet been tested under stress, they offer something quite attractive: a flexible tool, ticking away in the background, with the ability to prevent old-fashioned, self-fulfilling bank runs.

Klemperer, meanwhile, has his eye on wider horizons. His auction design could, in principle, be used to buy – or sell – dozens, even hundreds, of substitute products at the same time.

This is an idea with many uses. It could be used to run electricity networks by selling intermittent power and baseload power in the same auction, or to sell planning permission on green-field and brown-field sites – a prospect to gladden George Osborne’s heart. And in a future banking crisis, it could be used by a government wishing to establish a proper price for toxic assets. The Bank of England, I suspect, is hoping that particular use remains theoretical for many decades yet.

Also published at ft.com.

More or Less

Tomorrow’s More or Less will cover social mobility, ask whether unemployed full-time students should really count as being unemployed, and find out about one of the most distinctive mathematicians in the world, Grigori Perelman. Steve Ziliak fans will have to wait until next week; sorry.

Most importantly: we shall try to calculate the fiscal multiplier in Trumpton.

More or Less is on BBC Radio 4 at 1.30pm, tomorrow, repeated on Sunday at 8pm, and podcast (click on the link at the top right of the page.)

The cult of statistical significance

Steve Ziliak, haiku economist and co-author of The Cult of Statistical Significance, points me to this geeky but splendid video about a much-misunderstood subject:

Steve’s a guest on this week’s More or Less. Here’s an earlier piece about his work, although I am worried that I may have got some of the technical details wrong.



  • 1 Twitter
  • 3 RSS
  • 5 Podcasts
  • 6 Facebook


  • Fifty Inventions That Shaped the Modern Economy
  • Messy
  • The Undercover Economist Strikes Back
  • Adapt
  • Dear Undercover Economist
  • The Logic of Life
  • The Undercover Economist

Free Email Updates

Enter your email address to receive notifications of new articles by email (you can unsubscribe at any time).

Join 188,408 other subscribers.