Tim Harford The Undercover Economist

Articles published in June, 2017

Books about calling statistical bullshit

A friend recently emailed to ask me for books that might help navigate a world full of statistical bullshit. Here are some recommendations.

I can’t think of a better science writer than Ben Goldacre, who burns with righteous mischief. His Bad Science (UK) (US) isn’t always about statistics, but it’s excellent throughout and an essential read for anyone who wants to understand some of the faults of modern health and nutrition journalism. Wonderful book.

Of course you should subscribe to the More or Less podcast, but you could also enjoy The Tiger That Isn’t (UK) (US). This is the unofficial book of the series, written by More or Less founders Andrew Dilnot and Michael Blastland. A highly readable guide to making sense of numbers in the wild.

Also very good – with more US examples – is Stat-Spotting (UK) (US) by Joel Best. Best’s book has given me some of my favourite examples of bad stats, but it currently seems a bit overpriced on Amazon, alas.

The classic of the field is, of course, Darrell Huff’s How To Lie With Statistics (UK) (US). There’s a sad coda that will tarnish your view of Huff; but this is still a terrific book.

Brand new book by the very splendid Evan Davis is called Post Truth (UK) (US) – haven’t yet read much but looks good.

And finally try Naked Statistics (UK) (US) by Charles Wheelan, who with wit and clarity wrote the similarly excellent Naked Economics (UK) (US).

Best, Dilnot, Huff and Wheelan all cover quite similar ground. If I was picking just one of them I’d go for Dilnot for a UK audience and Wheelan in the US.

My new book is “Fifty Things That Made The Modern Economy” – coming very, very soon and available for pre-order. Grab yourself a copy in the US (slightly different title) or in the UK or through your local bookshop.

Free email updates

(You can unsubscribe at any time)

26th of June, 2017Other WritingComments off

Donald Trump and Theresa May Give U-Turns A Bad Name

A few weeks ago Donald Trump set a record that I assumed was unassailable; Theresa May has since sailed past it with ease. The leaders of the US and the UK have become so proficient at changing direction that “U-turn” no longer seems adequate. Donald and Theresa are spinning policy doughnuts.

Mr Trump has a notoriously flexible approach to his own pledges. He has reversed course on issues as diverse as whether he will put Hillary Clinton in jail (yes, then no), whether he would force a vote on healthcare reform (yes, then no) and whether it was wise to attack Syria (no, then yes).

His most dizzying day, in mid April, included U-turns on a bewildering array of substantial policy issues: Nato (“I said it was obsolete; it’s no longer obsolete”), whether China was a currency manipulator, a hiring freeze on federal workers, closing the US Export-Import bank and whether he might reappoint Janet Yellen as head of the Federal Reserve.

It was an impressive record — but surely a record no longer. Over the past year Mrs May has changed her mind on everything from Brexit to a bill of rights, energy pricing to nuclear power. She reversed a 2015 manifesto commitment, reversed the reversal, and has now taken the unprecedented step of tearing pages out of her own manifesto just days after launching it. She offers a “strong and stable” slogan, a weak and wobbly reality, and a rich seam of irony.

What most concerns me about all this is that Mr Trump and Mrs May are tarnishing the very idea of changing one’s mind. U-turns can be valuable, but they already have a poor reputation, reflecting a lazy journalistic trope. The U-turn is a gift to the journalist in a hurry: either the policy was wrong before, or it is wrong now. Little more need be said.

Of course there is something to be said for being consistent. Stubbornness in negotiations is risky but can be an advantage. Foreign policy requires that allies and enemies know where they stand. When politicians promise something to voters, they should make an effort to deliver.

But it is easy to take consistency too far. The most straightforward solution to many a grave policy error is to stop and do something else. This is something the Conservative party would be well advised to do with a damaging limit on immigration that it has promised three times and never come close to delivering.

Discarding what does not work is an essential part of progress in almost any sphere of life. Designers and engineers make prototypes. Programmers debug. Writers edit. Medical researchers use randomised trials to figure out whether a treatment works or is worthless. Evolution works through survival of the fittest. In each case, there’s a way to evaluate and discard what is failing.

Economic growth is built on trial and error, with good ideas spreading and bad ones disappearing. Agile businesses reinvent themselves, but often the market does the job for them through the bankruptcy courts.

Policymaking needs the same mechanism, and often lacks it. Because governments can levy taxes, dole out subsidies, and alter the rules of the game, they can do a great deal of good — but they can also prop up bad ideas indefinitely.

This is true even in democracies; in dictatorships, the pet projects of the powerful can squander money almost without limit. The Soviet Union was ruined, in part, by the inability of anybody to criticise and modify failing projects. And China began to move away from poverty when Deng Xiaoping allowed first farmers, then industry managers, to experiment with new ideas and shut down old ones.

For many government policies, it’s important to have an emergency stop to prevent bad ideas getting worse. But Mr Trump and Mrs May are like train passengers who hit the emergency stop because they’re having a nice chat on the phone and don’t want to be interrupted by a tunnel. There should be a penalty for misuse — and perhaps there will be.

I have no objection to bad ideas being reversed, but the problem here is that the reversals have been so nakedly political.

A wise policymaker changes course thus: “We had a promising idea, we tried it out on the smallest practical scale, we gathered data, we expanded our pilot programme, and then once the evidence was in, we decided that the idea wasn’t working. We’ve learned a lesson and will stop.”

Such changes of direction are what grown-ups do — and any well-run country should expect to see them regularly. Unfortunately there is no sense that either Mr Trump or Mrs May have changed direction on anything because they have been moved by new evidence on whether it works. Instead, they promised what seemed popular, and flinched at the first glimpse that it may not be popular at all.

The rest of us, meanwhile, conclude that politicians are inconstant liars who will say anything to win votes. Where did we get such a preposterous notion?

Written for and first published in the Financial Times on 26 May 2017.

My new book is “Fifty Things That Made The Modern Economy” – coming soon! If you want to get ahead of the curve you can pre-order in the US (slightly different title) or in the UK or through your local bookshop.

Free email updates

(You can unsubscribe at any time)

What are the best books about the history of technology?

I’ve had such fun working on the book and radio series, Fifty Things That Made the Modern Economy – and along the way I’ve had the opportunity to read some great books about the history of economics, finance, innovation and technology. Here are some of my favourites.

Alison Wolf The XX Factor (UK) (US) – Fun yet rigorous exploration of women’s participation in the workforce, and how and why it’s been changing over the years.

David Edgerton The Shock of the Old (UK) (US) – An appreciation of the humbler innovations in life, and a reminder that old technologies often stay with us a long time.

Felix Martin Money: An Unauthorised Biography (UK) (US) – Well-argued alternative history of money, packed with great stories.

Frank Trentmann Empire of Things (UK) (US) – An epic history of retail and consumerism.

Marc Levinson The Box (UK) (US) – The nerd-history to end all nerd-history. Levinson loves shipping containers and after reading this book, you will too.

Mark Kurlansky Paper (UK) (US) – I loved this book. Paper is even more underrated than shipping containers.

Mark Miodownik Stuff Matters (UK) (US) – Also good on paper – and concrete, and all sorts of miraculous materials we take for granted.

Robert Gordon The Rise and Fall of American Growth (UK) (US) – Magisterial history of technology and productivity. Closely-argued with lots of data but still feels rich and alive.

Steven Johnson How We Got To Now (UK) (US) – Stylishly-told history of six key innovations. You can’t fail to enjoy technological history the way Johnson tells it.

Steven Levy Hackers (UK) (US) – A modern classic, essential reading on the origins of modern computing.

William Goetzmann Money Changes Everything (UK) (US) – Like Gordon’s book, this is another academic read whose rich storytelling transcend the rigorous foundation. Great book.

Not the only sources for Fifty Things That Made the Modern Economy – but important ones. Enjoy!

 

Free email updates

(You can unsubscribe at any time)

 

19th of June, 2017Other WritingComments off

The age of the Microsoft and Amazon economy

One of my first economics lessons contrasted perfect competition, which was judged to be a good thing, with monopoly, which was not. There are worse places to begin than by being shown the difference between championing the miracle of the free market and favouring the depredations of dominant businesses.

But monopoly power has often seemed like yesterday’s issue. Standard Oil was broken up in 1911; AT&T in 1984. To the extent that we economists worried about companies being too big, we were thinking about the systemic risks from banks that were too big to fail. But we are starting to notice again the risks not of corporate failure but of corporate success.

The most obvious examples are the big digital players: Google dominates search; Facebook is the Goliath of social media; Amazon rules online retail. But, as documented in a new working paper by five economists, American business is in general becoming more concentrated.

David Autor and his colleagues looked at 676 industries in the US — from cigarettes to greeting cards, musical instruments to payday lenders. They found that for the typical industry in each of six sectors — manufacturing, retail, finance, services, wholesale and utilities/transportation — the biggest companies are producing a larger share of output.

For example, in the early 1980s the largest four players in any given US manufacturing industry averaged 38 per cent of sales; three decades later the figure was 43 per cent. In utilities and transportation the typical market share of the biggest four companies rose from 29 per cent to 37 per cent. In retail, overshadowed by Walmart and Amazon, the rise was dramatic: 14 per cent to 30 per cent.

This is surprising. As the world economy grows, one might expect markets to become more like the perfectly competitive textbook model, not less. Deregulation should allow more competition; globalisation should expose established players to pressure from overseas; transparent prices should make it harder for fat cats to maintain their position. Why hasn’t competition chipped away at the market position of the leading companies? The simplest explanation: they are very good at what they do. Competition isn’t a threat to them. It’s an opportunity.

What Professor Autor and his colleagues call “superstar firms” tend to be more efficient. They sell more at a lower cost, so they enjoy a larger profit margin. Google is the purest example: its search algorithm won market share on merit. Alternatives are easily available, but most people do not use them. But the pattern holds more broadly: superstar firms have grown not by avoiding competitors but by defeating them.

This is not entirely bad news. But it’s not entirely good news, either. The superstar firm phenomenon is the best explanation we have of a little-noticed but worrisome trend: since 1980, in the US and many other advanced economies, workers have been getting a steadily smaller slice of the economic pie (the distribution of this labour income also became much more unequal during the 1980s and 1990s).

Workers, from shelf-stackers to chief executives, have seen their total share of economic value-added fall from about 66 per cent to about 60 per cent in the US since 1980. This decline in “labour share” is often blamed on international trade making life harder for workers and easier for footloose capital. Prof Autor and his colleagues find little evidence for this idea.

Superstar firms, instead, seem to be the cause. The story is simple. These businesses are highly productive and achieve more with less. Because of this profitability, more of the value added by the company flows to shareholders and less to workers. And what happens in these groups will tend to be reflected in the economy as a whole, because superstar firms have an increasingly important role.

All this poses a headache for policymakers — assuming policymakers can pay attention to the issue for a long enough. The policy response required is subtle: after all, the growth of innovative, productive companies is welcome. It’s the unintended consequences of that growth that pose problems.

Those consequences are not easy to predict, but here are two possibilities. Either the US economy ends up like Amazon, or it ends up like Microsoft. The Amazon future is one of relentless competition, a paradise for consumers but a nightmare for workers, and with the ever-present risk that dominant businesses will snuff out competition as the mood takes them.

The Microsoft future epitomises the economist John Hicks’s quip: “the best of all monopoly profits is a quiet life”. Microsoft in the 1990s became famous as a once-brilliant company that decided to pull up the drawbridge, locking in consumers and locking out competitors.

In either scenario ordinary people lose out, unless they can enjoy returns from capital as well as returns from working. In the very long run a superstar economy could become a technological utopia, where nobody needs to work for a living. That would require quite a realignment in our economic system; I wouldn’t bet on such an outcome happening by chance.

 
Written for and first published in the Financial Times on 19 May 2017.

My new book is “Fifty Things That Made The Modern Economy” – coming soon! If you want to get ahead of the curve you can pre-order in the US (slightly different title) or in the UK or through your local bookshop.

Free email updates

(You can unsubscribe at any time)

I’m giving some talks. Come along!

50things-hb-uk-205x300A few upcoming talks, mostly about my new bookFifty Things That Made the Modern Economy. (UK) (US). Do come along if you can!

Interviewing Owain Service at Waterstones Piccadilly on 20 June 2017, 7pm. (Owain’s book, “Think Small“, is great.)

The UK launch of  Fifty Things That Made The Modern Economy at HowTo Academy, Conway Hall, 5 July 2017, 7pm.

Talking about Messy at OffGrid, Osea Island, 10 July 2017.

In conversation with the very smart Rohan Silva at Second Home, 19 July 2017, 7pm.

Talking about Messy and Fifty Things at the Edinburgh International Book Festival7.15pm 22nd August.

Talking about “What We Get Wrong About Technology” (inspired by Fifty Things) at the FT Weekend Festival, Kenwood House, London 2 September 2017.

Opining on “Fifty Things That Made the Modern Economy” at the Cheltenham Literature Festival at 5.30pm, Friday 6 October 2017. (details tbc)

The US editionFifty Inventions That Shaped the Modern Economy, will be out in late August. I’ll keep you posted!

Free email updates

(You can unsubscribe at any time)

12th of June, 2017MarginaliaSpeechesComments off

Where the truth really lies with statistics

As any magician can tell you, the real trick often takes place offstage. The deck has already been stacked; the card placed into the shoe. No matter how closely you look, you’ll never spot the sleight of hand.

So it is with statistics. We often pay attention to the wrong thing, scrutinising the numbers with a forensic eye without asking about what those numbers really describe. Sometimes there is no intent to deceive; there doesn’t need to be. We deceive ourselves.

A simple example: the recent official figures showing the number of homicides recorded by police in England and Wales jumped by more than 100 deaths, or 21 per cent. What could explain this? The Labour party called it “worrying”, which it is, and promised to hire more police officers.

Crime data cognoscenti know that the police database has long been regarded as a poor measure of anything very much, because the policy on recording crimes has changed over the years.

But the true reason not to be alarmed by the rise in homicides in 2016 is that the deaths did not take place in 2016. Ninety-six people died as a result of the disaster at the Hillsborough football stadium in April 1989. After years of campaigning these deaths were recorded as “unlawful killing” 27 years later. Deaths in 1989 became homicides in 2016. Sometimes the statistics are not counting what we assume they are counting.

Unexpected definitions can affect targets as well as trends. In the UK, the most notorious target is the one that keeps being missed: a promise to keep net migration under 100,000. In 2010 the then prime minister David Cameron challenged voters to kick him out if he missed the target. He did, and in a way, so did they. Encouraged by six years of failure to hit the target as home secretary, Theresa May has, now as prime minister, renewed the promise again.

How is this to be achieved? Leaving the EU won’t do the job alone: net immigration from outside the bloc has consistently exceeded 100,000. So attention has turned to a policy that many people regard as obvious: keep low-skilled immigrants out, and prioritise the highly skilled. For example, a recent policy paper published by the lobby group “Leave Means Leave” calls for a “moratorium on unskilled visas”. The paper proposes that working visas should be issued only to those who meet certain requirements, including a job offer on a salary of at least £35,000.

But this is an interesting slippage in the use of the word “unskilled”. About three quarters of UK employees earn less than £35,000, and as Jonathan Portes of King’s College London points out, the majority of nurses, primary schoolteachers, technicians, paralegals and chemists earn less than this figure.

Proposing an end to “unskilled migration” sounds reasonable to many voters; they might find it less reasonable if they realised that some definitions of “unskilled” would exclude a teacher or an intensive care nurse.

The word “cut” can also mean something different when it emerges from the mouth — or the Twitter account — of a politician. A few days ago the Labour leader, Jeremy Corbyn, tweeted: “Labour will stop the Conservatives’ £22 billion cuts to our #NHS immediately.”

No argument about the figure of £22bn: it is based on an estimate from NHS England two years ago. The problem is the word “cut”. The budget of the NHS in England is not being cut. It is increasing by £8bn after inflation.

The difficulty is that NHS England estimated that it needed not £8bn but £30bn. One might use many words to describe the £22bn shortfall — for example, “shortfall” — but “cut” is not one of them.

Amid all these surprising, confusing or deliberately vague labels, it was refreshing to see John McDonnell, the shadow chancellor, in action early in the election campaign. He promised that Labour would ensure that “the rich pay their way more”.

Normally, such statements mean very little. George Osborne, the former chancellor, was fond of saying that “those with the broadest shoulders” should bear the greatest burden, which sounds good but could mean almost anything. One definition of “rich” would be “earning more than £35,000” — most people don’t, as we’ve seen. Another might be “assets over £2.9m”, which according to the Institute for Fiscal Studies would put a British household in the top 1 per cent. Usually when politicians say “tax the rich” they hope that voters will hear “tax someone else”.

To Mr McDonnell’s credit, he actually produced a definition of “the rich”: people making more than £70,000 to £80,000 a year. That is admirably clear. Whether or not you agree with his definition or with his policy, at least we can understand it. This is rare.

Statisticians are sometimes dismissed as bean-counters. The sneering term is misleading and unfair. Most of the concepts that matter in policy are not like beans: they are not merely difficult to count, but difficult to define. If we don’t understand the definition there is little point in looking at the numbers. We have fooled ourselves before we have begun.

 
Written for and first published in the Financial Times on 12th May 2017.

My new book is “Fifty Things That Made The Modern Economy” – coming soon! If you want to get ahead of the curve you can pre-order in the US (slightly different title) or in the UK or through your local bookshop.

Free email updates

(You can unsubscribe at any time)

Undercover Monday 3

Books of the week

I read Elie Wiesel’s Night – his account of surviving Auschwitz. It’s a simple and powerful book, almost unbearable. I picked it up in a Bodleian Library reading room and for an hour or so, while I read, I was completely unaware of everything going on around me. Brilliant and horrifying. Everybody should read it. (UK) (US)

One thing that will stay with me was the desperation of Moishe, who knows what’s coming – but the Jews of Hungary simply cannot bring themselves to believe him. Wishful thinking triumphs. And the chilling words of a fellow prisoner to Wiesel: “I have more faith in Hitler than in anyone else. He alone has kept his promises, all his promises, to the Jewish people.”

 

Totally different in tone is an academic treatment of another serious topic, Alan Krueger’s What Makes A Terrorist? (UK) (US). It’s a useful antidote to the hysteria. Of course, I understand why people are terrorised by terrorists; that’s the whole point. I think we might expect more from our politicians, though. Here’s Krueger’s own take on his own book.

 

As an escape from all that, I’ve been reading Lloyd Alexander’s magnificent fantasy series The Chronicles of Prydain – available as five separate short novels or in a collected volume. (UK) (US). I can’t recommend them highly enough – simple enough for children to enjoy, but pacy, direct, full of imaginative twists. And the themes resonate into adulthood – for example, how does a child become an adult? Should we try to shield people from their own mistakes, or do they need to learn painful or even tragic lessons? I’ll be discussing the books soon on the Fictoplasm podcast.

 

On My Pile

Erik Barker: Barking up the wrong tree (UK) (US) – a guide to the latest and greatest in applied behavioural science.

Steve Mould: How To Be a Scientist (UK) (US) – fun science projects for children, but this one is a cut above the normal offering.

Tom Butler-Bowden: 50 Economics Classics (UK) (US) – TBB takes a broad view both of “economics” and “classic” here; for instance Naomi Klein is in here. But breadth is what you want from such a collection, I think.

Andrew Lo: Adaptive Markets (UK) (US) – Lo is a legend, looking forward to this, and to comparing it with the latest from another brilliantly original thinker…

Richard Bookstaber: The End Of Theory (UK) (US).

 

Recommended listening

Macro Musings interviews Paul Krugman.

Alphachat discusses the amazing Albert O Hirschman.

99 per cent Invisible does a live stage show about the Alaska Earthquake. It’s magnificent work.

And Soul Music is back. Best thing on BBC Radio.
My new book is “Fifty Things That Made The Modern Economy” – coming soon! If you want to get ahead of the curve you can pre-order in the US (slightly different title) or in the UK or through your local bookshop.

Free email updates

(You can unsubscribe at any time)

5th of June, 2017MarginaliaComments off

Personal finance sets traps for dinosaurs

It was free!” announces Bob the Dinosaur, an adorable moron from the Dilbert cartoon. Bob is driving a bright red convertible. “They just make you sign papers!”, he elaborates. That cartoon is a quarter of a century old, but some things never change. The suspicion lingers that too many people are buying cars using financial products they do not fully understand.

In the UK, the finger of suspicion is pointing at personal contract purchase agreements, or PCPs, which account for 80 per cent of new cars sold. The Prudential Regulation Authority and the Financial Conduct Authority are looking into the car finance sector (the FCA is supposed to prevent us being ripped off; the PRA is supposed to prevent banks accidentally ripping themselves off — thankless tasks).

It is difficult to explain quite how PCPs work, but easy to see the problem. Graham Hill, of the National Association of Commercial Finance Brokers, told the FT recently that using a PCP, drivers could pay less for a new BMW or Mercedes than for a second-hand Ford Focus. Or, as Bob the Dinosaur might put it, “they just make you sign papers!”

This miraculous effect is achieved by endlessly rolling over one quasi hire-purchase into another one, never quite buying a car. They are flattered by a buoyant used car market that is likely to sag before long. Yet manufacturers like them because they encourage people to buy new cars more often; car dealers like them because they generate more commission; and customers like them because the monthly payments are low. If you’re not worried yet, I have a car to sell you.

Some PCPs may be good value. The problem is that it is hard to tell. The closest analogy to a PCP — and it is not particularly close — is buying and selling a series of homes using interest-only mortgages. PCPs are a hybrid of several different financial products, part lease, part hire-purchase, and part option to choose between the two. Variables include contract length, the guaranteed value of the returned car, the deposit, purchase price of the car itself, maintenance contract tie-ins, mileage allowances, and (of course) the interest rate.

There is no reason to think customers can navigate these complexities. Suzanne Shu, an economist at UCLA, has shown that picking the cheapest mortgage deals is a problem that will fox even MBA students. PCPs are harder. We’ve seen this story play out before. There seems to be something about consumer finance that turns us all into Bob the Dinosaur.

George Akerlof and Robert Shiller, Nobel laureate economists and co-authors of Phishing for Phools (UK) (US), argue that the fallibility of consumers creates a profit opportunity. Consumer finance features heavily in their book, alongside junk food, cigarettes and slot machines — unflattering company. Given that a loan or insurance could be a life-saving product, why do financial services so often disappoint?

One reason is that finance shifts purchasing power over time and across different risky outcomes. We exhibit well-known biases when evaluating these different prospects, paying exorbitant prices to postpone a cost or eliminate a small risk. Payday loans, credit cards and extended warranties are real-world examples of that fallibility.

Second, financial contracts can create complexity out of simplicity. Even if we were thinking coolly about what was on offer, we might not be able to understand the small print — Suzanne Shu’s students couldn’t.

Third, many financial contracts are bundled up with other purchases — the PCP, plus mobile phone contracts and overpriced insurance for short-term car hire. Payment protection insurance is the infamous rip-off endgame.

Costly add-ons are not always a disaster. Everyone knows that popcorn is expensive at the cinema, and no regulatory intervention is needed. But all too often we’re seeing the primary product serving as bait for a consumer finance trap.

The best defence of laissez-faire in such cases is not that the market works well, but that regulators would make it worse. Is that true? Do regulators have a sensible response to the toxic tangle of slick salesmanship, financial wizardry, and consumer incompetence?

We could ban complex contracts: tempting, but heavy-handed. Most financial contracts have a rationale and a value to some customers. Richard Thaler and Cass Sunstein, authors of Nudge (UK) (US), have proposed an alternative system they call RECAP — for “Record, Evaluate, Compare Alternative Prices”. A RECAP rule would require finance companies to provide the entire pricing schedule in a computer-readable format, which would allow customers to go to third-party websites and compare different deals in a sophisticated way.

What auto finance needs — what most consumer finance needs — is for key information to be made simple and salient. Competition cannot work if consumers struggle to understand what they’re being sold and what it will cost. The car market’s heady mix of prestige products and bewildering finance will resist efforts at reform. Yet we must try. Bob the Dinosaur needs help.

Written for and first published in the Financial Times on 5th May.

My new book is “Fifty Things That Made The Modern Economy” – coming soon! If you want to get ahead of the curve you can pre-order in the US (slightly different title) or in the UK or through your local bookshop.

Free email updates

(You can unsubscribe at any time)

Elsewhere

  • 1 Twitter
  • 2 Flickr
  • 3 RSS
  • 4 YouTube
  • 5 Podcasts
  • 6 Facebook

Books

  • Fifty Inventions That Shaped the Modern Economy
  • Messy
  • The Undercover Economist Strikes Back
  • Adapt
  • Dear Undercover Economist
  • The Logic of Life
  • The Undercover Economist

Tim’s Tweets

Search by Keyword

Free Email Updates

Enter your email address to receive notifications of new articles by email (you can unsubscribe at any time).

Join 4,329 other subscribers

Do NOT follow this link or you will be banned from the site!