Tim Harford The Undercover Economist

Articles published in May, 2018

The world is not as gloomy, or wonderful, as you may think

Written for and first published in the Financial Times on 20 April 2018.

Is the glass half full, half empty, or laced with cyanide? Last week I wrote about “statistics, fast and slow” — the gap between the world as we intuitively perceive it, and the world as described in spreadsheets. Nowhere is this gap more obvious than when we are invited to reflect on whether things are going well, or badly.

With some telling exceptions, the situation is this: the world is getting better in many of the ways that matter, but we simply don’t realise that this is true. Population growth has slowed dramatically. Most of the world’s children have been vaccinated against at least one disease. Girls are rapidly catching up with boys in their access to education. The world is full of flaws, but progress is not only possible — it is happening.

A new book, Factfulness (UK) (US), by Anna Rosling Rönnlund, Ola Rosling and the late Hans Rosling, describes this knowledge gap, which is at times grotesque: two-thirds of US citizens believe the global proportion of people living in extreme poverty has doubled in the past couple of decades; it has halved. As Hans Rosling used to say, we don’t become this ignorant by accident.

Nor are our misperceptions limited to global development. Surveys by the polling company Ipsos Mori show that citizens of the developed world are also ignorant about our own countries. Most people vastly overestimate the prevalence of crime (which in the UK is dramatically down since the 1990s) and teenage pregnancy (which affects fewer than 1 per cent of 13-15 year old girls). We also seriously overestimate the size of the Muslim population in the west, which suggests that the concerns of tabloid newspapers loom large in our imaginations.

This is not just a statistical phenomenon — it’s a political and psychological puzzle. How worried should we be about unemployment, vandalism, immigration, litter, bad hospitals, or drug dealing? There is no objective answer, but there is a strong tendency for people to be concerned about these issues for their nation, but more relaxed about their local area. We don’t see a serious problem where we live, but we feel strongly that trouble is all around us, just over the horizon. The economist Max Roser — creator of Our World in Data — calls this “local optimism and national pessimism”.

The mismatch is particularly stark when people are asked about their own happiness. Almost all of us are reasonably content: in the UK, 92 per cent of us are “rather happy” or “very happy” with our lives. But we believe that fewer than half of our fellow citizens are in the same cheery situation. The UK is typical in this respect: full of happy people who believe they are surrounded by misery.

This generalised pessimism seems powerful. The one global question that people reliably get right, despite ferocious misinformation campaigns, is the one where the news is bad: do climate experts believe the planet will get warmer over the next century?

So it would be tempting to conclude that we are all systematically too pessimistic about everything except our own experience. That is not quite true. The FT’s chart doctor, Alan Smith, tells me that Saudi Arabians are far too sanguine about the prevalence of obesity: they think a quarter of the nation is overweight or obese, but the true figure is closer to three-quarters. Most people in most countries also underestimate wealth inequality; it’s worse than we think, although, here, the UK is an exception to this belief.

The optimists are not right about everything. Angus Deaton, Nobel laureate in economics, has found that we are too optimistic about our own futures: almost everywhere, people tend to feel that they will be living a strikingly better life in five years’ time. We are doomed to disappointment. Life satisfaction is already high, does not tend to move much, and if anything tends to fall as mid-life approaches.

This misplaced optimism about ourselves is a striking contrast to an equally misplaced despair about our children: across Europe and North America, according to the Pew Research Center, twice as many people believe their children will be worse off financially than they are, rather than better off. Given the past decade of recession and slow recovery, that is not impossible. But economies do tend to grow over the long term, so it is a remarkably grim forecast.

What should we conclude from all this? One plausible hypothesis is that we form many of our impressions about the world from the priorities of the mass media. That would explain why we are pessimistic about most things, but not about obesity, since television loves skinny people.

A second conclusion is that many of us — citizens, the media and mainstream politicians — need to take more interest in the way the world really is. I hardly need to list the political movements that have travelled from the lunatic fringe to positions of power by reinforcing people’s worst fears. But when your policy platform is built on misperceptions, little good is likely to come of it. Optimism and pessimism both have their merits, but right now the world needs a dose of realism.

My book “Messy: How To Be Creative and Resilient in a Tidy-Minded World” is now available in paperback both in the US and the UK – or through your local bookshop.

Free email updates

(You can unsubscribe at any time)

Statistics, Fast and Slow

Written for and first published in the Financial Times on 13 April 2018.

One way to understand China is to look at the statistics. Real income per person has increased nearly tenfold since 1990. Since the early 1980s, the number of extremely poor people in China has fallen by more than three-quarters of a billion people, more than half the population of the country. China consumed more cement in a recent three-year period than the US used in the entire 20th century.

Even on paper, it is the most dramatic explosion of economic activity in human history. Seeing it with your own eyes is another experience entirely.

Nothing in the statistics truly prepared me for a journey across Guangdong, the southern province of China that has been at the forefront of this growth. Start at Hong Kong — the ultimate high-rise city — and walk into its mainland twin, Shenzhen. Then in the shadow of the Ping An skyscraper, which dwarfs the Empire State building, catch a bullet train across the province.

Where London might have a single big block like Trellick Tower, Shenzhen will have a cluster of a dozen identical monoliths, crammed with apartments. Next to that cluster, another dozen of a different design. Then another, and another. Here and there, in the distance across the haze, would be a Manhattan-esque cluster of skyscrapers. The towers marched on and on, all the way — or so it seemed to me — to the city of Guangzhou: 45 minutes or so of high-speed travel through what seemed an infinite vista of concrete.

That night, tucked into bed in the picture-postcard landscape of Yangshuo, I couldn’t sleep. The endless tower blocks scrolled through my mind. What if we had lost our six-year-old son in the middle of Guangdong? So many people. So much concrete.

There was nothing in this experience to contradict the economic data; in fact, the two perspectives on China’s growth were perfectly complementary. But they felt very different. To borrow the terminology made famous by Daniel Kahneman’s book Thinking, Fast and Slow, (UK) (US) the statistics spoke to my mental “system 2” — the deliberate, effortful processing of logical or mathematical information. The train journey tapped into “system 1”, a swift and automatic forming of impressions, making of comparisons and recognising of dangers. This was statistics, fast and slow.

Some will be tempted to dismiss the statistics as irrelevant book-learning, and declare that only personal experience matters. There is certainly something in that, especially when a situation is fast-moving or contains soft, hard-to-quantify details. As the Nobel laureate economist Friedrich Hayek remarked, the “knowledge of the particular circumstances of time and place” is important and often neglected.

HR McMaster — who before he was US president Donald Trump’s former national security adviser, was a counterinsurgency pioneer in Iraq — had a similar concern. He once told me the army used to wrongly believe that “situational understanding could be delivered on a computer screen”. It would be convenient if that was possible, but as Gen McMaster and his colleagues learnt the hard way, it is not. Sometimes you have to be there to understand.

But while there is a lot to be said for the rich and vivid lessons of personal experience, they have an obvious limitation: we cannot be everywhere and see everything. And what we do see may be as unrepresentative as the sloppiest of surveys. My trip to China took in tourist spots and high-speed rail links. As a result, I formed an indelible impression of a very particular slice of China.

The skew in our personal experience affects us when at home almost as much as when travelling. We are surprised when an election goes against us: all our friends agreed with us, so why did the nation vote otherwise? Newspapers and television carry tales of lottery wins and fairytale romances, terrorist atrocities or gruesome assaults by strangers. None of these stories reflect everyday life; all of them are viscerally memorable and seem to take place in our living rooms.

And there are more subtle ways in which personal experience can mislead. For example, most of us who ride on London buses will attest that they are packed. Yet the average occupancy of a London bus is just 17 people. How so? Most people witness the full buses — that is why they are full — while empty buses are observed only by their drivers.

It is not quite fair to say that our fast-and-loose “system 1” impression is a lie. It really is true that most people travel on busy buses. But if we want to understand emissions per passenger, we need a statistical perspective.

A new book by the late Hans Rosling and his family, Factfulness (UK) (US), advocates the merits of understanding the world both through the data and through personal experience — not of news stories or tourist traps, but of the everyday lives being lived all over the world. “Numbers will never tell the full story of what life on Earth is all about,” wrote Rosling, despite being the world’s most famous statistical guru. But the story they do tell matters. In statistics, as elsewhere, hard logic and personal impressions work best when they reinforce and correct each other.


My book “Messy: How To Be Creative and Resilient in a Tidy-Minded World” is now available in paperback both in the US and the UK – or through your local bookshop.

Free email updates

(You can unsubscribe at any time)

Understanding the wisdom and madness of crowds

James Surowiecki’s modern classic The Wisdom of Crowds (UK) (US) set a very high bar for the field. (Why has James not written another book?) This is one of those books that gets talked about a lot, with the emphasis on the idea that the average opinion of the crowd can be very smart indeed – hence prediction markets, etc. etc. etc. All that is true and interesting, but in fact Surowiecki discusses lots of other situations where a group needs to make a decision and covers groupthink and all that good stuff. In short it is a messier and more complex – and also deeper and more interesting – book than many people realise. Well worth a read, or a re-read.

Then there’s Philip Ball’s superb book Critical Mass (UK) (US) – which really lit my fire when I read it back in 2005. Ball’s book asks what social scientists can learn from ideas in physics and chemistry about how large groups of decision-makers behave. Lots and lots of interesting ideas and good stories. A good alternative, although I do not recall it so vividly, there is Steven Strogatz’s Sync (UK) (US); it has been commercially successful so the wisdom of crowds suggests you might take it seriously.

Michelle Baddaley’s new book Copycats and Contrarians (UK) (US) is a good accessible survey of what different academic disciplines have to say about herding, fashion, group dynamics and all such things. I blurbed the book and said, “‘A wide-ranging cross-disciplinary perspective of why we run with–or avoid–the crowd, and why it matters, from choosing a restaurant in a tourist trap to believing fake news. I learned a lot, and you may too.”


Then on the psychology of group decision-making there is Wiser (UK) (US) by Cass Sunstein and Reid Hastie. I love this book – my favourite by Sunstein, even better than Nudge. Lots of fascinating ideas about polarisation, echo chambers – and plenty of intriguing research.

Or, try Scott Page’s The Diversity Bonus (UK) (US)Page writes with great clarity about complex ideas in algorithms and complexity science, so you’ll learn a lot about those subjects. But the book is also an excellent argument in favour of embracing cognitive diversity in problem-solving teams.

Next up: the always-interesting Francesca Gino has published a brand new book about breaking out of groupthink called Rebel Talent (UK) (US). It’s next on my list to read.

My own book Messy: How To Be Creative and Resilient in a Tidy-Minded World discusses group dynamics and creative friction in the second chapter, and that’s one of the chapters that seems to have struck a chord with readers.

Come for the complex network analysis of the teams which made the best computer games in history, stay for the mind-blowing “Lord of the Flies” research into 10 year old boys at summer camp. The book is now available in paperback both in the US and the UK – or through your local bookshop.

Free email updates

(You can unsubscribe at any time)


8th of May, 2018MarginaliaResourcesComments off

Judge the value of what you have by what you had to give up to get it

Written for and first published in the Financial Times on 6 April 2018.

I’m not one to collect inspirational slogans, but here’s one I like: “Judge the value of what you have by what you had to give up to get it.” Perhaps I took it more seriously because it was pinned to the corkboard of an inspirational friend; she always seemed to be off for another expedition to Mongolia or Patagonia.

But my fondness for the motto may reflect that it describes an under-appreciated idea in economics: that of opportunity cost. And I’ve come to realise that our collective failure to think rationally about opportunity costs can be used as a weapon against us.

The principle of an opportunity cost does not at first glance seem hard to understand. If you spend half an hour noodling around on Twitter, when you would otherwise have been reading a book, the lost book-reading time is the opportunity cost of the tweeting. If you decide to buy a fancy belt for £100 instead of a cheaper one for £20, the opportunity cost is the £80 shirt you could otherwise have bought. Everything has a cost: whatever you were going to do instead, but couldn’t.

We should weigh opportunity costs with some care, mentally balancing any expenditure of time or money against what we might do or buy instead. However, observation suggests that this is not how we really behave. Ponder the agonised indecision of a customer in a stereo shop, unable to decide between a $1,000 Pioneer and a $700 Sony. The salesman asks, “Would you rather have the Pioneer, or the Sony and $300 worth of CDs?”, and the indecision evaporates. The Sony it is.

This vignette was sketched in a research paper entitled “Opportunity Cost Neglect”, published by five behavioural scientists (in 2009, hence the mention of CDs). What makes the anecdote curious is that it is hardly an act of genius to figure out that buying the $700 Sony stereo would save $300, nor that $300 will buy $300 worth of CDs. It is not that the indecisive shopper couldn’t work this out, but that the explicit trade-off never crossed his or her mind.

Various experiments in the research paper supplement the anecdote with some data. And other research in psychology suggests that our attention is far narrower and more fleeting than it seems. As psychologist Nick Chater explains in a remarkable new book, The Mind is Flat (UK) (US), the brain generates powerful illusions of continuity. It stitches together what is actually a patchwork of fleeting impulses and perceptions.

We feel intuitively that we are able to check our phones while simultaneously keeping an eye on the road ahead, but we can’t. We think we can summon to mind a clear image of a tiger, whiskers twitching, fur shining, licking its lips. But asked to draw a tiger we start to struggle. Do the stripes on its legs loop laterally around, or run vertically?

It is the same with opportunity cost. We tend to feel that our choices reflect the whole picture: as crisp and vivid as the tiger, a balanced consideration of all the alternatives. But often we spend money simply out of habit or instinct.

Drawing our attention to opportunity costs, no matter how obvious, may change our decisions. The notorious falsehood on the campaign bus used by Vote Leave during the 2016 referendum campaign was well-crafted in this respect: not only could the UK save money by leaving the EU, we were told, but that money could then be spent on the National Health Service.

One could certainly debate the premise — indeed, the referendum campaign sometimes seemed to debate little else — but the conclusion was rock solid: if you have more money to spend, you can indeed spend more money on the NHS. (Just another way in which that bus was a display of marketing genius.)

We would make better decisions if we reminded ourselves about opportunity costs more often and more explicitly. Nowhere is this more true than in the case of time. Many of us have to deal with frequent claims on our time — “Can we meet for coffee so that I can pick your brains?” — and find it hard to say no. Explicitly considering the opportunity cost can help: if I meet for coffee I’ll have to work an hour later, and that means I won’t be able to read my son a story before bedtime.

There may also be situations where we make the opposite mistake. If you save £100 in some act of thriftiness, that is £100 you can spend on a case of wine, or a good shirt, or dinner for two. But you cannot spend the same £100 on all three. While we would be wise to explicitly consider what else we might do with our money, we should be careful not to spend it over and over again — something political manifestos have a tendency to do.

So, the inspirational motto is right. We should judge the value of anything by what we had to give up to get it. And we should get in the habit of doing this deliberately. If it was an automatic process, we would need no inspirational motto to remind us.

My book “Messy: How To Be Creative and Resilient in a Tidy-Minded World” is now available in paperback both in the US and the UK – or through your local bookshop.

Free email updates

(You can unsubscribe at any time)

4th of May, 2018Undercover EconomistComments off


  • 1 Twitter
  • 2 Flickr
  • 3 RSS
  • 4 YouTube
  • 5 Podcasts
  • 6 Facebook


  • Fifty Inventions That Shaped the Modern Economy
  • Messy
  • The Undercover Economist Strikes Back
  • Adapt
  • Dear Undercover Economist
  • The Logic of Life
  • The Undercover Economist

Search by Keyword

Tim’s Tweets

Free Email Updates

Enter your email address to receive notifications of new articles by email (you can unsubscribe at any time).

Join 162,945 other subscribers

Website Malware Scan
Do NOT follow this link or you will be banned from the site!