Tim Harford The Undercover Economist

Articles published in November, 2018

The untold career value of a little bit of luck

Almost exactly 20 years ago, fresh out of graduate school, I started work at a smallish strategy consulting firm. It was a poor choice, both for them and for me; I am not cut out to be a management consultant. I was even allergic to my own suit.

When one of my fellow recruits learnt this she pointed out that, in this job, “you only need to do two things: talk shit and wear a suit, and you can’t do either of them”. (I like to think that I have since mastered at least one of those skills.)

In any case, I was miserable and useless. My employers generously suggested that I might want to resign, and that if I did they would happily keep paying me for a while. I followed their advice and found a much more conducive job, most of which I was able to perform without embarrassment, wearing blue jeans. I had been very lucky.

But it is only recently that I realised that some of my luck required being born at the right time. Had I been a year younger I’d have crashed out of my job, not in the spring of 1999 but of 2000, just as the dotcom bubble was bursting.

Pity the babies of 1987 and 1990, then, who left school or university around the time that Lehman Brothers collapsed in 2008. Sending around a CV in the middle of the greatest financial crisis since their grandparents were born cannot have been a whole lot of fun.

Of course the financial crisis made life difficult for a lot of people, not just graduates. It is now over, although the scars remain. So here’s a question: are those people who were unfortunate enough to have been looking for their first job as a recession struck still disadvantaged after the recession itself has passed?

In 2006, an economist called Paul Oyer posed that very question about the young academics he was teaching. Let’s say that two equally able young economists, Alexandra and Betsy, are looking for work. Alexandra arrives on the job market during the good times, when departmental budgets are fat, and is hired by the 30th best university in the country. Betsy is a couple of years younger, tries to find a job during a lean year, and can only get a job at the 60th best university.

The question that Professor Oyer posed is: will this matter in the long run? Will the equally talented Betsy find a better job, given a few years? Or will she be trying to catch up with Alexandra for decades?

Prof Oyer assembled data describing the PhD students graduating from seven excellent graduate schools, and he concluded that Betsy would remain at a disadvantage for a long time. Students who graduate in good years are more likely to find good jobs — obviously — but there is also a strong correlation between getting a good job immediately and still having a good job four, eight or even 12 years later.

This makes some sense; if an economist applies for a mid-career job having already been an assistant professor at (say) the Massachusetts Institute of Technology, employers are unlikely to adjust for whether she secured that assistant professorship in an impossibly difficult year or at a somewhat easier time. The halo will shine, regardless. And regardless of whether it was a good year or a bad one, the young economist will have picked up skills from working at MIT, teaching MIT students and rubbing shoulders with Nobel Prize winners. If you want to work in a top research job, it helps a lot to start out in a top research job.

Prof Oyer conducted a similar analysis for MBA graduates looking for high-paying jobs in finance and consulting in the late 1980s. The results were similar. Could this be a problem only for the young elite — students on such a precision-engineered career path that their fate is sensitive to accidents of timing? Or might it be even more serious for those further down the educational pecking order?

A new study from economists Hannes Schwandt and Till Marco von Wachter suggests that the latter is true. Looking at young people entering the US labour market between 1976 and 2015, every group suffers lasting harm if they have to find their first job during a recession, but disadvantaged groups suffer more and for longer. High school dropouts fare particularly badly, both immediately and several years later, as do those from a racial minority. People with a college education suffer less.

Overall, for a typical recession, the unlucky cohorts can expect to lose the equivalent of seven months’ pay over the course of a decade, relative to their more fortunate peers, who are only a couple of years older or younger. That’s no trivial sum. People who can’t find the job of their dreams end up settling for something else, building up skills and contacts in a field that was never their first choice.

Few people, by now, need reminding that the financial crisis has had a lasting impact and that, in many ways (though not all), it is the younger generation that has been left to count the cost. But this research points towards other lessons. It’s easy to overlook luck; but good or bad, a single piece of luck can last in ways that we find it hard even to notice.


Written for and first published in the Financial Times on 2 November 2018.

My book “Fifty Things That Made the Modern Economy” (UK) / “Fifty Inventions That Shaped The Modern Economy” (US) is out now in paperback – feel free to order online or through your local bookshop.

Free email updates

(You can unsubscribe at any time)

What the Sydney Opera House teaches us about Brexit

Go down to Bennelong Point and make such progress that no one who succeeds me can stop this going through to completion.” That is how the ailing Joseph Cahill ensured that the Sydney Opera House would be built — at least according to one of his successors.

Excavations for the project duly began at Bennelong Point. Cahill, the premier of the state of New South Wales, died of a heart attack not long after. The Sydney Opera House was indeed completed, but in 15 years rather than five and at a cost of A$102m rather than A$7m — a truly impressive cost overrun of nearly 1,400 per cent.

Brexit, it seems, is also likely to overrun its original schedule, to the surprise of nobody who has been paying attention. I decided to look up Bent Flyvbjerg, perhaps the world’s leading authority on “megaprojects”, large and ambitious endeavours such as hosting the Olympic Games, building a high-speed rail line, or overhauling a big IT system. Such projects often — although not always — go badly wrong.

Is it useful, then, to think of Brexit as similar to a large construction or IT project? Professor Flyvbjerg’s answer: Yes, it’s a very useful analogy indeed. But, given the failure rate of megaprojects, it is not an encouraging one.

Brexiters will point out that some megaprojects live up to their promise, most famously the Guggenheim Bilbao— a tenacious, visionary scheme delivered on time, on budget and with benefits outstripping any reasonable expectations. Remainers anticipate the best-case Brexit scenario to be something more like the NHS National Programme for IT, abandoned after a decade and several billion pounds of wasted money.

Whichever side of this debate you are on, there is much to be learned from studying megaprojects. I gleaned six pieces of counsel from Prof Flyvbjerg to improve chances of successfully delivering a complicated project. Checking the list against what is happening with Brexit makes my heart sink.

First: prepare thoroughly. This one is awkward. Civil servants were banned by David Cameron’s government from preparing for Brexit before the referendum. Then, for her own reasons, his successor as prime minister, Theresa May, scrambled to begin the Article 50 countdown. “Zero preparation”, says Prof Flyvbjerg, “is as bad as it gets”.

Second: try to de-bias yourself, noting and adjusting for overconfidence, wishful thinking and other well-known cognitive biases. Alas, most British politicians are wary of seeming negative or timid about Brexit, for fear of implying the electorate was unwise. Instead, both government and opposition have embraced the goal of leaving while enjoying all the benefits of staying — hardly a clear-eyed exercise in spotting obstacles.

Third: choose an experienced team. Hmm . . . Sir Ivan Rogers, the UK’s ambassador to the EU and an experienced negotiator abruptly resigned early in the process.

Fourth: try to break a large project into smaller, standalone chunks, so that the failure of one is not a failure of everything. When everything is interconnected, small obstacles can snowball into major delays. (See also: The Irish border.) The logic of both politics and of diplomatic negotiation for Brexit points in the opposite direction: “nothing is agreed until everything is agreed”.

Fifth: key decision makers should be aligned, with everyone having an incentive to make things move smoothly. Alas, politics (again) pushes in the wrong direction here. Many of the people with power to smooth the way for Mrs May, from opposition leader Jeremy Corbyn to the former foreign secretary Boris Johnson to the EU’s chief negotiator Michel Barnier, have the incentive to make things difficult for her in one way or another.

Finally: have an early warning system so problems can be spotted and fixed before they grow. Yet warnings are routinely derided as “Project Fear”.

Even without political pressures, megaprojects are hard to deliver on time and on budget: their sheer scale opens up a thousand ways for things to go wrong. But they are always somewhat political, and Brexit is more political than most. Even as a pure organisational challenge, it is likely to take far more time and money than advertised.

The Sydney Opera House itself is, of course, a stunning achievement. But unlike the Guggenheim Bilbao it is an achievement built on lies. Those lies came from politicians who decided that an honest account of the likely costs would not achieve their goals. They tarnished the reputation of Jørn Utzon, the architect who became the scapegoat for their impossible promises. Lauded far too late, he received no other major commissions and never saw the finished Opera House. Would the truth really not have served?

As far as Brexit is concerned, we have dashed down to Bennelong Point and started shovelling frenetically, desperate that no one should “stop this going through to completion”. Perhaps one day we will get the Sydney Opera House, although that seems unlikely. At the moment, we’re at the bottom of a deep hole and we are still digging.


Written for and first published in the Financial Times on 26 October 2018.

My book “Fifty Things That Made the Modern Economy” (UK) / “Fifty Inventions That Shaped The Modern Economy” (US) is out now in paperback – feel free to order online or through your local bookshop.

Free email updates

(You can unsubscribe at any time)

Self help books that work

The self-help genre gets a bad press, and not without reason, but there are a few self-help books that I’ve read, enjoyed, and felt wiser as a result.

The Tao of Pooh (UK) (US) by Benjamin Hoff is a wise, funny meditation on life that changed the way I looked at the world when I read it as a 19 year old. I may be older and a little more cynical these days but this still feels like a book worth reading.

TED Talks (UK) (US) by Chris Anderson is the best book on public speaking I’ve ever read, and I’ve read a lot of books on public speaking. While I admire what Chris has done with TED, I expected to disagree with a lot of his advice. Nope; he won me over.

Getting Things Done (UK) (US) by David Allen is a modern classic with a cult following. The book feels a bit fussy, full of jargon, and over-complex. But the truth is that GTD wouldn’t have so many fans (including me) if it didn’t work on some basic level. The key idea of GTD is that you need to write down what’s on your mind, somewhere where you trust yourself to check at the right moment – and as a result, you’re more relaxed and more confident that at any particular moment you’re focusing on something sensible rather than leaving a time-bomb ticking away in your inbox. The rest is detail but the details do seem to matter. A strong recommendation from me.

Deep Work (UK) (US) by Cal Newport. Stop messing around and focus on something hard.

The Creative Habit (UK) (US) by Twyla Tharp. A superb book about creativity and the effort involved. Some great stories and advice – and it’s an intensely pragmatic guide to living a creative life.

Designing Your Life (UK) (US) by Bill Burnett and Dave Evans. Measured by the number of copies I’ve given away this must be my favourite book. It’s humane and practical, proposing that we use designers’ methods such as prototyping and brainstorming to create better, more fulfilling lives and careers. Full of good-yet-unusual ideas.


Free email updates

(You can unsubscribe at any time)

21st of November, 2018ResourcesComments off

There is a fine line between stupid and clever

Recently a loyal reader emailed me with a disarmingly simple question: is it better to invest small amounts every month, or in a large lump sum? What seems a narrow topic has broader implications.

The popular argument in favour of regular investment into shares — often called “cost averaging” — has been made by retail investment advisers many times. “Take advantage of their downs as well as their ups,” says one, so that “if you invest a fixed sum every month you will be able to buy more units when a fund’s value falls”.

To see how this cost averaging might work, consider a share that costs $60 half the time and $120 the other half. Investing $600 a month returns 10 shares in each of the six months when it is cheap and five shares in each of the expensive months. At the end of the year, 90 shares have cost $7,200. The average cost per share is only $80, much closer to $60 than $120 and thus excellent value.

The argument is plausible, intuitive, and wrong — both in theory and when tested with hindsight against historical market data.

Here’s the problem with the theory: while it is unclear exactly how best to describe the fluctuations of share prices, they do not simply move up and down in the way my simple example describes. If they did, a far better approach would be to invest only when they cost $60, not when they cost $120.

A more realistic description of the stock market is that it follows a random walk with an upward drift. The random walk implies that we cannot expect prices to tend back towards some average. From any particular point there is no way to say in which direction they will lurch next. That means that we cannot expect that when a share is $120 it is likely to fall, and when it is $60 it is likely to rise.

The upward drift — the simple fact that share prices tend to rise over time — suggests that we should invest everything we plan to invest at the earliest opportunity. Drip-feeding delays profitable investment, and so costs money.

If theory is unkind to the cost-averaging principle, what about the historical evidence? We can ask, with hindsight, when investors with a large lump sum would have done better to drip it gradually into the market over the course of a year. The answer is unsurprising: drip-feeding has done better only when the market then fell, and since markets rise more often than they fall, lump-sum investing is a better bet. Cost averaging worked well in the last bear market, but, with access to that kind of hindsight, an even better strategy in a bear market is to wait until it is over.

Cost averaging, then, is wrong in theory and has not usually worked in practice.

It may, nevertheless, be excellent advice.

The simplest point in favour of drip-feeding is that it reflects the situation of a typical salaried investor. Purists will say that “cost averaging” should only be used to describe a deliberate strategy of delaying investment, but retail advisers often speak of the magic of cost averaging while praising regular investment. The magic may be illusory, but the benefits of regular investment are not.

Many of the empirical tests of cost averaging begin from the premise that an investor is sitting on some vast pile of cash, pondering whether to invest gradually or all at once. That is a pleasant dream to consider, and if you find yourself with a million dollars in your pocket then by all means invest promptly.

If you are nervous about risk, academic research suggests that drip feeding is not the most efficient way to reduce your risk. Better to keep a small portion of your wealth out of the stock market entirely.

But efficiency is a treacherous goal for an ordinary investor.

The strongest argument in favour of cost averaging is simply to ask ourselves what we are likely to do instead. The answer is not pretty. Researchers have found that retail investors tend to make two simple errors: they lose money by trading too much, and they tend to buy high and sell low. Without getting too technical, let me assure you that this is not the aim. If regular investments displace a Gordon Gekko complex, that is enough for me.

While markets do not swing back and forth with the metronomic predictability of my earlier illustration, they do fluctuate a lot, as we have seen in the past couple of weeks. That fluctuation is distressing for most people. If facile arguments for cost averaging reassure small investors and stop them from selling at the bottom, that is no bad thing.

Greg Davies, a behavioural finance expert at Oxford Risk, describes cost averaging as “deliberately doing something slightly inferior, to prevent the likelihood of something very inferior”. Just so. And it is worth looking for other areas where we might benefit from being guided by a slightly inferior rule of thumb — anything from “if it takes less than two minutes, do it immediately” to “never drink alone”. There are exceptions to these rules, but you may be better off just sticking to the rules.

As a wise man once said, it’s such a fine line between stupid and clever. Cost averaging seems clever, but we should recognise that its true value lies in not being stupid.



Written for and first published in the Financial Times on 19 October 2018.

My book “Fifty Things That Made the Modern Economy” (UK) / “Fifty Inventions That Shaped The Modern Economy” (US) is out now in paperback – feel free to order online or through your local bookshop.

Free email updates

(You can unsubscribe at any time)

Does economic growth need to end?

What are the limits to economic growth — and have we already recklessly exceeded them? Such questions were raised (again) recently by (another) alarming report about climate change. Many of my environmentalist friends are convinced that economic growth itself is the fundamental problem.

It was a timely moment, then, to give a Nobel Prize to two economists who’ve tackled that question head on. William Nordhaus and Paul Romer have tried to find ways to understand the invisible and sometimes ineffable causes and consequences of growth.

The modern world produces two things in abundance: carbon dioxide and ideas. Both swirl around, defying our attempts at control. We’d like more ideas but already have more than enough carbon dioxide. The future of humankind may depend on a strange race: can we keep living standards rising yet restrain consumption of resources and production of pollutants?

Economics being economics, Nordhaus and Romer received their prizes for technical achievements in economic modelling. Mr Nordhaus analysed the interaction between climate change and the economy; Mr Romer developed an elegant way to model innovation as an intrinsic part of the growth process, rather than falling from heaven. These are impressive intellectual accomplishments, but my fascination with both men concerns some of their more informal work.

In one of the economics papers I truly love, Mr Nordhaus tracked the price of illumination over the millennia, from the days when people could create light only with a campfire, through the time when they would use beef tallow — or clean, bright-burning spermaceti oil from whales — to the invention and improvement of incandescent bulbs.

Mr Nordhaus chopped and burnt wood, and tested antique lamps with a Minolta light meter. He concluded that in Babylonian times, a day’s hard work would produce enough to light a room for 10 minutes. By the end of the 20th century, the return on a day’s labour had improved from 10 minutes of light to 10 years. That is the kind of progress that gives one hope for us all.

The environmental toll paid for that light has also plummeted, which is good news for the whales and good news for us. Perhaps it really might be possible to enjoy the comforts of modernity without destroying the planet.

Since the early 1960s, UK carbon dioxide emissions per person have almost halved, yet the country’s economic output per person has tripled in real terms. This is partly due to moving production abroad, but most of it is from producing more value with fewer physical resources and a lot less coal.

That is where Mr Romer comes in. Like Mr Nordhaus, he is impressed by our capacity to make (and then take for granted) innovative progress and argues that there is room for much more. Consider the compact, self-repairing, mobile, renewable-resource-powered chemical reactor that we call a “cow”. Courtesy of evolution, it is vastly more impressive than human-designed facilities. This elegance, suggests Mr Romer, tells us that there is plenty of room for us to do things better.

That is also true for the institutions that produce new ideas. While Mr Romer’s prizewinning work makes particular assumptions about who pays for new ideas and who benefits when they are produced, his informal writing and policy work highlights that these things cannot be taken for granted. He wrote not long ago that “only a failure of imagination” allows us to conclude that in today’s universities, intellectual property rules and scientific norms we have perfected the way we develop and diffuse new ideas.

We should constantly be searching for better ways to do things — as Mr Romer himself did with a successful foray into digital learning, ahead of the trend, and later with his bold and controversial push for “ charter cities”, in which a country with weak institutions might outsource the governance of a greenfield city site to Canada or Norway.

In particular, we should do more to encourage innovation that attacks the climate change problem. It is conceivable that we will manage to solve the problem anyway, courtesy of dramatic progress in the cost of solar power and battery storage. If so, that is luck that we have done precious little to earn. The most obvious first step (among several worth trying) is a stiff tax on carbon dioxide emissions. That would encourage everything from clean energy to putting on a thermal vest in the cold.

There is still every reason to believe that material progress is consistent with the survival of the ecosystem. Human ingenuity is astonishing. It would be nice if policymakers tried harder to direct it toward low-carbon energy.

If policymakers matched climate change talk with action, my guess — just a guess — is that we would find that the transition to a vastly cleaner economy is smooth. I realise that my friends mean well when they demand that economic growth must stop, and soon. But I am pretty sure that they are wrong — and that their pessimism merely convinces others to do nothing.

Written for and first published in the Financial Times on 12 October 2018.

My book “Fifty Things That Made the Modern Economy” (UK) / “Fifty Inventions That Shaped The Modern Economy” (US) is out now in paperback, and coincidentally the final, triumphant chapter is all about Bill Nordhaus’s work on the cost of light. Feel free to order online or through your local bookshop.

Free email updates

(You can unsubscribe at any time)

Exploitative algorithms are using tricks as old as haggling at the bazaar

A few years ago I received a text message from my mobile phone network informing me of the good news that I was already on the cheapest possible tariff. They should have let sleeping dogs lie: I called their bluff, and within minutes they offered a cheaper one. Another time, I quit only to receive phone calls pleading for forgiveness and offering me an iPad if only I’d come back. It was like being in an emotionally abusive relationship with Santa Claus.

Nobody wants to feel that they are being taken for a fool. It is hardly surprising, then, that the UK business secretary Greg Clark has made some noise about his plans to scrutinise how firms may use big data or other digital tools to produce “abusive outcomes” such as exploiting loyal customers.

Another review is under way courtesy of the UK’s Civil Aviation Authority into how budget airlines “use algorithms” to seat families separately if they don’t pay extra for assigned seats. You and I don’t want to be on the sharp end of an exploitative algorithm, do we?

There seems no harm in having a good hard think about how well competition works in a data-rich age. Customers have new tools at their disposal to find the best deals; companies, in response, can pick off lucrative customers like stray wildebeest, offering hidden discounts to some, even targeting adverts and offers by sex or race.

Yet the striking thing about the concerns of consumer champions is that for all the digital window-dressing, this struggle is as old as haggling at the bazaar.

When companies take time and trouble to make their wares less attractive, we call this “product sabotage”. Printers may come in a high-cost professional version and a lower-cost home version with a chip to slow it down. “Value” supermarket pasta or rice is packaged to look like famine relief. And airlines may split families who do not pay extra — a practice that hardly requires a mysterious “algorithm”.

In each case, the company is seeking a premium from premium customers while grasping for volume by offering low prices to the masses. In order to achieve both goals, it may need to damage the mass-market offering. If the cheap product is insufficiently dreadful, the risk is that even wealthy customers may buy it.

The economist Jules-Emile Dupuit spotted an example in 19th-century France. “It is not because of the few thousand francs which would have to be spent to put a roof over the third-class carriage . . . that some company or other has open carriages,” he wrote of the railways. “What the company is trying to do is prevent the passengers who can pay the second-class fee from travelling third class; it hits the poor, not because it wants to hurt them, but to frighten the rich.”

The problem, then, is more than 150 years old. And it is not clear that the situation would be improved by insisting on equal treatment for all passengers. The railway company (or airline) might then offer only the first-class service at the first-class prices, perhaps even higher.

This is aggravating, no doubt. But the root of the problem is that the company has some market power, which allows it to squeeze customers and raise prices. The product sabotage is the symptom — and not necessarily a harmful one.

What of the idea that loyal customers are exploited rather than rewarded? That was my experience with the phone companies, but the infuriating practice is, again, not new. Every time I shave I can praise King Camp Gillette for inventing the disposable razor blade, and curse him for embracing the pricing model of cheap razor, expensive blades. What is that, if not a penalty for loyalty?

In truth the word “loyalty” leads us astray here. Any profit-seeking company will want to exploit customers who never walk away, so considerable effort is devoted both to identifying those customers and to inducing them not to look elsewhere.

“Loyalty cards”, whether an airline gold card or a rubber-stamped bit of cardboard from your local espresso bar, are designed to persuade high-volume, high-value customers both to identify themselves and to stick around. The result is a less competitive market in which everyone pays a higher price.

It is possible that in the initial scramble to sign up new customers, companies reward them so lavishly as to compensate them in advance for years or decades of locked-in high prices. But it’s not likely.

Who loses out from such behaviour? Understandably, we worry about “vulnerable” consumers. But for the companies, their target is clear: they will try to price-gouge the customers most likely to pay. Often, those customers will be rich and busy, while the ones who enjoy the bargains will be poorer and have more time to shop around. That is no calamity. When the victim is a lonely octogenarian in the early stages of dementia, the cat-and-mouse game between producer and consumer takes on a cruel and tragic edge.

Regulators are right to be vigilant. Still — the very fact that such tricks are as old as commerce itself suggest that we will not succeed in stamping them out. Buyer, beware.

Written for and first published in the Financial Times on 5 October 2018.

My book “Fifty Things That Made the Modern Economy” (UK) / “Fifty Inventions That Shaped The Modern Economy” (US) is out now in paperback – feel free to order online or through your local bookshop.

Free email updates

(You can unsubscribe at any time)


  • 1 Twitter
  • 3 RSS
  • 5 Podcasts
  • 6 Facebook


  • Fifty Inventions That Shaped the Modern Economy
  • Messy
  • The Undercover Economist Strikes Back
  • Adapt
  • Dear Undercover Economist
  • The Logic of Life
  • The Undercover Economist

Free Email Updates

Enter your email address to receive notifications of new articles by email (you can unsubscribe at any time).

Join 188,297 other subscribers.