Tim Harford The Undercover Economist

Undercover EconomistUndercover Economist

My weekly column in the Financial Times on Saturdays, explaining the economic ideas around us every day. This column was inspired by my book and began in 2005.

Undercover Economist

Why prize-winners are heading for a fall

With hindsight, the timing was awkward. PRWeek had barely engraved Oscar Munoz’s name on the trophy before the chief executive of United Airlines and Communicator of the Year 2017 was engulfed in a spectacular public relations crisis. The airline had summoned the police to throw David Dao out of the seat he had paid for; the police broke his nose and knocked out two teeth; and Mr Munoz’s first response was to criticise Mr Dao and apologise to the other passengers. At least the PRWeek award has a glorious future as the answer to a pub quiz question.

I’m not even sure this qualifies as the award that turned out to seem the most ridiculous. The American Institute of Architects honoured the Kemper Arena in Kansas City with a national honour award, and then held its annual convention there in 1979. Alas, the roof of the arena collapsed a few hours after the architects’ convention left the site.

Tempting as it may be to mock the judges who hand out such prizes, having a more objective benchmark for achievement does not confer immunity. Just ask Claudio Ranieri, the football manager who masterminded Leicester City’s underdog triumph in the English Premier League last year. As soon as the points dried up, Mr Ranieri was sacked.

Why does life seem to deliver such strange reversals? The first explanation is simple but easy to miss: there are a lot of awards in the world. Many of the people or organisations notable enough to make news when they screw up will also be notable enough to have won a prize that will prove embarrassing.

The second explanation is more subtle: for statistical reasons, outstanding performances tend to be followed by something less impressive. This is because most performances involve some randomness. On any given day, the worst observed outcomes will be incompetents having an unlucky day and the best observed outcomes will be stars having a lucky day. Observe the same group on another day and, because luck rarely lasts, the former outliers will not be quite as bad, or as good, as at first they seemed. This phenomenon is called “regression to the mean”.

If you place a speed camera beside a stretch of road where the accident rate has been exceptionally high, the accident rate is likely to fall. But a garden gnome in the same place will also seem to save lives. That is because exceptionally high accident rates are partly the result of bad luck. While the speed camera may encourage safer driving, unlucky driving will tend to disappear whether it is treated with a camera or with a gnome.

Daniel Kahneman, psychologist and winner of the Nobel memorial prize in economics, was advising the Israeli air force when he noted a memorable example of how regression to the mean can mislead us. A flight instructor told Mr Kahneman that when the instructor praised cadets for a skilful landing, they usually did worse next time; when he bawled them out for clumsiness they tended to improve. The instructor concluded that harsh criticism worked; Mr Kahneman pointed out that a more likely explanation was sheer chance.

“Because there is regression to the mean, it is part of the human condition that we are statistically punished for rewarding others and rewarded for punishing them,” Mr Kahneman wrote. PRWeek now knows what he meant.

Regression to the mean probably explains why many award winners subsequently disappoint. And the disappointment will be spectacular if some people are taking bigger risks than others. The most impressive performance may combine skill with luck. In a financial market — or a casino — the easiest way to become an outlier is to make a big bet. Unfortunately, there is no way to be sure whether you will be an outlier on the upside or the downside. Treading a different path is a good way to look spectacularly right, or spectacularly wrong — or, given enough time, both.

While randomness can explain much, hubris may also play a role. A few years ago, the economists Ulrike Malmendier and Geoffrey Tate examined what happened to companies whose chief executives won accolades such as Forbes’s “Best Performing CEO” or BusinessWeek’s “Best Manager”. Ms Malmendier and Mr Tate picked a statistical control group of nearly-men and nearly-women who might have been expected to win an award, but did not.

Like the near-winners, the winners ran large, profitable companies. But those run by the winners did far worse in the three years following the award, lagging behind the near-winners by about 20 per cent. The prizewinning CEOs nevertheless enjoyed millions of dollars more in pay. They were also more likely to write books, accept seats on other corporate boards and improve their golf handicap.

Winning a prize may strengthen the hands of already dominant CEOs, enabling them to extract more money from shareholders and distracting them with the opportunity to write self-congratulatory books. Right now, I doubt that anyone is rushing to offer Mr Munoz an advance for his management insights.

Written for and first published in the Financial Times on 21 April 2017.

My new book is “Fifty Things That Made The Modern Economy” – coming soon! If you want to get ahead of the curve you can pre-order in the US (slightly different title) or in the UK or through your local bookshop.

Free email updates

(You can unsubscribe at any time)

Undercover Economist

Life is getting ever more volatile – or is it?

We live in an age of unrelenting change. That, at least, is what we are told by a consulting industry that thrives on a gospel of disruption, and journalists who overgeneralise from the earthquake in their own profession.

Anecdotal evidence of volatility is easy to find: the financial crisis; the cultural dominance of inventions such as Facebook (barely a teenager) and the iPhone (younger still); the rise of fringe political parties and a maverick president.

But the statistical evidence for disruption is less compelling. The most straightforward evidence of that is low productivity growth in many advanced economies. If the pace of change is really so frenetic, how come we don’t see it in the productivity statistics?

One obvious measure of volatility would be the frequency with which people change jobs. When I entered the UK labour market in the late 1990s, the received wisdom was that a job for life was a thing of the past. Yet typical job tenure is longer today than it was then. People are choosing to move between jobs less frequently.

This is true both in the UK and in the US. In a 2015 study of the UK labour market for the Resolution Foundation, a think-tank, Paul Gregg and Laura Gardiner found a striking fall in the tendency of people to move from one job to another. In the 1990s and early 2000s, about 3-3.5 per cent of people with jobs would leave them each quarter in favour of new jobs. During the financial crisis that rate fell to well below 2 per cent. It has rebounded since, but is still far below historical norms. The decline was particularly pronounced for the under-thirties.

Why does this matter, if people have jobs? It matters because, as Mr Gregg and Ms Gardiner point out, moving from one job to another often means a promotion and a substantial pay rise. When people — especially young people — find themselves in the same job for year after year, there comes a time when we no longer call this stability. We call it stagnation.

In the US, there is a similar story to be told — a long-term decline in job-to-job transition rates that cannot simply be explained away by a change in the demographics of the workforce.

A parallel trend in the US is the decline in geographical mobility. The percentage of Americans moving from one state to another in a particular year has fallen by about half since 1990 — a striking fall, and one that seems surprising given the standard narratives of a scattered and atomised society. If it’s really true that Americans don’t know their neighbours any more, then they must be working hard to avoid them, because they have those neighbours for far longer than once they did.

Perhaps we fear change more than ever. Tyler Cowen’s new book The Complacent Class (UK) (US) argues that America has become less adventurous in many ways. An album from two decades ago would sound just fine to modern ears; films wrap new special effects around well-worn plots and characters. The chief expression of our cultural courage now is to eat at a trendy new restaurant. Ethiopian or Peruvian cuisine may offer delicious fresh flavours but it is hardly a force for cultural revolution.

Mr Cowen’s argument is fascinating. But one need not invoke culture to explain the stasis in the job market. A 2016 paper from economists Mike Konczal and Marshall Steinbaum argued that the main reason people don’t quit or move to find good jobs is because there are fewer good jobs available. If companies were hungry for talent, but people were reluctant to move, wages should be soaring in a scramble to persuade them. In most industries, they aren’t.

A more upbeat account, at least superficially, comes from economists Greg Kaplan and Sam Schulhofer-Wohl. They point out that, thanks to the internet, people can find the perfect job in the perfect place and never need to move again. (The same logic implies that dating apps reduce the number of fruitless first dates. I am not sure anybody believes that.)

But Messrs Kaplan and Schulhofer-Wohl also note that economies are now more homogenous than once they were. When once people would move to Michigan to build cars, or to coastal ports to work on the docks, now every job is a service-sector job. Whether you flip burgers or perform brain surgery, you can work in any state, so there is no strong need to move.

This is plausible, but it is also sad. It suggests that given the ability to move anywhere, we stay put. I am reminded of a study of college friendships conducted by psychologists Angela Bahns, Kate Pickett and Christian Crandall. They found that students in a large, diverse campus sought out and befriended other students very much like themselves. In smaller universities with fewer friendship options, young people had more varied groups of friends because the alternative was to have no friends at all. Our bias towards the status quo is not new — but perhaps we are taking advantage of new opportunities to indulge it.

Written for and first published in the Financial Times on 13 April 2017.

My book “Messy” is available online in the US and UK or in good bookshops everywhere.

Free email updates

(You can unsubscribe at any time)

Undercover Economist

Why we should be grateful to immigrants

In 1929, the great economist Irving Fisher found his fortune and reputation ruined by his utter failure to forecast the great crash. Rival forecaster Roger Babson commented, not without sympathy, that Fisher’s problem was that “he thinks the world is ruled by figures instead of feelings”.

What a pitiable error. And it’s a warning to economists like me. We like to believe that the numbers speak for themselves, but the political world prefers to think with its gut. Nowhere is this more true than in the case of immigration, which economists tend to conclude is broadly a positive force. Politicians, aware that the world is ruled by feelings, tend to view immigration as something closer to a series of prison breaks: if you can’t get a grip on it, apologise and resign.

Immigration was a centrepiece of Donald Trump’s “build the wall” election campaign. It is a stated reason that the British government intends to go well beyond the referendum result and leave not only the EU but the single market. Immigration inspires strong feelings, and those feelings aren’t of happiness and gratitude. That is a shame.

Is there a gut-based case that we should be grateful to immigrants? I’d like to think so. Perhaps we should start with the golden rule of “do unto others”. It has a clarifying effect on our thinking about immigrants — or “expats”, as the golden rule suggests we call them, since that is what we call ourselves if we go to live overseas. The words are synonyms but the difference in perspective and respect is enormous.

The golden rule quickly exposes much of the talk about immigrants — expats — as hypocrisy. One popular idea is that immigration should be based on some kind of “points” system — or as the White House describes it, “merit-based” immigration. It seems reasonable. But not for a moment would we think of telling someone they couldn’t move from Detroit to Dallas because they “lacked merit”. Vox columnist Matthew Yglesias observes that he wouldn’t get away with describing white Americans without college degrees as people “without merit”. Quite.

Another seductive folly is the idea that the authorities should in their bureaucratic wisdom decide how easy it would be for different industries to recruit workers. As migration expert Madeleine Sumption has recently pointed out, an attempt to fine-tune the labour market through immigration policy might seem attractive, but it would require an extensive and expensive bureaucracy that would swiftly be surrounded by eager lobbyists. We need only ask how we’d feel if such a policy applied to us, with a Whitehall department deciding whether Cambridge required another software engineer.

We might also think that concerns about the “brain drain” were not a persuasive reason to force people from Leicester to stay there on the grounds that Leicester needs their skills. All these policies have managed to pass as moderate, sensible and even compassionate when applied to foreigners. The instant we ask if we’d apply them to ourselves we see them for what they are: a ludicrously cumbersome attempt at economic planning, and a woefully illiberal way to treat human beings.

There are many analyses of the costs and benefits of immigration. What’s not widely appreciated is that most of them simply ignore any benefits to the migrants — expats — themselves. Given this handicap, it’s striking that many serious studies find some modest net economic benefits. If I told you that a school or a hospital could pass a cost-benefit test even after ignoring the benefits to the pupils or patients, you might reasonably conclude that the school and hospital were impressive organisations. You’d also tell me it was a very strange way to do cost-benefit analysis.

In 18th-century France, workers had to show their papers to get permission to move from one town to another. The objection wasn’t that immigrants might arrive. It was that valuable workers might leave.

The French nobles were on to something. In most circumstances we’re keen to live close to other people. Densely populated areas tend to be richer, more productive, and more innovative. Because housing is compact and people travel on public transport, cities are also more environmentally friendly. And we know that cities are desirable places to live because people are willing to pay so dearly to live in them. Usually we view other people as customers, colleagues, and friends. From the church to the high street to the nightclub, other people are the lifeblood of our communities. It is only when we call other people “immigrants” that they seem to cause such anxiety.

I think we should be more grateful to the people who have the courage and energy to leave their homes and make a life somewhere new. But perhaps I’m the one who should be more grateful. As the economist Paul Seabright observes in his 2004 book The Company of Strangers (UK) (US), we humans are the recent descendants of shy, murderous apes. Somehow we have figured out a way to live together and co-operate. We have some way to go, but I am grateful for the progress we have made so far.
First published in the Financial Times on 7 April 2017.

My book “Messy” is available online in the US and UK or in good bookshops everywhere.

Free email updates

(You can unsubscribe at any time)

5th of May, 2017Undercover EconomistComments off
Undercover Economist

Marginal gains matter but gamechangers transform

“As Olympic athletes have shown, marginal improvements accumulated over time can deliver world-beating performance,” said Andrew Haldane in a speech on Monday, which is quite true. Mr Haldane, the Bank of England’s chief economist, went on to suggest that the productivity of companies in general would benefit from the same approach.

It’s a bold claim, made more so by the regrettable fact that the most celebrated exponents of “marginal gains”, Team Sky and British Cycling, have been in the headlines for the wrong reasons.

But Mr Haldane may have a point; he usually does. The marginal gains philosophy tries to turn innovation into a predictable process: tweak your activities, gather data, embrace what works and repeat. In British cycling such tweaks reportedly include rubbing alcohol on tyres to improve grip, electrically heated overshorts to maintain muscle temperature and a ban on bikini waxes to prevent saddle sores.

But the same basic approach — using quick-and-dirty experiments, or “A/B testing” — has paid dividends elsewhere. David Cameron’s Behavioural Insight Team, known unofficially as the “nudge unit”, has used simple randomised trials to improve the wording of tax demands and the advice given to job seekers. Google tested 41 shades of blue for its advertising hyperlinks. Designers rolled their eyes — then Google claimed that the experiment had netted an extra $200m in annual revenue. As Mr Haldane says, marginal improvements can add up.

But can they add up to productivity gains for the economy as a whole? The question matters. There is no economic topic more important than productivity, which in the long run determines whether living standards surge or stagnate. Productivity growth has been disappointing for more than 40 years, particularly disastrous since the financial crisis, and worse in the UK than in most other rich nations. The idea that developed economies can A/B test their way back to brisk productivity growth is a seductive one.

An alternative view is that what’s really lacking is a different kind of innovation: the long shot. Unlike marginal gains, long shots usually fail, but can pay off spectacularly enough to overlook 100 failures. The marginal gain is a heated pair of overshorts, the long shot is the Fosbury Flop. If the marginal gain is a text message nudging you to finish a course of antibiotics, the long shot is the development of penicillin. Marginal gains give us zippier web pages; long shots gave us the internet.

These two types of innovation complement each other. Long shot innovations open up new territories; marginal improvements colonise them. The 1870s saw revolutionary breakthroughs in electricity generation and distribution but the dynamo didn’t make much impact on productivity until the 1920s. To take advantage of electric motors, manufacturers needed to rework production lines, redesign factories and retrain workers. Without these marginal improvements the technological breakthrough was of little use.

The latest evidence suggests Mr Haldane is right to remind us of the importance of marginal gains. In 2016 economists Daniel Garcia-Macia, Chang-Tai Hsieh and Peter Klenow sought to quantify different kinds of innovation in the US economy. They concluded that most productivity growth came from existing companies improving existing products, rather than new businesses or products. That was true in the 1980s and still is. In the UK, as Mr Haldane observes, leading companies have been improving their productivity, while typical ones have not. That suggests a problem not with the long shots at the frontier of innovation but with the details of everyday management.

Yet two questions remain. One is why so many businesses lag far behind the frontier. A research programme led by economists Nicholas Bloom and John Van Reenen has tried to quantify management practices, and has found that many countries have a long tail of poorly managed companies. The culprit may be a lack of competition: vigorous competition tends to raise management quality by spurring improvements and by punishing incompetents with bankruptcy. It’s no coincidence that the philosophy of marginal gains is popular in the unforgiving arena of elite sport.

But the second question is why productivity growth has been so disappointing. A/B testing has never been easier or more fashionable, after all.

The obvious answer is that the long shots matter, too. In almost every field except computing, we’ve hoped for revolutionary breakthroughs and they haven’t yet happened. Google may A/B test its way to greater profits but the company’s success has been built on a leap forward in search technology in the late 1990s, and even more fundamentally on the publicly funded efforts to develop the web and the internet.

In a data-driven world, it’s easy to fall back on a strategy of looking for marginal gains alone, avoiding the risky, unquantifiable research. Over time, the marginal gains will surely materialise. I’m not so sure that the long shots will take care of themselves.

Written for and first published in the Financial Times on 24 March 2017.

My book “Messy” is available online in the US and UK or in good bookshops everywhere.

Free email updates

(You can unsubscribe at any time)

Undercover Economist

Some things are best left to the technocrats

In case anyone needed reminding of the majestic power of democracy, Boaty McBoatface embarked upon its first mission last week. The robotic yellow submarine will explore the depths of the seas surrounding Antarctica. McBoatface may even encounter Mister Splashy Pants, a humpback whale that frequents the South Pacific.

Both names were chosen by popular vote and each, in different ways, was resisted by the organisations that decided to ask the public in the first place. McBoatface and Splashy Pants remind us that, while democracy is all very well, there’s a risk in asking the public what they think.

For all its merits, democracy has always had a weakness: on any detailed piece of policy, the typical voter — I include myself here — does not understand what is really at stake and does not care to find out. This is not a slight on voters. It is a recognition of our common sense. Why should we devote hours to studying every policy question that arises? We know the vote of any particular citizen is never decisive. It would be a deluded voter indeed who stayed up all night revising for an election, believing that her vote would be the one to make all the difference.

So voters are not paying close attention to the details. That might seem a fatal flaw in democracy but democracy has coped. The workaround for voter ignorance is to delegate the details to expert technocrats. Technocracy is unfashionable these days; that is a shame.

One advantage of a technocracy is that it constrains politicians who are tempted by narrow or fleeting advantages. Multilateral bodies such as the World Trade Organization and the European Commission have been able to head off popular yet self-harming behaviour, such as handing state protection to which ever business has the best lobbyists.

Meanwhile independent central banks have been the grown-ups of economic policymaking. Once the immediate aftermath of the financial crisis had passed, elected politicians sat on their hands. Technocratic central bankers were — to borrow a phrase from Mohamed El-Erian, economic adviser — “the only game in town” in sustaining a recovery.

A second advantage is that technocrats can offer informed, impartial analysis. Consider the Congressional Budget Office in the US, the Office for Budget Responsibility in the UK, and Nice, the National Institute for Health and Care Excellence.

Technocrats make mistakes, it’s true — many mistakes. Brain surgeons also make mistakes. That does not mean I’d be better off handing the scalpel to Boris Johnson. Better a flawed expert than a flawed amateur. Democratically elected politicians are not well placed to do technical work and neither are voters. Where democracy is not up to the job, we turn to technocracy instead.

Ultimately, democracy must trump technocracy, and it does. The Bank of England has control of monetary policy but it must aim at a target set by the chancellor, who must answer to the electorate. The chancellor, meanwhile, controls fiscal policy directly. The indirect system is working better. The monetary policy of successive chancellors has been consistent for 20 years. The fiscal policy of the incumbent Philip Hammond — raising taxes on the self-employed — did not survive a week.

But there’s a problem. Technocrats may not be too interested in politics, but politics is interested in technocrats. Successive chairs of the US Federal Reserve have been accused of treason, and (worse) being “more political than Hillary Clinton” by senior Republicans Rick Perry and Donald Trump, respectively. There is now a real sense that its independence may be under threat.

In the UK, too, Brexiters have noticed that most economic experts believe Brexit will damage the economy. Many have responded not with evidence but with smears and insults — most memorably Michael Gove’s comparison of academic economists with Nazi scientists.

The dragging of technocratic advice into the political arena has in it the makings of tragedy. I realise that sounds grandiose, but the stakes here are high. On almost any issue, logic and evidence can be eaten away once partisan polarisation takes hold.

Any democracy must debate the big political issues: how much we protect the vulnerable, the appropriate size of the state, the importance of individual freedom. But technical matters are different. How safe is the MMR vaccine? Are humans changing the climate? Does fiscal stimulus work with interest rates at the zero lower bound? Once these questions become chew toys for political attack dogs, there’s no easy way out.

It is better to keep such topics away from politics as much as possible. Complex problems cannot just be wished away. Reality cannot be fooled.

“Nobody knew that healthcare could be so complicated,” said President Donald Trump last month, while David Davis, the UK’s Brexit minister, has admitted that his department had not analysed the implications of leaving the EU without a deal.

Democratic policymaking has entered the era of Boaty McBoatface.

Written for and first published in the Financial Times.

My book “Messy” is available online in the US and UK or in good bookshops everywhere.

Free email updates

(You can unsubscribe at any time)

Undercover Economist

Society and the profiteroles paradox

Ken is in a restaurant, pondering his choice of dessert. Ice cream, profiteroles or a cheese plate? He’s about to request a scoop of ice cream when the waiter informs him that the profiteroles are off the menu. “I see,” says Ken. “Well, I’ll have the cheese, please.”

Ken’s behaviour is odd enough to be a piece of surrealist comedy. But what seems ludicrous from an individual is easy to imagine in an election. Think of George W Bush as ice cream, Ralph Nader as profiteroles and Al Gore as the cheese plate. If Nader had not been on the menu in the 2000 US presidential election, then Gore would have been president instead of Bush. Since Nader himself was never a serious contender, it seems odd that his presence changed the result. But we’ve grown used to this sort of thing in politics.

Still, we might ask: is there a way to assemble individual preferences into social preferences without generating surreal outcomes? That was the first of many big problems studied by the great economist Kenneth Arrow, who died last month at the age of 95. His answer: no.

To understand Arrow’s answer, imagine a society in which every individual has a ranking expressing their preferences over every possible outcome. Let’s say that we can read minds, so we know what each person’s ranking is. All we need is some system for combining those individual rankings into a social ranking that tells us what society as a whole prefers.

Arrow named this putative system a constitution. What properties would we like our constitution to have? It should be comprehensive, giving us an answer no matter what the individual rankings might be. And it wouldn’t fall prey to the profiteroles paradox: if society prefers ice cream to cheese, then whether profiteroles are available or not shouldn’t change that fact.

We want the constitution to reflect people’s preferences in common-sense ways. If everyone expresses the same preference, for example, the constitution should reflect that. And we shouldn’t have a dictator — an individual who is a kind of swing voter, where the constitution reflects only her preferences and ignores everyone else.

None of these properties seem particularly stringent — which makes Arrow’s discovery all the more striking. Arrow’s “impossibility theorem” proves that no constitution can satisfy all of them. Any comprehensive constitution will suffer the profiteroles paradox, or arbitrarily ignore individual preferences, or will simply install a dictator. How can this be?

Let me now attempt the nerdiest move in more than 11 years of writing this column. Since there’s no idea in economics more beautiful than Arrow’s impossibility theorem, I’m going to try to outline a proof for you — very sketchily, but you may get the idea.

Imagine that our constitution must deliver a choice between ice cream, profiteroles and cheese. Step one in the proof is to note that there must be a group whose preferences determine whether society as a whole prefers cheese or ice cream — if only because the constitution must respect a unanimous view on this. Call this group the Cheese Group. The Cheese Group might include everyone in society, but maybe it’s a smaller group of swing voters.

The next step is to show that the Cheese Group doesn’t merely swing the decision between cheese and ice cream, but also over profiteroles and any other dessert we might add to the menu. We can show this by creating cases where it’s impossible for the Cheese Group to express a preference between cheese and ice cream without profiteroles being caught in the middle. This means that the Cheese Group actually gets to decide about everything, not just cheese and ice cream.

Finally, having established that the Cheese Group is all-powerful, we show that we can make it smaller without destroying its power. Specifically, we can keep dividing it into pairs of sub-groups, and show that at each division either one of the sub-groups is all-powerful, or the other one is.

In short: we prove that if any group of voters gets to decide one thing, that group gets to decide everything, and we prove that any group of decisive voters can be pared down until there’s only one person in it. That person is the dictator. Our perfection constitution is in tatters.

That’s Arrow’s impossibility theorem. But what does it really tell us? One lesson is to abandon the search for a perfect voting system. Another is to question his requirements for a good constitution, and to look for alternatives. For example, we could have a system that allows people to register the strength of their feeling. What about the person who has a mild preference for profiteroles over ice cream but who loathes cheese? In Arrow’s constitution there’s no room for strong or weak desires, only for a ranking of outcomes. Maybe that’s the problem.

Arrow’s impossibility theorem is usually described as being about the flaws in voting systems. But there’s a deeper lesson under its surface. Voting systems are supposed to reveal what societies really want. But can a society really want anything coherent at all? Arrow’s theorem drives a stake through the heart of the very idea. People might have coherent preferences, but societies cannot. We will always find ourselves choosing ice cream, then switching to cheese because the profiteroles are off.

Written for and first published in the Financial Times.

My book “Messy” is available online in the US and UK or in good bookshops everywhere.

Free email updates

(You can unsubscribe at any time)

Undercover Economist

Has Facebook ruined the news?

“Our goal is to build the perfect personalised newspaper for every person in the world,” said Facebook’s Mark Zuckerberg in 2014. This newspaper would “show you the stuff that’s going to be most interesting to you”.

To many, that statement explains perfectly why Facebook is such a terrible source of news. A “fake news” story proclaiming that Pope Francis had endorsed Donald Trump was, according to an analysis from BuzzFeed, the single most successful item of news on Facebook in the three months before the US election. If that’s what the site’s algorithms decide is interesting, it’s far from being a “perfect newspaper”.

It’s no wonder that Zuckerberg found himself on the back foot after Trump’s election. Shortly after his victory, Zuckerberg declared: “I think the idea that fake news on Facebook, which is a very small amount of the content, influenced the election in any way . . . is a pretty crazy idea.” His comment was greeted with a scornful response.

I should confess my own biases here. I despise Facebook for all the reasons people usually despise Facebook (privacy, market power, distraction, fake-smile social interactions and the rest). And, as a loyal FT columnist, I need hardly point out that the perfect newspaper is the one you’re reading right now.

But, despite this, I’m going to stand up for Zuckerberg, who recently posted a 5,700-word essay defending social media. What he says in the essay feels like it must be wrong. But the data suggest that he’s right. Fake news can stoke isolated incidents of hatred and violence. But neither fake news nor the algorithmically driven “filter bubble” is a major force in the overall media landscape. Not yet.

“Fake news” is a phrase that has already been debased. A useful definition is that fake news is an entirely fabricated report presenting itself as a news story. This excludes biased reporting, satire and lies from politicians themselves.

At first glance, such hoaxes appear to be ubiquitous on Facebook. The BuzzFeed analysis finds that the five most popular hoax stories were more successful than the five most popular true stories. (This list of true stories includes the New York Post’s “Melania Trump’s Girl-on-Girl Photos From Racy Shoot Revealed”, a reminder that not all mainstream journalism is likely to win a Pulitzer.)

But hoax stories are less significant than this analysis suggests — partly because Facebook is not the main source of news for Americans (that’s still television news), and partly because true reports will generally be covered in some form by dozens of outlets, which will dilute the popularity of any one version. Each hoax, however, is unique. No wonder the most popular hoaxes outperform the most popular true reports.

In January 2017, two economists, Hunt Allcott and Matthew Gentzkow, published research studying exactly how prevalent fake news had been before the election. Their clever method tested people’s recall of fake news, as compared with true news stories and “placebo” stories — fake fake news, invented by the researchers. People didn’t remember many fake news stories, and claimed to remember quite a few placebos. Overall, there just didn’t seem to be enough fake news to swing the election result — unless it was potent stuff indeed, even in small doses.

“The average voter saw one fake news story before the election,” Gentzkow told me. “That number is a very different picture from what you might get from watching the public discussion.”

Of more concern is that Facebook — and its “most interesting to you” algorithm — simply supplies news that panders to each user’s ideological biases. It’s undoubtedly true that we surround ourselves with people who agree with us on social media. But it’s not clear that Facebook’s algorithm is the biggest problem here. Twitter was politically polarised even in the days when it used no algorithm at all. And newspapers have ideological biases too.

One recent study of online news reading was conducted by Seth Flaxman, Sharad Goel and Justin Rao, who had access to browser data from Microsoft, and used it to examine how people consumed news online. They found a mixed picture: social media did seem to push stories that were further from the centre of the political spectrum but they also exposed people to a greater variety of ideological viewpoints. That makes sense. Reading the same newspaper every day is a filter bubble too.

Gentzkow studied the contrast between online and offline news using data from 2004-2009, working with fellow economist Jesse Shapiro. They found little evidence then that online news consumption was more polarised than traditional media. But things are changing quickly. “My guess is that segregation is noticeably and meaningfully higher than in the past,” Gentzkow says, “but still quite modest.”

This feels like an important moment. Fake news is not prevalent, but it could become so. Filter bubbles are probably no worse than they have been for decades — but that could change rapidly too.

“A lot ultimately hinges on what the motivations of American voters are,” says Gentzkow. “Do people actually care at all about getting the truth and having accurate information?”

He’s hopeful that, deep down, people watch and read the news because they want to learn about the world. But if what voters really want is to be lied to, then Facebook is the least of our problems.

Written for and first published in the Financial Times.

My book “Messy” is available online in the US and UK or in good bookshops everywhere.

Free email updates

(You can unsubscribe at any time)

Undercover Economist

What raspberry farms can teach us about inequality

Raspberries are a petit-bourgeois crop, while wheat is a proletarian crop — or so says political scientist James C Scott in his remarkable 1998 book Seeing Like a State (UK) (US). That makes it sound as though Scott is musing on matters of taste. In fact, he’s highlighting the link between what we produce, and the political and economic structures that production makes possible. Wheat is a proletarian crop, says Scott, because it works well on industrial farms. Harvesting can be mechanised. Not so easy with raspberries, which are best cared for on a small farm. They are difficult to grow and pick on an industrial scale.

Such distinctions once mattered a great deal. We associate the invention of agriculture with the rise of ancient states but, as Scott points out in a forthcoming book, Against the Grain, much depends on the crop. Wheat is well-suited to supporting state armies and tax inspectors: it is harvested at a predictable time and can be stored — or confiscated. Cassava works differently. It can be left in the ground and dug up when needed. If some distant king wanted to tax the cassava crop, his armies would have had to find them and dig them up one by one. Agriculture made strong states possible, but it was always agriculture based on grain. “History records no cassava states,” he writes.

The technologies we use have always affected who gets what, from the invention of the plough to the creation of YouTube. Economists know this but our analytical tools are not well-suited to distinguishing wheat from raspberries or cassava. The brilliance of gross domestic product is the way it manages to measure all economic activity with the same yardstick — but that is also, of course, its weakness. Nevertheless, we try. Many researchers have examined whether countries with rich endowments of mineral resources — oil, copper, diamonds — tend to do better or worse as a result. The balance of opinion is that there’s a “resource curse”. Why?

Sometimes the problem is obvious enough — for example, natural resources sustained a quarter-century of civil war in Angola, where the government could fund itself with oil while the rebels mined and sold diamonds. Sometimes it’s more subtle: a country that exports a valuable commodity will experience a strengthening of its exchange rate. This makes it harder to sustain any sort of industry that isn’t connected to the commodity itself.

Still, we’ve lacked the statistical tools to paint a compelling picture of these issues, important though they seem to be. Now a new research paper from a team at the Massachusetts Institute of Technology tries to explore how the mixture of products a country produces might influence a critical economic outcome: income inequality. The team includes César Hidalgo, author of Why Information Grows (UK) (US), about whose work I’ve written several times. Over the past few years, Hidalgo has been trying to map what he calls “economic complexity”, using statistical techniques from physics rather than economics.

Complexity isn’t straightforward to measure — is a million dollars of reinsurance more or less complex than a million dollars of liquefied natural gas or a million dollars of computer games? Hidalgo’s method looks at a country’s merchandise exports. Sophisticated economies tend to export many different products, including the most complex. Complex products tend to be exported only by a few economies.

In previous work, Hidalgo and colleagues have shown that economic complexity is correlated with wealth, but there are some economies that are spectacularly sophisticated but only modestly wealthy (South Korea is one) while other economies are very rich but not especially sophisticated (such as Qatar). This new analysis finds a relationship between inequality and lack of economic complexity.

Holding other things constant, the simplest economies tend to be the most unequal; the more sophisticated ones tend to be more equal. It’s raspberries and wheat all over again. Or, if you prefer, the difference between a business such as oil (which employs a few people at high wages), textile work (which generates lots of jobs, but at low wages) and making precision components (which requires many skilled and well-paid workers). The oil-based economy will tend to be the most unequal, while the precision-engineering economy will tend to be the most equal.

There are exceptions: Australia’s economy is surprisingly simple thanks to a dependence on natural resources, but not especially unequal. Mexico is an outlier in the other direction, with a sophisticated but unequal economy. This research answers some questions and raises others. There’s a large and unsatisfying literature on the relationship between inequality and growth. Are unequal societies dynamic and entrepreneurial or dysfunctional patron-client states? The MIT study suggests that what’s been missing from these questions is a measure of economic complexity.

And what about financial services? They seem both sophisticated and highly unequal — an exception to the rule? Hidalgo’s data are silent on the topic. But Hidalgo himself isn’t persuaded that banking is particularly complex.

“Most countries have financial services,” he tells me. “But few countries know how to design new microprocessors or new medicines.” By that measure, and others, he thinks financial services are cruder than we tend to think. Perhaps. If so, the City of London has more in common with the oilfields of the North Sea than we are inclined to admit.
Written for and first published in the Financial Times.

My book “Messy” is available online in the US and UK or in good bookshops everywhere.

Free email updates

(You can unsubscribe at any time)

Undercover Economist

How to catch a cheat

Should the rules and targets we set up be precise, clear and sophisticated? Or should they be vague, ambiguous and crude? I used to think that the answer was obvious — who would favour ambiguity over clarity? Now I am not so sure.

Ponder the scandal that engulfed Volkswagen in late 2015, when it emerged that the company had been cheating on US emissions tests. What made such cheating possible was the fact that the tests were absurdly predictable — a series of pre-determined manoeuvres on a treadmill. VW’s vehicles, kitted out with sensors as all modern cars are, were programmed to recognise the choreography of a laboratory test and switch to special testing mode — one that made the engine sluggish and thirsty, but that filtered out pollutants.

The trick was revealed when a non-profit group strapped emissions monitors to VW cars and drove them from San Diego to Seattle. In some ways, that’s a crude test: outside the laboratory, no two journeys can be compared precisely. But the cruder test was also the test that revealed the duplicity.

The VW case seems like a strange one-off. It isn’t. Consider the “stress tests” applied by regulators to large banks. These stress tests are disaster scenarios in which a bank calculates what would happen in particular gloomy situations. But, in 2014, US regulators started to notice that banks had made very specific, narrow bets designed to pay off gloriously in specific stress-test scenarios. There is no commercial logic for these bets — but they certainly make it easier to pass the stress test. VW all over again — with the difference that what the banks were doing was apparently perfectly legal.

If tests and targets can fail because they are too predictable, they can also fail because they are too narrow. A few years ago, UK ambulance services were set a target to respond to life-threatening situations within eight minutes of receiving an emergency call. Managers soon realised that they could hit the target more easily if they replaced a two-person ambulance with an independent pair of paramedics on bikes. And many responses were written down as seven minutes and 59 seconds, but few as eight minutes and one second — suspiciously timely work.

Perhaps we’d be better off handing over the problem to computers. Armed with a large data set, the computer can figure out who deserves to be rewarded or punished. This is a fashionable idea. As Cathy O’Neil describes in her recent book, Weapons of Math Destruction (UK) (US), such data-driven algorithms are being used to identify which prisoners receive parole and which teachers are sacked for incompetence.

These algorithms aren’t transparent — they’re black boxes, immune from direct scrutiny. The advantage of that is that they can be harder to outwit. But that does not necessarily mean they work well. Consider the accuracy of the recommendations that a website such as Amazon serves up. Sometimes these suggestions are pretty good, but not always. At the moment, Amazon is recommending that I buy a brand of baby bottle cleanser. I’ve no idea why, since all my children are of school age.

A teacher-firing algorithm might look at student test scores at the beginning and end of each school year. If the scores stagnate, the teacher is presumed to be responsible. It’s easy to see how such algorithms can backfire. Partly, the data are noisy. In a data set of 300,000, analysts can pinpoint patterns with great confidence. But with a class of 30, a bit of bad luck can cost a teacher his or her job. And perhaps it isn’t bad luck at all: if the previous year’s teacher somehow managed to fix the test results (it happens), then the new teacher will inherit an impossible benchmark from which to improve.

Just like humans, algorithms aren’t perfect. Amazon’s “you might want to buy bottle cleanser” is not a serious error. “You’re fired” might be, which means we need some kind of oversight or appeal process if imperfect algorithms are to make consequential decisions.

Even if an algorithm flawlessly linked a teacher’s actions to the students’ test scores, we should still use it with caution. We rely on teachers to do many things for the students in their class, not just boost their test scores. Rewarding teachers too tightly for test scores encourages them to neglect everything we value but cannot measure.

The economists Oliver Hart and Bengt Holmström have been exploring this sort of territory for decades, and were awarded the 2016 Nobel Memorial Prize in Economics for their pains. But, all too often, politicians, regulators and managers ignore well-established lessons.

In fairness, there often are no simple answers. In the case of VW, transparency was the enemy: regulators should have been vaguer about the emissions test to prevent cheating. But in the case of teachers, more transparency rather than less would help to uncover problems in the teacher evaluation algorithm.

Sometimes algorithms are too simplistic, but on occasions simple rules can work brilliantly. The psychologist Gerd Gigerenzer has assembled a large collection of rules of thumb that perform very well in predicting anything from avalanches to heart attacks. The truth is that the world can be a messy place. When our response is a tidy structure of targets and checkboxes, the problems really begin.

Written for and first published in the Financial Times.

My book “Messy” is available online in the US and UK or in good bookshops everywhere.

Free email updates

(You can unsubscribe at any time)

Undercover Economist

How being wrong can help us get it right

There’s an academic I know — very well respected — who especially values one of his collaborators. This particular colleague isn’t invaluable because of his creativity or intellect, says my professor friend, but because “he is willing to tell me when I’m wrong, and that’s rare”. It is indeed rare. Perhaps even rarer is the practice of seeking out colleagues because they give frank criticism.

I certainly don’t enjoy being told that I’m wrong. And it seems that I’m not alone. A recently published working paper from Paul Green and Francesca Gino of Harvard, and Bradley Staats of the University of North Carolina, caught people in the act of avoiding criticism. The particular kind of criticism that interested the researchers was where I think I’m doing a good job, and then you tell me that I’m not. (In the jargon, this is “disconfirmatory feedback”.)

Green, Gino and Staats looked at data from an internal peer feedback process in a medium-sized company over several years. They were able to show that when disconfirmatory feedback arrived, workers would then avoid contact with the people who had given them the unwelcome comments. This is the exact opposite of my professor friend’s behaviour — but, I think, a much more typical response. We don’t like it when people tell us that we’re failing.

The irony is that disconfirmatory feedback is the most useful kind of feedback imaginable. If I’m making serious mistakes while cruising along in a complacent bubble of self-satisfaction, I badly need someone to explain exactly what I’m doing wrong. But what I need and what I might enjoy are, of course, quite different.

In certain corners of the business world, it’s become fashionable to talk openly about failure — some would say to “celebrate failure”, although that’s a lazy description. A botched surgical procedure or a fatal traffic accident aren’t things to be celebrated, but they should be discussed, analysed and learnt from. The most obvious way to do this is through a postmortem: all is lost, the project failed, the patient died, so let’s at least try to do better next time.

Less painful, though, is psychologist Gary Klein’s idea of the “pre-mortem”. The pre-mortem is an exercise in which you try to imagine scenarios in which your project fails. Such scenarios are likely to suggest simple ways to prevent disaster.

Alternatively, one can try to learn from other people’s mistakes rather than one’s own. In 2009, Cass Phillipps, a conference producer based in San Francisco, found herself feeling alienated by Silicon Valley conferences packed with gung-ho founders describing their runaway successes. She set up an alternative, “FailCon”, where people came to dissect their failures — in many ways, an enormously more informative experience for attendees.

Phillipps has now stopped organising FailCon — in part, she tells me, because the urgency has gone: “The internet is filled with postmortem stories.” But the basic idea is sound and has spread. For instance, a new book by development economists Dean Karlan and Jacob Appel, Failing in the Field (UK) (US), is simply a catalogue of all the mistakes they’ve made while trying to evaluate projects, and a discussion of what others might do to avoid the same errors.

Still, the real trick, says Cass Phillipps, is to spot and fix your mistake before it becomes fatal. But while “how I failed and what I learnt” is a safe enough topic for a conference or a book, “Help! Help! I need help now!” is a much rawer message.

So I turned to Ashley Good, the CEO of “Fail Forward”, a consultancy that helps organisations turn failures into more productive experiences. I asked her why it’s difficult to cope with failures in real time. One reason: panic.

“Failure tends to push us into a stress response,” she says, and that promptly leads to denial, finger-pointing, self-flagellation, cover-ups or “any number of dysfunctional reactions that limit our ability to learn”. That means that the first step after discovering some major screw-up is to take a deep breath and try to calm down.

The second step, says Good, is to “be respectful and kind”. That’s good advice at any time, of course — but particularly when emotions are running high and there’s a problem to be solved. And a third step is to take some individual responsibility: to ask, “What could I, personally, do differently now to avoid this sort of thing in future?”

Ultimately, the aim of all this is not to “celebrate” a disaster but to make things better, by fixing the current problem, if possible, and by preventing a recurrence. That means asking cool questions about what the problem really is.

When the dancer and choreographer Twyla Tharp earned scathing early reviews for her musical Movin’ Out, she asked a trusted colleague to transform the pile of criticism into a checklist for improvement. This depersonalised the criticism and turned it into a to-do list. And it worked: the revised show won Tony awards and enjoyed a long run.

Tharp has something in common with the professor who values his frankly critical collaborator: both of them recognise that thoughtful criticism isn’t something to be avoided — or, for that matter, something to be celebrated. It’s something to be used.

Written for and first published in the Financial Times.

My book “Messy” is available online in the US and UK or in good bookshops everywhere.

Free email updates

(You can unsubscribe at any time)



  • 1 Twitter
  • 2 Flickr
  • 3 RSS
  • 4 YouTube
  • 5 Podcasts
  • 6 Facebook


  • Messy
  • The Undercover Economist Strikes Back
  • Adapt
  • Dear Undercover Economist
  • The Logic of Life
  • The Undercover Economist

Tim’s Tweets

Search by Keyword

Free Email Updates

Enter your email address to receive notifications of new articles by email (you can unsubscribe at any time).

Join 4,150 other subscribers

Do NOT follow this link or you will be banned from the site!