Tim Harford The Undercover Economist

Undercover EconomistUndercover Economist

My weekly column in the Financial Times on Saturdays, explaining the economic ideas around us every day. This column was inspired by my book and began in 2005.

Undercover Economist

Why we need to disagree

A few days after Christmas in 1978, United Airlines Flight 173 ran into trouble on its descent into Portland, Oregon. The landing gear should have descended smoothly and an indicator light blinked on to indicate all was secure. Instead, there was a loud bang and no light.

While the crew tried to figure out whether the landing gear was in position or not, the plane circled and circled. The engineer mentioned that fuel was running low, but didn’t manage to muster enough forcefulness to convey the urgency to the captain, who was focused on the landing gear. Finally, when the first officer said “we’re going to lose an engine, buddy”, the captain asked, “why?”

The plane crashed shortly afterwards. Ten people died. The lesson: sometimes we can’t bring ourselves to speak up, even when lives are at stake.

It might seem strange, in this politically divided age, to call for people to speak out if they see things differently. But our current political discourse doesn’t quite qualify. (Abuse is not an argument, as any Monty Python fan knows.)

Useful dissent means serious engagement with people who see the world differently — or, perhaps, the courage to puncture the consensus of one’s own tribe. It is far more common to see people seeking out like-minded groups, while politicians are happy to deliver hellfire sermons to their own choirs.

That is a shame. Within a cohesive group, the mere demonstration that disagreement is possible can have liberating effects. Charlan Nemeth, a psychologist at the University of California, Berkeley, studies dissent. (Her recent book is titled, No!: The Power of Disagreement in a World that Wants to Get Along – or in the US, In Defense of Troublemakers; at least we can reliably expect transatlantic disagreement over titles.) When she arrived at the university she found her office a little too austere, and decided to put down a rug.

“These offices are all the same for a reason,” remonstrated a colleague. She kept the rug anyway — and before long, her colleagues started putting rugs in their offices, too. Apparently, few people had liked the austere offices but nobody was willing to admit that. It took Prof Nemeth’s low-level troublemaking to shatter the illusion of consensus.

Prof Nemeth has studied disagreement during brainstorming sessions. One rule of brainstorming is not to criticise the ideas of others. When she and colleagues ran their sessions, they found that groups produced more ideas if the “do not criticise” rule was reversed, encouraging participants to “debate and even criticise each other’s ideas”.

Dissent can free us to place rugs in our offices, or express our individuality in more important ways. It can also stimulate our ideas and creativity. And — as the case of Flight 173 suggests — if we hesitate forcefully to disrupt a group conversation, that can deny others a vital piece of information.

Matthew Syed, in his book Rebel Ideas (this one also has a different title in the US; there’s something in the air…) draws the same conclusion from a disastrous attempt on Everest in 1996. Mr Syed argues that junior members of the expedition had useful pieces of information about the weather and their equipment but tended to stay silent, deferring to the team leaders.

A similar dynamic is at play in lower-stakes environments. One study, conducted by Garold Stasser and William Titus, asked undergraduates to discuss hypothetical candidates for a student society president.

The researchers gave each participant a different fact sheet; some facts were given to everyone in the discussion, but others were disclosed to only one person. People rarely spoke up about their private information, and the conversation revolved — redundantly — around what the whole group knew already rather than trying to find out what wasn’t widely known. There was an opportunity for everyone to learn from everybody else, but it proved more comfortable to focus on knowledge that they all had in common.

The truth is that disagreement is hard. We find it unpleasant to be disagreed with, and it can be painful to be a dissenter. Prof Nemeth notes that when she hired actors to play the role of dissenters in experiments studying group dynamics, the actors found it distressing to be on the receiving end of hostility. Some even asked for “combat pay”.

Even in gentler settings, we underestimate the benefit of friction. One study of problem solving (conducted by Katherine Phillips, Katie Liljenquist and Margaret Neale) simply contrasted small groups of friends with those of three friends plus a stranger. The groups with an outsider did much better at solving the problems, even though the strangers had no special expertise: their mere presence raised everyone’s game.

Nevertheless, the groups of friends enjoyed themselves more and had more confidence in their answers — confidence that was, of course, badly misplaced.

We rarely appreciate it when someone is speaking out rather than fitting in. But whether it is as trivial as a rug, or as vital as a fuel gauge in a circling aircraft, we need people who see things that we don’t. We need them to speak up. And we also need to listen when they do.

Written for and first published in the Financial Times on 31 January 2020.

Catch up on the first season of my podcast “Cautionary Tales” [Apple] [Spotify] [Stitcher]

Receive these posts by email

(You can unsubscribe at any time)

Undercover Economist

The prisoner’s dilemma at 70 – at what we get wrong about it

Once upon a time, a pianist was arrested by the secret police and accused of spying. He was carrying sheets of paper covered with a mysterious code. Despite protesting that it was merely the sheet music for Beethoven’s Moonlight sonata, the poor man was marched to the cells. A couple of hours later, a sinister interrogator walked in. “You’d better tell us everything, comrade,” he announced with a thin smile. “We have caught your friend Beethoven. He is already talking.”

This sets up the most famous problem in game theory: the prisoner’s dilemma. The interrogator explains that if one man confesses and the other does not, the talkative prisoner will go free and the other will do 25 years in a gulag. If they both remain silent, they will each spend five years in prison. If they both confess, 20 years each. The dilemma is clear enough: each would do better to confess, regardless of what the other does; yet collectively they could profit by sticking together.

The dilemma is now 70 years old — it was developed in a simple mathematical form in 1950 by mathematicians Merrill Flood and Melvin Dresher and wrapped in a story by Albert Tucker. (My own retelling owes a debt to economists Avinash Dixit and Barry Nalebuff.)

Dresher, Flood and Tucker worked at the Rand think-tank. The prisoner’s dilemma distilled the tension between selfishness and co-operation into a potent form, making it emblematic of the risk of nuclear destruction and much more besides. The dilemma received a second burst of attention in 1981, after the publication of “The Evolution of Cooperation” by political scientist Robert Axelrod and evolutionary biologist William Hamilton. Their article is not only the most cited in political science, but as popular as the next three works put together.

I hope readers will forgive my dredging up such a venerable idea, because it remains relevant, instructive, and widely misunderstood. One common misunderstanding is that the problem is one of communication: if only the pianist and Beethoven could get together and agree a strategy, they’d figure out that they should stick together. Not so. Communication doesn’t solve anything. The attraction of teaming up is obvious; so is the temptation to betray. Those who believe talking helps much should watch Golden Balls, a game show based on a modified prisoner’s dilemma. What makes the show fun to watch is the emptiness of the promises contestants make to each other.

More problematic is the mistaken belief that the prisoner’s dilemma means we are doomed to selfish self-destruction. Moral philosophers have tied themselves in knots trying to refute it, to show that it is somehow rational to collaborate in a one-shot prisoner’s dilemma. It isn’t. Fortunately, most human interaction is not a one-shot prisoner’s dilemma. The 1981 paper — and subsequent book — may have pushed the pendulum too far in an optimistic direction. Prof Axelrod ran tournaments in which computer programs competed against each other, playing the prisoner’s dilemma hundreds of times. Repeating the game allows co-operation to be enforced through the threat of punishment — something game theorists had known since the 1950s. When Prof Axelrod enshrined that idea in a simple program called “Tit for Tat”, it routinely triumphed.

Tit for Tat responds to co-operation with co-operation, and betrayal with betrayal. Whatever you do to it, it does right back. Prof Axelrod highlighted the fact that although the program was tough, it was “nice” — it tried co-operation first. And he drew broader parallels, arguing that the success of the strategy explains why soldiers in the trenches of the first world war were able to agree informal ceasefires. His inspiring message was that in the worst possible circumstances, nice guys finish first — provided they have an inner steel.

But that goes too far. A simpler explanation of “live and let live” in the trenches is that popping up to shoot at the enemy is nothing like ratting out Beethoven. It is dangerous. One needs no game theory to explain why soldiers might prefer to lie low.

Prof Axelrod also set far too much store by Tit for Tat’s “niceness”. Other strategies prosper in prisoner’s dilemma tournaments, depending on details of the rules. Among them is “Pavlov”, a strategy that tries to exploit suckers and changes tactics when it encounters a punishing response. It can be co-operative, sure — but it is hardly “nice”.

Prisoner’s dilemmas do exist. The most pressing example today is climate change. Every nation and every individual benefits if others restrain their pollution, but we all prefer not to have to restrain our own. It would be foolish to hope that Tit for Tat will save the day here — and we don’t have to. We have tools available to us: domestically, taxes and regulations; internationally, treaties and alliances. Such tools change the incentives. We could and should be using them more. The pianist and his suspected accomplice were trapped. We are not. Unlike them, we can change the game.

Written for and first published in the Financial Times on 24 January 2020.

My book “Fifty Things That Made the Modern Economy” (UK) / “Fifty Inventions That Shaped The Modern Economy” (US) is out now in paperback – feel free to order online or through your local bookshop.

Receive these posts by email

(You can unsubscribe at any time)

Undercover Economist

Why my purchase choices have the kiss of death

Steve Eisman, the investment manager made famous by Michael Lewis’s The Big Short, did a lot of homework in his quest for terrible assets to bet against. But when he was introduced to another investment manager — Wing Chau — he saw the opportunity to accelerate the decision-making process: “Whatever that guy is buying, I want to short it.”

For Mr Eisman, Wing Chau was the equivalent of a watch that is six hours off: a perfect guide, as long as you realise that you need to look at the opposite side of the deal — or the clock face.

A few years ago, four business school academics won attention for the discovery that the same logic may work for retail products. Eric Anderson, Song Lin, Duncan Simester and Catherine Tucker found what they called “harbingers of failure” — consumers who simply adored the Ford Edsel, the Betamax video format, or those squeezy bottles loaded with Heinz EZ Squirt ketchup in bright blue, green and purple, a kind of edible paint. These people thought nothing cried out “sophisticated lady” more loudly than a packet of Bic disposable knickers.

Product development teams have long prized the idea that “lead customers” could give them insight into where the mass market might be going. A celebrated example is the mountain bike, a product assembled by enthusiasts who, starting in the early 1970s, modified old bikes by adding balloon tyres and motorcycle brakes to cope with demanding off-road conditions. Fifteen years later, the mountain bike was a mainstream retail product.

Pointing to such examples, Eric von Hippel, a professor at MIT, argued that companies shouldn’t just show product ideas to focus groups made up of generic, average consumers. They should find the early adopters and the trend setters, and pay particular attention to them.

But the “harbingers of failure” study reminds us that we could equally seek customers with the opposite quality: an unerring nose for products that the mass market will despise. Perhaps it shouldn’t be a surprise that such people exist. Prof Anderson and his colleagues suggested that companies could identify harbinger customers by examining their purchase decisions, and then use them as a guide to what not to stock in future. They also concluded that these customers provided a strong signal of a product’s prospects: “The more they buy, the less likely the product will succeed.”

Recently, the plot thickened like a glob of EZ Squirt: a research paper from professors Simester and Tucker and Clair Yang reported on “The Surprising Breadth of Harbingers of Failure”. This study found that “not only are there customers who are harbingers, but there are also harbinger zip codes”.

People in these accursed neighbourhoods buy doomed products, and also niche products that nearby zip codes don’t find attractive. The tendency is broad-based: they buy unpopular products at a big-box warehouse store, but they also buy unpopular garments at a clothing retailer. This is rather convenient for market researchers — Prof Simester and colleagues argue that zip codes provide all the information needed to learn from the harbinger effect. The harbinger zip codes are even losing propositions in electoral campaigns: they are more likely to donate money to political candidates who lose, and less likely to donate to popular ones.

And then I realised: they’re talking about me. While I’d prefer not to reveal too much about my voting habits, it has been a very long time since I was on the winning side: I didn’t vote for Boris Johnson, I didn’t vote for David Cameron and I didn’t vote for Tony Blair. I was on the losing side in all the referendums, too. Politically, I am Crystal Pepsi. I am Colgate ready meals.

Come to think of it, as a student I did go through a phase of drinking the monumentally unsuccessful soft drink, Tab. I’ve never owned an iPhone and when my wife bought me an iPad, I sent it back because I couldn’t figure out how to make it work. In the pool, I wear Speedos. I am the Wing Chau of retail and politics: come study me, oh trendspotters and psephologists, for a glimpse into what the future does not hold.

All this makes me wonder: what makes a harbinger of failure, and why is our taste for the unpopular so wide-ranging? Why would someone who admires Clairol’s Touch of Yogurt shampoo feel the same way about the Liberal Democrats’ Vince Cable? Perhaps we harbingers are open-minded, happy to take a risk on something new and unusual? Perhaps; but harbingers don’t just try Frito-Lay lemonade, we swig it down and then come back for more.

Perhaps the answer is that ordinary, well-adjusted people notice what other people are doing, and fit in. In contrast, we harbingers are simply oblivious. Jacket and jeans? Socks and sandals? Why not? I have yet to see a completely convincing explanation — or even to be fully persuaded that the whole idea isn’t one big statistical fluke. But if anyone in market research would like to follow me around a supermarket, get in touch.


Written for and first published in the Financial Times on 17 January 2020.

My book “Fifty Things That Made the Modern Economy” (UK) / “Fifty Inventions That Shaped The Modern Economy” (US) is out now in paperback – feel free to order online or through your local bookshop.

Receive these posts by email

(You can unsubscribe at any time)

Undercover Economist

Two cheers for the dematerialising economy

If past trends continue, the world’s gross domestic product will be about twice as big by 2040 as it is today. That’s the sort of growth rate that translates to 30-fold growth over a century, or by a factor of a thousand over two centuries. Is that miraculous, or apocalyptic? In itself, neither. GDP is a synthetic statistic, invented to help us put a measuring rod up against the ordinary business of life. It measures neither the energy and resource consumption that might worry us, nor the things that really lead to human flourishing.

That disconnection from what matters might be a problem if politicians strove to maximise GDP, but they don’t — otherwise they would have hesitated before imposing austerity in the face of a financial crisis, launching trade wars or getting Brexit done. Economic policymaking has flaws, but an obsession with GDP is not one of them.

Nevertheless the exponential expansion of GDP is indirectly important, because GDP growth is correlated with things that do matter, good and bad. Economic growth has long been associated with unsustainable activities such as carbon dioxide emissions and the consumption of metals and minerals.

But GDP growth is also correlated with the good things in life: in the short run, an economy that is creating jobs; in the long run, more important things. GDP per capita is highly correlated with indicators such as the Social Progress Index. The SPI summarises a wide range of indicators from access to food, shelter, health and education to vital freedoms of choice and from discrimination. All the leading countries in the Social Progress database are rich. All the strugglers are desperately poor.

So the prospect of a doubling of world GDP matters, not for its own sake, but for what it implies — an expansion of human flourishing, and the risk of environmental disaster. So here’s the good news: we might be able to enjoy all the good stuff while avoiding the unsustainable environmental impact. The link between economic activity and the use of material resources is not as obvious as one might think. There are several reasons for this.

The first is that for all our seemingly insatiable desires, sometimes enough is enough. If you live in a cold house for lack of money, a pay rise lets you take off the extra cardigan and turn up the radiators. But if you win the lottery, you are not going to celebrate by roasting yourself alive.

The second is that, while free enterprise may care little for the planet, it is always on the lookout for ways to save money. As long as energy, land and materials remain costly, we’ll develop ways to use less. Aluminium beer cans weighed 85 grammes when introduced in the late 1950s. They now weigh less than 13 grammes.

The third reason is a switch to digital products — a fact highlighted back in 1997 by Diane Coyle in her book The Weightless World [pdf]. The trend has only continued since then. My music collection used to require a wall full of shelves. It is now on a network drive the size of a large hardback book. My phone contains the equivalent of a rucksack full of equipment.

Dematerialisation is not automatic, of course. As Vaclav Smil calculates in his new book, Growth, US houses are more than twice as large today as in 1950. The US’s bestselling vehicle in 2018, the Ford F-150, weighs almost four times as much as 1908’s bestseller, the Model T. Let’s not even talk about the number of cars; Mr Smil reckons the global mass of automobiles sold has increased 2,500-fold over the past century.

Still, there is reason for hope. Chris Goodall’s research paper “Peak Stuff” concluded that, in the UK, “both the weight of goods entering the economy and the amounts finally ending up as waste probably began to fall from sometime between 2001 and 2003”. That figure includes the impact of imported goods.

In the US, Jesse Ausubel’s article “The Return of Nature” found falling consumption of commodities such as iron ore, aluminium, copper, steel, and paper and many others. Agricultural land has become so productive that some of it is being allowed to return to nature.

In the EU, carbon dioxide emissions fell 22 per cent between 1990 and 2017, despite the economy growing by 58 per cent. Only some of this fall is explained by the offshoring of production. (For a good summary of all this research, try Andrew McAfee’s book More From Less.)

Can we, then, relax? No. To pick a single obvious problem, global carbon dioxide emissions may be rising more slowly than GDP — but they are rising nevertheless, and they need to fall rapidly. Yet the fact that dematerialisation is occurring is heartening. We all know what the basic policies are that would tilt the playing field in favour of smaller, lighter, lower-emission products and activities. Adopting those policies means we might actually be able to save the planet, preserve human needs, rights and freedoms — and still have plenty of fun into the bargain.


Written for and first published in the Financial Times on 10 January 2020.

Catch up on the first season of my podcast “Cautionary Tales” [Apple] [Spotify] [Stitcher]

My book “Fifty Things That Made the Modern Economy” (UK) / “Fifty Inventions That Shaped The Modern Economy” (US) is out now in paperback – feel free to order online or through your local bookshop.

Receive these posts by email

(You can unsubscribe at any time)

Undercover Economist

Will the 2020s be the decade that the robots finally come for our jobs?

Will this decade finally be the one in which the machines take our jobs? Such concerns have been aired many times over the centuries and they have always been wrong. But they are not intrinsically absurd.

In 1979, the economist Wassily Leontief pointed to the fate of the horse. Horses had long been of vital economic importance, but faded in the second half of the 20th century as the internal combustion engine became the dominant source of horsepower. Horses still have a niche, but will never outcompete engines, no matter how cheap oats become.

Might large numbers of human workers go the way of the horse? In 2003, the economists David Autor, Frank Levy and Richard Murnane published a study of the economics of technological change that made two influential observations.

First, they pointed out (correctly) that it is misleading to talk of robots — or any other technology — taking jobs. Instead, machines perform tasks, a narrower unit of work. Since most jobs involve many different tasks, robots do not take jobs, but they may radically reshape them. A robot accountant is not C-3PO; it’s Excel or QuickBooks. As with the horse, there is no wage at which human calculators can compete with a computer at the task of adding up a spreadsheet. Still, human accountants exist in large numbers. Their jobs simply look very different today.

Second, argued Profs Autor, Levy and Murnane, the tasks that machines took on were best described not as “skilled” or “unskilled” but as “routine” or “non-routine”. Recalculating a spreadsheet is a skilled but routine task, easily automated. Cleaning a toilet requires little skill — even I can do it — but is non-routine and therefore hard to automate.

This way of looking at the world proved very useful. It explained why technology could disrupt our jobs without destroying them. And why both the low-paid and high-paid ends of the labour market were proving robust, while the middle, packed with skilled-yet-routine tasks, was hollowed out.

But in a new book, A World Without Work, Daniel Susskind argues that the second part of the Autor-Levy-Murnane perspective is proving more questionable. He observes that the boundaries of the “routine” are blurring fast. Consider, for example, CloudCV, a system that answers open-ended questions about images. Upload an image and ask any question you like.

One photograph showed some 20-somethings sitting on a sofa with white wine and cans of Kronenbourg lager in front of them, with one fellow standing in a dramatic pose. “What are they doing?” I asked the computer. “Playing Wii,” it replied, correctly. “What are they drinking?” Probably beer, it said. “How’s the weather?” I asked of an outdoor snapshot. “Cloudy.” It was.

The system gives accurate answers to informally phrased questions about random photographs. Is that task routine? Hardly.

Neither is the performance of Alpha Zero, the game-playing algorithm developed by DeepMind, a sister company of Google. In 2017, AlphaZero trained itself in a few hours to thrash the best chess-playing engine and the best Go program, both of which easily beat the best humans. Some claim this performance is less impressive than it first appears — but 10 years ago the mere idea that a computer could beat a human at Go seemed implausible. What DeepMind’s supercomputers can do today will be achievable on laptops and phones by 2030.

In task after task, the computers are overtaking us. In the Visual Question Answering challenge that CloudCV attempts, humans score 81 per cent. The machines were at 55 per cent as recently as 2016; by the summer of 2019 they were at 75 per cent. It’s only a matter of time before they do a better job than us — just as AlphaZero does.

The Artificial Intelligence Index project, based at Stanford University, tracks a wide variety of benchmarks. The machines are making rapid progress at symbolic achievements — such as playing poker — but also at translation, speech recognition, and classifying diseases such as skin cancer (from images of moles) and diabetes (from images of retinas).

These achievements are real. And despite the fact that there are many things computers cannot do, when an algorithm does a narrow task cheaply and well, we humans end up contorting ourselves to unleash the new capability while sweeping up the tasks the software leaves behind. Just look at the self-checkout at your local supermarket.

So — will the machines take all the jobs in the coming decade? No, and that remains an unhelpful way to phrase the question. Machines encroach on tasks, and we reorganise our jobs in response, becoming more productive as a result. But there is good reason to believe that such reorganisations will be wrenching in the decade to come, and also that some people will be permanently unable to contribute economically in the way they would have hoped and expected. Above all, it is likely that our political institutions will be unable to adapt to the challenge.

Written for and first published in the Financial Times on 3 January 2020.

My book “Fifty Things That Made the Modern Economy” (UK) / “Fifty Inventions That Shaped The Modern Economy” (US) is out now in paperback – feel free to order online or through your local bookshop.

Receive these posts by email

(You can unsubscribe at any time)

Undercover Economist

Ten years of social media have left us all worse off

The last decade has had plenty of landmark moments — but one big change crept up on us slowly: our experiences in the liminal space of social media. Somewhere between Silicon Valley and our vibrating pockets, between our closest friends and some faceless trolls, our privacy, politics, economy and above all our attention were reshaped by Facebook and its outriders.

Social media existed before 2010, but not as we now know it. Few of us had smartphones in 2009. Facebook’s active user base has grown sevenfold over the past 10 years, and there simply aren’t enough people for that to happen again. Instagram and WhatsApp were both launched about a decade ago, and swiftly absorbed into the mother of all social networks. As for Twitter, let me simply note that Donald Trump only started tweeting in earnest in 2011.

What effect has all this had? It’s plausible to argue that social media enabled major events such as the Arab spring and the election of Mr Trump, although of course there is never a single explanation for such things. There have been some telling little moments, however — such as when the UK Conservative party press office took the low-rent Orwellian step of posing as an independent fact-checking organisation on Twitter. (No doubt they would describe that incident differently, while adding that Oceania has always been at war with Eastasia.)

I should not exaggerate. This isn’t 1984. Partisan news sources were popular long before we self-selected into online echo chambers. Propaganda is not new. And there are benefits from social media: it gives a platform to all sorts of people who deserve to be heard. But it is hard to make the case that social media has led to a more thoughtful, rigorous or compassionate discourse about politics. Amid the bullying, the misogyny, and the endless outrage, it’s hard to tell the bots and the people apart, largely because so many humans have lowered themselves to the level of the bots.

What about the economics? Network effects mean that social media platforms tend to spiral towards monopoly. You want to be where your friends are. It might be hard for a new search engine to displace Google, but if I am tempted by an alternative, I don’t need to persuade my friends and family to move too.

An obvious antitrust measure would be to force Facebook to divest WhatsApp and Instagram, two services that could and should be its competitors. A more radical approach is to require social networks to improve their interoperability and data portability — effectively allowing other services to piggyback, or users to flit among services. If I switch email providers or phone companies I can bring my phone number and contact database with me, or automatically forward messages sent to my old email address. It’s possible to imagine social media working more like that in future, although it would require substantial effort both technologically and legislatively.

Yet none of this solves perhaps the most basic problem. Ten years ago all we had to worry about was email overload. Now we carry around powerful and highly distracting devices. They observe our behaviour, buzz insistently to get attention, and leverage our desire to fit in, communicate and reciprocate. We did not consciously sign up for this, and each of us needs to think carefully about what we really want from social media.

Last Christmas I vowed to spend less time on my smartphone. It worked — until a couple of months ago, when I started using Twitter much more. Why? I had something to sell. That seems wretchedly appropriate. Still, another decade is starting. I cannot break Facebook up by myself, but I can plan to do something more constructive with the time and energy I often spend on social media. I hope I am not the only one.


Written for and first published in the Financial Times on 27 December 2019.

My book “Fifty Things That Made the Modern Economy” (UK) / “Fifty Inventions That Shaped The Modern Economy” (US) is out now in paperback – feel free to order online or through your local bookshop.

Receive these posts by email

(You can unsubscribe at any time)

Undercover Economist

‘Salvator Mundi’ and the limits of certainty

Mona Lisa may be famously inscrutable, but “Salvator Mundi” has surely replaced her as Leonardo da Vinci’s most enigmatic work. It has been two years since it was reported that the long-lost painting had been sold to a Saudi prince as a gift to the Louvre Abu Dhabi, for an astonishing $450m — two and a half times the previous record for any painting sold at auction.

Since then the unveiling has been postponed without explanation, and the painting’s whereabouts are unknown: on a yacht, says one report; in secure storage in Switzerland, says another.

No doubt the mystery of its whereabouts will be resolved. The mystery of its provenance is deeper. In 2005, “Salvator Mundi” was bought for about $1,000 at an auction in New Orleans by two art dealers, Alexander Parish and Robert Simon. (Mr Parish later told Vulture that they had been willing to go as high as $10,000, but it proved unnecessary.)

On the surface, the painting was worth little: it was in very bad shape. But Messrs Parish and Simon thought it might be by a disciple of Leonardo; in which case it might easily be worth several hundred thousand dollars — a gamble worth taking. As a painting by Leonardo’s studio, with a touch or two by the master himself, it might have been worth $20m.

So what is it? Ben Lewis, author of The Last Leonardo, notes that the debate rages “over whether it belongs in the first division autograph Leonardo category or the second division Leonardo+Workshop category”. Apparently that is a $430m distinction. And the desire for clarity is not merely financial. When we gaze at a painting on a gallery wall, we like to know.

It is hard, too, to disentangle the time-scarred original work from its substantial restoration by Dianne Modestini — which, in turn, was influenced by the close inspection of known works by Leonardo.

Yet as the criminologist Federico Varese points out, it is curious that we insist on a binary distinction. We feel powerfully that the painting is either an autograph Leonardo, or it is not. As a matter of logic that may be true, but as a matter of practicality we do not know and we will never know. There is some evidence of Leonardo’s involvement, but the evidence is circumstantial. We are relying heavily on intuition — albeit the intuition of people with deep expertise. Regrettably but unsurprisingly, the experts differ.

This is partly a problem of knowledge: we cannot travel back in time to see who painted what. But it is also a problem of definition. Philosophers might recognise the “bald man paradox” here. Plucking out a single hair from a full head of hair does not produce a bald man. Keep going, however, and baldness will result. And yet it seems absurd to identify any particular hair as the crucial one that made the difference between baldness and non-baldness. Similarly with “Salvator Mundi”: how many brushstrokes from Leonardo does it take to distinguish a workshop piece from an autograph work?

So “Salvator Mundi” is the Schrödinger’s cat of paintings — perhaps one thing, perhaps another. We can’t know.

Schrödinger’s cat discomfited the Austrian physicist Ernst Schrödinger, for good reason. But to a statistician or a social scientist, this sort of irresolvable uncertainty is part of life.

I just tossed a coin. Did it come up heads or tails? One or the other, clearly. But even after the fact, if you haven’t seen the result it is not absurd to say that there is a 50 per cent chance of either outcome. And if I then put the coin back in my pocket without checking, 50-50 is the closest we will ever get to knowing.

We should be able to live with such fuzziness. When asking a question such as “who is the greatest ever Formula 1 driver?”, we know that we can have a fun argument — Lewis Hamilton, Michael Schumacher, Ayrton Senna, Juan Fangio? And we also know that the argument cannot be resolved.

But we forget this in other parts of life. Who would be the better UK prime minister, for example, Jeremy Corbyn or Boris Johnson? Which Democratic candidate would be most likely to defeat US president Donald Trump in the 2020 elections? Is it Joe Biden, Elizabeth Warren, Bernie Sanders or Pete Buttigieg? How serious a threat is climate change, and how drastic a change is required to deal with it?

The answers matter far more than the question of how much Leonardo contributed to “Salvator Mundi”, if he contributed at all. But we will never know for sure what the answers are.

One approach to all this fuzziness is to demand sharpness. I have often written admiringly about the work of Philip Tetlock, who has examined the problem of forecasting — a field dominated by vague prognostications — by asking forecasters to make verifiable predictions with deadlines.

But there are limits. The world defies our attempts to confine it with neat definitions.

It is not wrong to debate these vast questions of policy and politics. Indeed, it is vital that we do. But it is futile to expect a certain answer.

Written for and first published in the Financial Times on 6 December 2019.

Catch up with the first season of “Cautionary Tales” [Apple] [Spotify] [Stitcher]

For more on the joys of ambiguity, try my book, Messy

Receive these posts by email

(You can unsubscribe at any time)

Undercover Economist

How Sesame Street set a gold standard for education

The children’s television show Sesame Street just celebrated its 50th birthday. I know my favourite character should be Count von Count, who shares my fondness for numbers. But I’ve always had a soft spot for Mr Snuffleupagus, Big Bird’s best friend.

Mr Snuffy was thought by every adult on Sesame Street to be imaginary despite being as real as Elmo. It’s a good joke: Mr Snuffy, a strange anteater-mammoth hybrid, is colossal. How could the adults not notice him?

After the gag had run for 14 years, the adults finally realised that Mr Snuffleupagus was real, and apologised to Big Bird for doubting him. This was a weighty decision: Sesame Street’s writers were concerned about child abuse, and reflected that it might be unwise to portray the adults as endlessly disbelieving what the childlike Big Bird told them.

This was typically painstaking behaviour from a show that has always had ambitious ideas about helping children. In 1967, a former TV producer named Joan Ganz Cooney wrote a report for the Carnegie Corporation titled “The Potential Uses of Television in Pre-school Education”. She made the case that carefully crafted television could “foster intellectual and cultural development in pre-schoolers”. Two years later, her vision became reality, in the Children’s Television Workshop and Sesame Street.

It was a radical idea: just a few years earlier, Marshall McLuhan had infamously argued that “the medium is the message”. It seemed natural enough to many that television was an inherently superficial medium with, therefore, a superficial message.

By contrast, Sesame Street was a bet that good television could make a real difference to children’s readiness for school, particularly for those starved of other opportunities to learn. Not only would it help them to read and count, but it would be racially integrated. Over the years it would tackle issues including death, divorce, autism, infertility, adoption and HIV.

Researchers swarmed all over Sesame Street, trying to figure out whether it actually worked. This wasn’t as easy as one might think. One early study, conducted by Samuel Ball and Gerry Ann Bogatz, aimed at a conventional experiment: some families, chosen at random, would be encouraged to sit preschoolers in front of this brand new show, while a control group of other families would receive no encouragement.

The problem was that Sesame Street became so popular, so quickly, that it became hard to distinguish between the two groups; everyone was watching. Nevertheless, the study authors did the best they could. They found that children who watched more Sesame Street learnt more, and that “in terms of its own stated goals, Sesame Street was in general highly successful”. Perhaps the message is the message after all.

Yet it is hard to be sure about causation. Did Sesame Street help kids learn? Or was the programme attractive to children who were already flourishing?

A recent study by two economists, Melissa Kearney and Phillip Levine, approaches the problem from a different angle. Professors Kearney and Levine noted that in the early years of Sesame Street, some geographical areas simply couldn’t receive the broadcast signals that carried the show. Two-thirds of US children could watch the show, and many did, but one-third could not.

Based on this accidental experiment, Profs Kearney and Levine concluded that the children who had lived in a region where Sesame Street was available were less likely to fall behind at school. The effect was about as large as attending the US Head Start early childhood education programme — impressive, given that TV is so cheap. The benefits were particularly large for children who lived in deprived areas.

It is hard to read about this study without being reminded that Sesame Street was born in a very different world — one where children received Sesame Street via UHF broadcast, rather than watching Baby Shark on YouTube, where a version produced by the South Korean media brand Pinkfong has nearly 4bn views.

Like the Children’s Television Workshop 50 years ago, Pinkfong has lofty educational goals: its videos are supposed to teach English to Korean children. It has more than twice as many YouTube subscribers as Sesame Street, which struggled financially in recent years before cutting a deal with HBO.

But the vast, cosmopolitan and mysterious world of toddler YouTube seems unlikely to deliver the same educational benefits to children as Sesame Street, which was continually tweaked to help children learn rather than being relentlessly optimised for the clicks. As Alexis Madrigal observed in a long report for The Atlantic on toddler YouTube, the viral videos tend to be fast-paced and full of superfluous details. These features may attract the attention of preschoolers, but educational experts think they are unhelpful.

I’m an optimist. Online video could surely be even more educational than Sesame Street, given its ability to be interactive and to gather data on an individual child’s progress. But it would have to be carefully designed and tested, in the same way that Sesame Street was. An educational revolution doesn’t happen by accident.

Written for and first published in the Financial Times on 8 November 2019.

My new podcast is “Cautionary Tales” [Apple] [Spotify] [Stitcher]

Receive these posts by email

(You can unsubscribe at any time)

Undercover Economist

Exit, voice, or loyalty… what should we do when things go wrong?

“Under any economic, social or political system, individuals, business firms, and organizations in general are subject to lapses from efficient, rational, law-abiding, virtuous, or otherwise functional behavior.”

That is the first sentence of the economist Albert Hirschman’s book Exit, Voice, and Loyalty, published in 1970. No kidding; look around. Rational, law-abiding, virtuous and otherwise functional behaviour is in short supply.

Hirschman’s book is about how we register our discontent with such lapses, and whether our discontent makes a difference. Do we walk away? Do we protest? Or do we suffer in silence?

The instinct of the economist, used to studying competitive markets, is to think of “exit” as the most straightforward and powerful protest. If we don’t like the product or we don’t like the price, we take our custom elsewhere. The alternative is “voice”: we complain, in any form from a muttered grumble to a Molotov cocktail.

In many ways, exit is easier than it has ever been. A citizen of the US can move to a different state; a citizen of the EU can move to a different country. (Irish passports have become popular among the British citizens who can obtain them; people value the option to exit.) We have an endless choice of entertainments to enjoy, news sources to consume, companies from which to purchase. Niche political movements abound. The exit door never seems far away.

And yet, many of us have rarely felt so trapped. Yes, it’s possible to leave Facebook or stop using Google. But it is hardly as simple as switching to a different brand of toothpaste.

And for an age in which politics is supposed to be in endless flux, it is surprising how little changes. US president Donald Trump’s unpopularity is astonishingly consistent, with disapproval ratings of 53-56 per cent. The popularity of his predecessors ebbed and flowed; Mr Trump’s is frozen in ice.

In the UK, the sense of paralysis is palpable: we’ve had an election, a generationally defining referendum, a new prime minister, another election, another new prime minister, and now yet another election. Lots of politics but not a lot of progress. And nobody has managed to assemble a policy platform that commands broad support.

What is going on? Hirschman pointed to an intriguing case study: railways in Nigeria in the 1960s. Despite poor roads and an 800-mile journey from the peanut farms of northern Nigeria to the ports of Lagos and Port Harcourt, Hirschman observed that trucks comfortably outcompeted the railways. Why?

One might have expected that as peanut shippers quit trains and leased trucks instead, the railways would have responded. Hirschman argued that the reverse was true. The railways were propped up by the Nigerian state, so exit was no threat. Instead, the threat was voice, in the form of unhappy customers lobbying the government and generally raising hell. But those customers didn’t bother; they quit instead.

Typically, we think of exit and voice as complementary. Your complaints will be taken more seriously if you can credibly threaten to leave, as anyone who has called to cancel a mobile phone contract can attest. But sometimes exit can silence voice. That is particularly true when “voice” means something more than a mere complaint — taking time-consuming action such as attending council meetings, going on strike or actively campaigning. If you have another option, it is tempting to walk away and take it.

A similar logic applies to the two-party system that defines the US and remains strong in the UK. Like the Nigerian railways, the dominant parties seem to be a part of the landscape. They are propped up by tradition and the logic of first-past-the-post voting. Exit seems to be no threat to them, especially not to the hardliners who would rather lose than compromise.

What about voice? As with the Nigerian railways, voice has been weakened by exit. Some moderates have been thrown out of their own parties — notably Philip Hammond, who was UK chancellor of the Exchequer just a few months ago. Many others have decided they’ve had enough, and few people begged them to stay. What is true for members of parliament is true for party members, too. The extremists are delighted. The moderates have quit in disgust. The parties have moved ever further from the median voter.

That might matter less if both parties had not decided to give the final say over leadership to the party base, rather than to MPs, who know what it takes to get elected. It is an idea that looks better on paper than when put to the test. As it is, both the Conservatives and Labour are led by men who seem to inspire rapture among a narrow clique of supporters, but whom many voters find somewhere between laughable and contemptible.

The situation seems unsustainable. But will it change? As Hirschman was finishing his book, Nigeria was consumed by civil war. The country’s railways continued to stagnate for decades. I am hopeful that British democracy will bounce back a little more quickly; I just wish I could see exactly how.

Written for and first published in the Financial Times on 29 November 2019.

My new podcast is “Cautionary Tales” [Apple] [Spotify] [Stitcher]

Receive these posts by email

(You can unsubscribe at any time)

Undercover Economist

Algorithms judge us; how can we judge them?

If there was ever a demonstration that people think with their guts, it was the furore over the idea that Apple Card is “a f***ing sexist program”. David Heinemeier Hansson, a successful entrepreneur and programmer, complained on Twitter that his wife had a far lower credit limit than he did, and soon everyone from the US senator Elizabeth Warren to Apple co-founder Steve Wozniak to the New York Department of Financial Services were weighing in to show their support.

The idea of women being treated badly by Big Tech and by banks seems all too plausible. Apple is quite literally an iconic brand. Goldman Sachs, the bank that issues and manages the Apple-branded credit card, is nearly as famous. So the ingredients for a viral story are all there, however thin the anecdotal evidence. I

s the Apple Card actually sexist? One definition of equal treatment for men and women would be that credit was extended equally to both, regardless of the fact that women tend to be paid less than men. Another would be that people with the same income got the same credit, regardless of gender. You might have spotted the problem: it’s impossible to offer both forms of equal treatment simultaneously.
This isn’t just some clever piece of logic-chopping. If two groups of people are measurably different, then any rule about how they are treated — be it an algorithm or human judgment — will end up looking unfair, if not by one measure then by another. Is the Apple Card sexist? Arithmetic suggests that, for one definition of sexism or another, it must be.

This doesn’t excuse cases where decision processes — algorithmic or otherwise — are grossly biased, grotesquely inaccurate or both. Our problem is that we don’t know which ones they are, so we tend instead to believe emotionally resonant stories about famous brands. In the algorithm-saturated world we are entering, we need a way to distinguish the good from the bad, the ethical from the outrageous. We should be demanding better evidence that the algorithms that shape our lives are doing so fairly and effectively.

Goldman Sachs says that gender, race, age and sexual orientation are never explicitly part of the decision-making process. The company also says that the process is scrutinised both by consultants and an internal department to ensure that there is no accidental bias. You and I, however, are just going to have to take their word for the robustness of that scrutiny.

Companies are learning the hard way that people now want serious explanations: Goldman claims the Apple Card is unusually transparent, but people evidently want more.

Transparency might help — but it is neither a panacea nor an easy option. Netflix once released anonymised data about movie preferences as part of a competition to improve its recommendations. Alas, because some customers had posted reviews for both Netflix and the Internet Movie Database, it wasn’t hard to link the anonymous serial numbers with real names and intimate film reviews. One woman sued Netflix for potentially revealing her sexual orientation to her husband and children. Transparency is hard; Goldman cannot simply dump its data set and invite us all to poke around. But it could give access to independent assessors.

The philosopher Onora O’Neill argues that anyone who would like to be trusted should be trying to demonstrate trustworthiness. Trustworthiness, she adds, can be bolstered by “intelligent openness”. In the case of algorithms, we should expect a clear and prominent explanation of how the algorithm is making its decision — and perhaps more importantly, we should expect independent experts to be able to assess the claims that are being made.

There are arguably more important algorithms out there than the one that sets your Apple Card credit limit — such as the Facebook news feed or Compas, which is widely used in justice systems to assess the risk that a criminal will reoffend. I am not qualified to assess their fairness or effectiveness. But I know people who are, if they were allowed to see more information.

Compas has now been exhaustively analysed by academics, and worrying features have been exposed. But the analysis was only possible after a team at ProPublica published a painstakingly assembled data set for all to use. It should be easier for independent experts to scrutinise the algorithms that shape our lives.

One reason I am sanguine about the Apple Card is that other credit cards are available. If Goldman is mistakenly turning down creditworthy people, other companies will want their business. That is not a guarantee of fairness but it is, at least, a powerful force pulling in that direction.

In other cases there is no such force: if a criminal is denied parole on the word of an algorithm, there is no option to shop around. When companies peddle software systems that are supposed to identify the best teachers or the worst criminals or the children most at risk of domestic violence, we should demand proof. If not, we will be sold statistical snake-oil.


Written for and first published in the Financial Times on 22 November 2019.

My new podcast is “Cautionary Tales” [Apple] [Spotify] [Stitcher]

Receive these posts by email

(You can unsubscribe at any time)



  • 1 Twitter
  • 3 RSS
  • 5 Podcasts
  • 6 Facebook


  • Fifty Inventions That Shaped the Modern Economy
  • Messy
  • The Undercover Economist Strikes Back
  • Adapt
  • Dear Undercover Economist
  • The Logic of Life
  • The Undercover Economist

Search by Keyword

Free Email Updates

Enter your email address to receive notifications of new articles by email (you can unsubscribe at any time).

Join 179,368 other subscribers

Do NOT follow this link or you will be banned from the site!