Tim Harford The Undercover Economist

Articles published in December, 2019

Cautionary Tales Ep 8 – You Have Reached Your Destination

More than two and a half thousand years ago – so the story goes – King Croesus of Lydia consulted the oracle at Delphi. And the oracle assured him that if he went to war against Persia he would destroy a mighty empire. Reassured, Croesus launched his war, and was defeated. The oracle had been correct, but the mighty empire that Croesus destroyed was his own.

Our modern oracles are predictive algorithms. And perhaps the strange old tale of King Croesus has a great deal to teach us about how to interact with these silicon prophets.

Featuring: Archie Panjabi, Toby Stephens, Rufus Wright, Melanie Gutteridge, Mircea Monroe and Ed Gaughan.

Producers: Ryan Dilley and Marilyn Rust. Sound design/mix/musical composition: Pascal Wyse. Fact checking: Joseph Fridman. Editor: Julia Barton. Recording: Wardour Studios, London. GSI Studios, New York. PR: Christine Ragasa.

Thanks to the team at Pushkin Industries, Heather Fain, Mia Lobel, Carly Migliori, Jacob Weisberg, and of course, the mighty Malcolm Gladwell.

[Apple] [Spotify] [Stitcher]


Further reading and listening

Both stories about the oracle at Delphi are in Herodotus: The Histories.

Tom Knudson did the original reporting on “Death by GPS” for the Sacramento Bee. Reuters covered the Carpi / Capri confusion. Both stories – and others – are discussed in Greg Milner’s  excellent book Pinpoint.

Gretchen Morgenson covered AIG’s woes for the New York Times in “Behind Insurer’s Crisis, Blind Eye to a Web of Risk” 27 Sep 2008.

Esther Eidinow discusses what we can learn from how the Greeks consulted their oracles in “Oracles and Models” at The Conversation.

The Pierre Wack quote about forecasts is in “Scenarios: Uncharted Waters Ahead” Harvard Business Review Sep/Oct 1985.

The original study of the illusion of explanatory depth is Rozenblit, Leonid, and Frank Keil. “The misunderstood limits of folk science: an illusion of explanatory depth.” Cognitive science vol. 26,5 (2002): 521-562. doi:10.1207/s15516709cog2605_1

The study of how forecasting tournaments nurture humility is Barbara Mellers, Philip Tetlock, Hal R. Arkes, Forecasting tournaments, epistemic humility and attitude depolarization, Cognition, Volume 188, 2019, Pages 19-26

The study of a 1980s diagnostic aid is Wyatt J., Spiegelhalter D. (1991) Evaluating Medical Expert Systems: What To Test, And How ?. In: Talmon J.L., Fox J. (eds) Knowledge Based Systems in Medicine: Methods, Applications and Evaluation. Lecture Notes in Medical Informatics, vol 47. Springer, Berlin, Heidelberg

The study of navigating around Kashiwa with or without GPS is Toru Ishikawa, Hiromichi Fujiwara, Osamu Imai, Atsuyuki Okabe, “Wayfinding with a GPS-based mobile navigation system: A comparison with maps and direct experience” Journal of Environmental Psychology, Volume 28, Issue 1, 2008, https://doi.org/10.1016/j.jenvp.2007.09.002.



Receive these posts by email

(You can unsubscribe at any time)

Algorithms judge us; how can we judge them?

If there was ever a demonstration that people think with their guts, it was the furore over the idea that Apple Card is “a f***ing sexist program”. David Heinemeier Hansson, a successful entrepreneur and programmer, complained on Twitter that his wife had a far lower credit limit than he did, and soon everyone from the US senator Elizabeth Warren to Apple co-founder Steve Wozniak to the New York Department of Financial Services were weighing in to show their support.

The idea of women being treated badly by Big Tech and by banks seems all too plausible. Apple is quite literally an iconic brand. Goldman Sachs, the bank that issues and manages the Apple-branded credit card, is nearly as famous. So the ingredients for a viral story are all there, however thin the anecdotal evidence. I

s the Apple Card actually sexist? One definition of equal treatment for men and women would be that credit was extended equally to both, regardless of the fact that women tend to be paid less than men. Another would be that people with the same income got the same credit, regardless of gender. You might have spotted the problem: it’s impossible to offer both forms of equal treatment simultaneously.
This isn’t just some clever piece of logic-chopping. If two groups of people are measurably different, then any rule about how they are treated — be it an algorithm or human judgment — will end up looking unfair, if not by one measure then by another. Is the Apple Card sexist? Arithmetic suggests that, for one definition of sexism or another, it must be.

This doesn’t excuse cases where decision processes — algorithmic or otherwise — are grossly biased, grotesquely inaccurate or both. Our problem is that we don’t know which ones they are, so we tend instead to believe emotionally resonant stories about famous brands. In the algorithm-saturated world we are entering, we need a way to distinguish the good from the bad, the ethical from the outrageous. We should be demanding better evidence that the algorithms that shape our lives are doing so fairly and effectively.

Goldman Sachs says that gender, race, age and sexual orientation are never explicitly part of the decision-making process. The company also says that the process is scrutinised both by consultants and an internal department to ensure that there is no accidental bias. You and I, however, are just going to have to take their word for the robustness of that scrutiny.

Companies are learning the hard way that people now want serious explanations: Goldman claims the Apple Card is unusually transparent, but people evidently want more.

Transparency might help — but it is neither a panacea nor an easy option. Netflix once released anonymised data about movie preferences as part of a competition to improve its recommendations. Alas, because some customers had posted reviews for both Netflix and the Internet Movie Database, it wasn’t hard to link the anonymous serial numbers with real names and intimate film reviews. One woman sued Netflix for potentially revealing her sexual orientation to her husband and children. Transparency is hard; Goldman cannot simply dump its data set and invite us all to poke around. But it could give access to independent assessors.

The philosopher Onora O’Neill argues that anyone who would like to be trusted should be trying to demonstrate trustworthiness. Trustworthiness, she adds, can be bolstered by “intelligent openness”. In the case of algorithms, we should expect a clear and prominent explanation of how the algorithm is making its decision — and perhaps more importantly, we should expect independent experts to be able to assess the claims that are being made.

There are arguably more important algorithms out there than the one that sets your Apple Card credit limit — such as the Facebook news feed or Compas, which is widely used in justice systems to assess the risk that a criminal will reoffend. I am not qualified to assess their fairness or effectiveness. But I know people who are, if they were allowed to see more information.

Compas has now been exhaustively analysed by academics, and worrying features have been exposed. But the analysis was only possible after a team at ProPublica published a painstakingly assembled data set for all to use. It should be easier for independent experts to scrutinise the algorithms that shape our lives.

One reason I am sanguine about the Apple Card is that other credit cards are available. If Goldman is mistakenly turning down creditworthy people, other companies will want their business. That is not a guarantee of fairness but it is, at least, a powerful force pulling in that direction.

In other cases there is no such force: if a criminal is denied parole on the word of an algorithm, there is no option to shop around. When companies peddle software systems that are supposed to identify the best teachers or the worst criminals or the children most at risk of domestic violence, we should demand proof. If not, we will be sold statistical snake-oil.


Written for and first published in the Financial Times on 22 November 2019.

My new podcast is “Cautionary Tales” [Apple] [Spotify] [Stitcher]

Receive these posts by email

(You can unsubscribe at any time)

Cautionary Tales Ep 7 – Bowie, jazz, and the unplayable piano

He’d played with Miles Davis and Art Blakey and this was to be the biggest solo concert of Keith Jarrett’s career – but the Virtuoso pianist was in for a shock when he entered Cologne’s opera house. The only piano at the venue was a wreck. His musical contemporaries David Bowie and Brian Eno proved through their collaboration that staying in your comfort zone isn’t always the best option and that disruption can feed creativity. But Jarrett was famed for liking things just so…. would he risk humiliation in Cologne and play the broken piano or would he walk away?

Featuring: Archie Panjabi, Ed Gaughan, Rufus Wright, and Mircea Monroe.

Producers: Ryan Dilley and Marilyn Rust. Sound design/mix/musical composition: Pascal Wyse. Fact checking: Joseph Fridman. Editor: Julia Barton. Recording: Wardour Studios, London. GSI Studios, New York. PR: Christine Ragasa.

Thanks to the team at Pushkin Industries, Heather Fain, Mia Lobel, Carly Migliori, Jacob Weisberg, and of course, the mighty Malcolm Gladwell.

[Apple] [Spotify] [Stitcher]


Further reading and listening

I urge you to listen to Keith Jarrett’s Koln Concert, David Bowie’s “Heroes”, and Brian Eno’s Music for AirportsBut you should also listen to a superb oral history, “For One Night Only: the Koln Concert” produced by the BBC.

For a fuller exploration of the ideas in this episode I tentatively suggest my own book, Messy. Paul Trynka’s biography of David Bowie is Starman. Sasha Frere-Jones has a fine profile of Brian Eno in the New Yorker, but my main source is my own discussions with Brian.

The font study is : Diemand-Yauman, C., et al. “Fortune favors the bold (and the italicized): Effects of disfluency on educational outcomes.” Cognition (2010), DOI: 10.1016/j.cognition.2010.09.012

The murder mystery study is: Katherine W. Philips, Katie A. Liljenquist and Margaret A. Neale “Is the Pain Worth the Gain? The Advantages and Liabilities of Agreeing With Socially Distinct Newcomers.” Personality and Social Psychology Bulletin Vol 35 No 3 March 2009 p. 336-350

The tube-strike study is: Shaun Larcom, Ferdinand Rauch, Tim Willems, The Benefits of Forced Experimentation: Striking Evidence from the London Underground Network, The Quarterly Journal of Economics, Volume 132, Issue 4, November 2017, Pages 2019–2055, https://doi.org/10.1093/qje/qjx020



Receive these posts by email

(You can unsubscribe at any time)

Extend gratitude beyond platitudes

You know that blur of movement on Christmas morning when children meet presents and presents meet children, scraps of wrapping paper fly into the air and float down, confetti-style, all over the living room? Five minutes later, the presents are all unwrapped, the children sit panting, and everyone wonders what to do next.

It does not happen in the Harford household. For the past few years we have had a rule: you cannot unwrap the next present until you have written a thank-you note for the last one.

At first, this was merely my inner economist thinking about efficient incentives. I want my children to write thank-you letters and this generally requires some kind of bribe. At Christmas, that is easy: we are surrounded by gift-wrapped bribes. Using the next gift to incentivise the previous thank-you letter is an idea so elegant I am surprised it is not ubiquitous.

I then realised that this system had unexpected benefits. It forced us to slow down, to look seriously at each gift, to think about what was good about it, and to reflect on the giver. Gratitude as a chore became replaced by gratitude as a mindful counting of blessings.

The practice of gratitude is fashionable; some people advocate a daily gratitude journal as a way to get a quick shot of happiness. Robert Emmons, who studies gratitude, complains of the spread of “gratitude lite”, the sense of gratitude as an easy means to an end. He is the author of books including Gratitude Works! A 21-Day Program for Creating Emotional Prosperity (US) (UK), so perhaps he should take some ownership of this trend. But I know what he means about treating gratitude as a mere tool.

Gratitude is, after all, a complex thing. There is gratitude as mindfulness: noticing something good in the world. There is gratitude as politeness: remembering to say thank you to the shop assistant, even if you cannot be bothered to make eye contact. There is gratitude as reciprocity: writers from Adam Smith to Seneca underlined the idea of gratitude as a partial repayment of a debt.

A Catholic friend of mine says that gratitude is hard, like forgiveness. That seems true, when it is an attempt to find the silver lining to a lowering cloud. But gratitude often is, or should be, a simple matter of counting blessings in a world where there are many to count.

Violence and disease are rarer; most people are richer, healthier, and better educated than ever. There are still evils in the world, but not as many as 100 years ago. Why should gratitude be hard?

Nevertheless, it is. There are practical reasons why we pay more attention to what is difficult than what is easy. Problems require solving; non-problems do not. Psychologists call this “the headwinds / tailwinds asymmetry” inspired by the observation that a cyclist never notices when the wind is at her back. So being grateful requires some attention to what we easily ignore.

Mr Emmons has intensively studied the practice of keeping a gratitude journal, and he says that it is worth sticking with the habit, which gets easier. But rather than simply jotting down the same gratitude platitudes each evening, I’ve been thinking about expanding both the depth and the breadth of my thanks.

On depth, consider a study conducted by psychologists Martin Seligman, Tracy Steen, Nansook Park and Christopher Peterson. Participants were asked to write a letter expressing gratitude “to someone who had been especially kind to them but had never been properly thanked”. Then — this is the squirm-inducing bit — they hand-delivered the letter, read it to the recipient, and had a conversation. The warm glow of this process, for the letter writer, lasted for weeks. I have never hand-delivered such a letter. But I have written one, and it is a powerful experience.

As for breadth, expressions of gratitude are often about or directed to the usual subjects: family and friends, nice weather, good health. It is worth thinking more broadly — as did AJ Jacobs, author of Thanks A Thousand (US) (UK). He started by trying to be mindful of all the people around the world who contributed to the food he enjoyed, saying a sort of secular grace before each meal. But — challenged by one of his children — he decided to go much further, seeking out the many people who contributed to his morning coffee, and thanking them on the phone or in person.

Thanking the barista was easy, but he went on to thank the coffee taster, the lid designer, the pest control expert at the coffee warehouse, the farmers who grew the beans, the steel workers who made the pulping machine, the workers at the reservoir that was the source of the water, and about 1,000 others.

Writing a thoughtful thank-you letter when someone sends you a present requires far less imagination. It is the most elementary expression of gratitude. Still, it is not a bad place to start.


Written for and first published in the Financial Times on 28 Dec 2018.

My new podcast is “Cautionary Tales” [Apple] [Spotify] [Stitcher]

Free email updates

(You can unsubscribe at any time)

Why we fall for cons

There may be times and places where it’s a good idea to talk back to a military officer — but Germany in 1906 wasn’t one of them. So the young corporal didn’t. The corporal — let’s call him Muller — had been leading his squad of four privates down Sylterstrasse in Berlin, only to be challenged by a captain.  Captain Voigt was in his fifties, a slim fellow with sunken cheeks, the outline of his skull prominent above a large, white moustache. Truth be told, he looked strangely down on his luck — but Muller didn’t seem to take that in. Like any man in uniform, Captain Voigt appeared taller and broader thanks to his boots, smart grey overcoat and Prussian-blue officer’s cap. His white-gloved hand rested casually on the hilt of his rapier.

“Where are you taking these men?” he barked.

“Back to barracks, sir,” replied Muller.

“Turn them around and follow me,” ordered Voigt. “I have an urgent mission from the “all-highest” command.”

Direct orders from the kaiser himself!

As the small group marched towards Putlitzstrasse station, the charismatic Captain Voigt saw another squad and ordered them to fall in behind. He led his little army on a train ride towards Köpenick, a charming little town just south-east of the capital.

On arrival, the adventure continued: bayonets were to be fixed for inspection. It had been an extraordinary day for Corporal Muller and his men. But it was going to get a lot more extraordinary: what they were about to do would be the talk of newspapers around the world.


Captain Voigt’s impromptu strike force burst into Köpenick town hall and into the office of the mayor, a man named Georg Langerhans. Langerhans, a mild-looking fellow in his mid-thirties with pince-nez spectacles, a pointed goatee and a large, well-groomed moustache, stood up in astonishment and demanded an explanation. Voigt promptly placed him under arrest, by order of the kaiser.

“Where is your warrant?” stammered Langerhans.

“My warrant is the men I command!”

Voigt ordered the town treasurer to open the safe for inspection: fraud was suspected. The safe contained three thousand five hundred and fifty seven marks, forty-five pfennigs. Captain Voigt was punctilious about the count, confiscated the money, and handed over a receipt to be stamped.

It was nearly a quarter of a million dollars in today’s money.

Captain Voigt sent a pair of soldiers to find and detain Mayor Langerhans’s wife. She, too, was a suspect. He then searched the town hall office while his men kept the officials under arrest. Failing to find what he sought, he decided to wrap up the mission. The officials were to be driven to a police station where they would be detained and interrogated.

Captain Voigt himself walked to Köpenick railway station. He collected a package from the left-luggage office, and stepped into a toilet cubicle. A minute or two later, he stepped out again — and he was almost unrecognisable, having changed into shabby civilian clothes. He ambled, bandy-legged, across the station concourse. This anonymous fellow boarded the train back to Berlin, with his uniform neatly folded under one arm, and a bag of money under the other. Just like that, the “Captain of Köpenick” was gone.

Meanwhile, Corporal Muller dutifully presented his prisoners at the police station in central Berlin. The situation quickly became baffling to all concerned. Nobody had heard anything about the “all-highest” demanding the interrogation of the Mayor of Köpenick — nor his wife. After a phone call to headquarters, the head of the German general staff himself, General Helmuth von Moltke the Younger, arrived to resolve the situation. But nobody had received any orders from the kaiser. Nobody could see any reason to detain the mayor, or his wife, or his treasurer. And nobody could recall ever having met a “Captain Voigt” before.  No wonder. Except in the minds of the bemused soldiers and their civilian prisoners, Captain Voigt never existed. They met instead Herr Wilhelm Voigt, an ex-convict, an ex-shoemaker, a nobody, who possessed nothing more than a confident manner . . . and a very nice uniform.


The tale I just told you is a famous one in Germany. It became a play, and an Oscar-nominated film. (The most comprehensive English-language account I could find is by the historian Benjamin Carter Hett.) When the Germans tell the story they tend to linger on the prelude to the heist. What kind of a man does this? Who was Wilhelm Voigt, and what inspired his audacious confidence trick? Voigt was a crook, no doubt about it — his crimes included armed robbery. But the judicial system had treated him harshly, stuffing a legitimate appeal into a filing cabinet. In this version of the story, Voigt was persecuted by a cruel bureaucracy, driven to ransacking the mayor’s office looking not for money but for the paperwork he needed to get a job. No wonder he became seen as a sympathetic figure in German literature.

The English-speaking world drew a different lesson from the reports that filled their newspapers: that the Germans are a sucker for a shouty man in a uniform. The Morning Post named Voigt “the most humorous figure of the century”. The writer GK Chesterton could scarcely contain his glee upon reading the “comic” reports from Köpenick of the “absurd fraud (at least, to English eyes)”. An Englishman, mused Chesterton, would have seen through the bluster immediately.

Yet four years later, a group of young upper-class pranksters including the novelist Virginia Woolf and the artist Duncan Grant managed to arrange for a tour of the Royal Navy’s flagship, HMS Dreadnought, by putting on turbans, brown make-up and fake beards, and claiming to be from the royal family of Abyssinia.

“Bunga bunga!” they boomed as they greeted each other, and when they had to improvise further, they spoke scrambled fragments of ancient Greek poetry they’d learnt at school. Faced with this ridiculous, and to our modern eyes profoundly offensive prank, the Royal Navy responded with a commensurate display of ignorance: it treated the visitors with all the honour it could muster, including the flag and anthem of the nation of Zanzibar rather than Abyssinia. That was apparently close enough to satisfy everyone.

It’s easy to laugh — as GK Chesterton did — when it happens to someone else. But the closer I looked at the story of the Captain of Köpenick, the less funny it looks. Faced with the right con, we’re all vulnerable. Any one of us could have been the hapless Corporal Muller. And if we don’t understand how the trick worked, Wilhelm Voigt’s modern-day successors will do far more damage than he could ever have imagined.


Since Wilhelm Voigt persuaded people to obey orders that they should not have obeyed, you may already be thinking about Stanley Milgram. Milgram is the psychologist who, in the 1960s, conducted the most famous and controversial psychological experiment of all time — an experiment that I think we tend to misunderstand. Milgram recruited unsuspecting members of the American public — all men — to participate in a “study of memory”. On showing up at the laboratory, in a basement at Yale University, they met a man — apparently a scientist, just as Voigt had apparently been a Prussian army captain — dressed in a tie and grey lab coat.

“Very straightforward and professional, just what you’d expect from Yale,” one participant recalled. (Gina Perry’s book Behind The Shock Machine is an authoritative account of the experiments.)

The man-dressed-as-a-scientist supervised proceedings. Participants would be assigned the role either of “teacher” or “learner”. The learner was then strapped into an electric chair while the teacher retreated into another room to take control of a machine with switches labelled with terms including: “slight shock”, “moderate shock”, “danger: severe shock” and, finally, “XXX”.

As the learner failed to answer questions correctly, the teacher was asked to administer steadily increasing electric shocks. Although the teachers had received a painful shock themselves as a demonstration and had witnessed the learner complaining of a heart condition, many proved willing to deliver possibly fatal shocks while listening to screams of pain from the other side of the wall.  Of course, there were no shocks; both the screaming “learner” and the scientific supervisor were actors. The true experiment was studying the “teachers”: how far would they go when following direct orders?

In the best known study, 65 per cent of experimental subjects went all the way to 450 volts, applying shocks long after the man in the other room had fallen silent. Under the guise of science, Stanley Milgram had perpetrated yet another of these grim hoaxes.

Milgram’s research agenda was influenced by the shadow of the Holocaust and a desire to understand how it had been possible. He made the link explicit, and argued that his experiment was all about “obedience to authority”. But modern scientists no longer see Milgram’s research in quite that way.

There’s a lot we could say about those experiments — about their ethics, and about the more than 20 experimental variations. But the most fundamental objection is that these experiments may not be about obedience at all. Alex Haslam, a psychologist who has re-examined the studies in recent years, found that when the man in the lab coat gave direct orders, they backfired. One pre-scripted instruction produced universal disobedience: “you have no other choice . . . you must continue”. Experimental subjects concluded that this was simply untrue; nobody continued after that order. People need to be persuaded, not bullied, into participating.

So if these experiments weren’t about blind obedience, what were they about? Here’s a detail that is usually overlooked: Milgram’s shock machine had 30 settings, fine increments of 15 volts. It’s hard to object to giving someone a tiny 15-volt shock. And if you’ve decided that 15 volts is fine, then why draw the line at 30 volts? Why draw the line at 45? Why draw the line at all?

At 150 volts, the “learner” yelled out in distress. Some people stopped at that point. But those who continued past 150 volts almost always kept going to the full 450 volts. They were in too deep. Refusing to administer a shock of 225 volts would be an implicit admission that they had been wrong to deliver 210. Perhaps Stanley Milgram’s experiments weren’t a study of obedience so much as a study of our unwillingness to stop and admit that we’ve been making a dreadful mistake. We’re in too deep; we’re committed; we can’t turn back.

Think back to that day in Berlin, in 1906. Voigt stopped Corporal Muller in the street and demanded to know where he and his men were going. What was Muller to do? Demand proof of identification? Of course not. Muller didn’t want to risk a court martial over answering a simple question.

Voigt then asked Muller’s squad to follow him. That’s a bit more of a stretch, but Muller had already obeyed one order, already addressed this stranger-in-a-uniform as “sir”. Marching down the street behind him was just one small action further.

The pattern repeated itself with the second squad: when they first saw Captain Voigt, he was already at the head of half a dozen men; that was the evidence he was who he said he was. Why not fall in? Why not get the train to Köpenick? Why not fix bayonets for inspection? It’s really only at the moment that they burst into the town hall that the doubts might occur.

But by then, the whole business was already well beyond the 210-volt mark. They had travelled all the way across Berlin. They had been following Wilhelm Voigt’s instructions for a couple of hours. It would have been very late in the day for Corporal Muller, or anyone else, to have the presence of mind to stop, think and challenge their new captain.

Georg Langerhans, the young mayor, saw the situation very differently — he immediately demanded to see a warrant. Langerhans, of course, was effectively being asked to apply a 450-volt shock without preamble. No wonder he was sceptical.

At first glance, then, Wilhelm Voigt’s con and Milgram’s shock experiments are evidence for the idea that we’ll do anything for a figure of authority wearing the right outfit. But look deeper and they’re evidence for something else — that we’re willing to help out with reasonable requests, and that step by step we can find ourselves trapped in a web of our own making. Each small movement binds us more tightly to the con artist. We become complicit; breaking free becomes all but impossible.

That said, the right outfit matters. And here I want to think bigger than the world of the con artist. Yes, we fall for cons. But we fall for all kinds of other superficial things that shouldn’t matter, like a nice uniform, and those superficial things are constantly influencing our decisions — including decisions that we may later come to regret.


Almost exactly 110 years after Wilhelm Voigt’s audacious heist, Hillary Clinton and Donald Trump squared off in one of three televised debates. You might remember it. In a town-hall format, the candidates were able to roam the stage. And Trump certainly did roam, following Clinton around as she answered questions, looming behind her, always on camera, clearly visible over the top of Clinton’s head.

After the debate, that was all anyone could talk about. Was it an attempt at intimidation? Perhaps. But there’s something else about that footage of Donald Trump stalking Hillary Clinton: he towers over her.

Voters were being offered all kinds of choices in that election but one that was never really articulated was this: would you like to elect the third-tallest president ever, or the shortest president since James Madison two centuries ago?

There’s not much doubt that some voters were influenced by the disparity in height. The US does elect a lot of tall presidents. Trump was taller than Hillary Clinton. Obama was taller than McCain. Bill Clinton and George Bush Sr were the same height — towering over tiny Ross Perot, the feisty independent challenger they beat into third place. Bush Sr was taller than Dukakis. Reagan was taller than Carter, Nixon was taller than Humphrey, Kennedy was taller than Nixon, Truman taller than Dewey. Lyndon Johnson was taller than pretty much everyone. Are we electing a president here, or picking a basketball team? Of course there are some exceptions to the rule: when Carter beat Ford, it was a victory for the little guy.

But serious statistical analysis concludes that taller presidential candidates are more likely to win the election, more likely to win re-election, and more likely — unlike Donald Trump — to win the popular vote. Since the dawn of the television age, the only person ever to have overcome a height deficit of more than three inches was the incumbent George W Bush running against John Kerry.

Hillary Clinton would have been the first female president, true. She would also have been the first president to win despite a 10in height disadvantage since 1812. Americans may not have elected any female presidents over the years — but they haven’t elected any short men, either — not in a long, long time.

This isn’t just about presidential elections and it isn’t just about height. Across the world, voters favour candidates based on the most superficial characteristics imaginable. For example, one study — by economists Daniel Benjamin and Jesse Shapiro — found that people were fairly good at predicting the victor of an election for state governor after being shown a brief piece of video of a gubernatorial debate with the sound turned off: just looking at the candidates seemed to be enough to judge who voters would pick. In fact, giving people audio too actually made the predictions worse, presumably because it distracted them from what mattered: appearances.

We hairless apes seem to go for simple proxies when judging someone’s capacity for leadership. That 400-page manifesto? We’re not going to read it. But we pay close attention, whether we realise it or not, to the fine details of a candidate’s posture, styling, clothes — and, of course, height.  Corporal Muller and his men were completely taken in by Wilhelm Voigt’s appearance and mannerisms. But they’re not the only ones to pay attention to appearances.

Consider the advertising classic, “I’m not a doctor but I play one on TV.”  And then, as though it was the most natural thing in the world, the man who admits he isn’t a doctor goes on to tell us what brand of cough syrup to buy. Even Wilhelm Voigt would not have been quite as audacious as to announce: “I’m not a captain, I’m just wearing the uniform.”

And yet the advertisements work. We buy the cough syrup from the man who tells us, “I only look like a doctor”. That’s how powerful appearances can be. And what about “I’m not a successful businessman, but I play one on TV?” Oh — I think I know that guy.


Fraudsters using the playbook of Wilhelm Voigt trick people every day. First, they get the appearances right. Maybe it’s a text message that looks like it’s from your bank — the phone number is right, after all. Maybe the doorbell rings and the man is standing there with an official-looking ID; he wants to come and check your electricity meter. That ID does look genuine. Maybe it’s a smooth-talking politician with a good suit. Milgram well understood the need to get the clothes right. In a variation where the experimenter didn’t wear a lab coat, few people went to 450 volts.

Second, fraudsters put people into what psychologists call a “hot state”. We don’t think so clearly when we’re hungry, or angry, or afraid. Wilhelm Voigt yelled at Corporal Muller. A politician who wanted to put people into a hot state might announce that the country was being taken over by gangs and terrorists, and that his opponent should be locked up. Whatever works.

Third, they pull the heist one small step at a time. They start with the request for information: where are you taking these men? You are Ms Jane Doe, aren’t you? I’m sorry to report that your bank account has been compromised, Ms Doe. Just enter your password and username — just like you usually do — and we’ll sort it out for you.  Give us someone who looks or sounds the part; apply a bit of fear, anger, lust or greed; and then proceed in salami slices from the reasonable to the insane, so smoothly that we don’t stop to think. That’s how Wilhelm Voigt fooled Corporal Muller. But it’s how he would have fooled any of us, if he caught us at the wrong moment.

At first it looked as though Voigt would enjoy the fruits of his acting skills in peace. But as he relaxed with his money, a former accomplice of his saw the reports of the daring heist in all the newspapers and remembered a prison conversation in which Voigt had dreamt of such a coup. He promptly reported Voigt to the authorities.

When four detectives burst in to his apartment at six o’clock in the morning, they found Voigt enjoying breakfast. He protested that the timing was inconvenient. “I should like a moment to finish my meal.”

So the detectives watched him break open another crusty white roll, spread on a thick layer of butter, and wash it down with his coffee. You can’t help but admire the audacity.

At trial, Voigt became a folk hero. The judge sympathised with the way he had been treated, gave him an unexpectedly short sentence, then took off his judge’s cap and stepped down to clasp Voigt by the hand. “I wish you good health throughout your prison term, and beyond.”

The German authorities felt that — in light of the popularity of the Captain of Köpenick — even more ostentatious clemency was required. They pardoned him after less than two years in jail. The kaiser himself was said to have chuckled, “amiable scoundrel” at the deed.

Statues of Voigt were erected and waxworks made of him — including one in Madame Tussauds in London. He was paid to record his story so that people could listen to him recount his deeds. He went on tour, posing in his uniform and signing photographs of himself for money.

A local restaurateur begged him to come and dine as often as he wanted, free of charge, knowing that his presence would attract other customers. A wealthy widow gave him a pension for life. Never let it be said that the Germans lack a sense of humour. But while the comedy is undeniable, we should not be too fond of the Prussian prankster. Perhaps Wilhelm Voigt’s adventure did little harm in the long run. The same cannot be said for some of the con artists who followed in his footsteps. It is exciting to read about a fraud — from a distance. It is not so funny to live through one.


This article is based on Episode 2 of my new podcast,“Cautionary Tales”. [Apple] [Spotify] [Stitcher]

Published in FT Magazine, 16/17 November 2019.


Further reading

The best English-language account I could find of the Kopenick story is by Benjamin Carter Hett. “The ‘Captain of Köpenick’ and the Transformation of German Criminal Justice, 1891-1914,” Central European History 36 (1), 2003.

I first read about the story in Nigel Blundell’s The World’s Greatest MistakesOther accounts are at Strange History  and The Rags of TimeKoepenickia offers various contemporary German newspaper accounts. There are many small differences in the accounts but the overall story remains just as remarkable.

The definitive account of Stanley Milgram’s experiments is Gina Perry’s Behind the Shock Machine and Alex Haslam was interviewed by Radiolab in a great episode about the same topic.

An overview of the evidence on tall presidents is Gert Stulp, Abraham P. Buunk, Simon Verhulst, Thomas V. Pollet, “Tall claims? Sense and nonsense about the importance of height of US presidents” The Leadership Quarterly  Volume 24, Issue 1, 2013.

The study of gubernatorial elections is Daniel J Benjamin & Jesse M Shapiro, 2009. “Thin-Slice Forecasts of Gubernatorial Elections” The Review of Economics and Statistics, MIT Press, vol. 91(3), pages 523-536, 02.

Daniel Hamermesh’s Beauty Pays looks at the overall evidence that appearances matter – including in politics.

Receive these posts by email

(You can unsubscribe at any time)

Cautionary Tales Ep 6 – How Britain Invented, Then Ignored, Blitzkrieg

Blitzkrieg means “lightning war”, but despite the German name it was not a German invention. Back in 1917 a brilliant English officer developed a revolutionary way to use the latest development in military technology – the tank. The British army squandered the idea but two decades later later Hitler’s tanks thundered across Europe, achieving the kind of rapid victories that had been predicted back in 1917.

This is a common story: Sony invented the digital Walkman, Xerox the personal computer, and Kodak the digital camera. In each case they failed to capitalise on the idea. Why?

Featuring: Toby Stephens, Ed Gaughan and Rufus Wright.

Producers: Ryan Dilley and Marilyn Rust. Sound design/mix/musical composition: Pascal Wyse. Fact checking: Joseph Fridman. Editor: Julia Barton. Recording: Wardour Studios, London. GSI Studios, New York. PR: Christine Ragasa.

Thanks to the team at Pushkin Industries, Heather Fain, Mia Lobel, Carly Migliori, Jacob Weisberg, and of course, the mighty Malcolm Gladwell.

[Apple] [Spotify] [Stitcher]


Further reading

Mark Urban’s book The Generals has an excellent chapter on J.F.C. Fuller. Other sources on Fuller include Brian Holden Reid’s J.F.C. Fuller: Military Thinker and Harold Winton’s To Change An Army

Other sources on the development of the tank include Macksey and Batchelor’s TankNorman Dixon’s classic On The Psychology of Military Incompetence and Basil Liddell Hart’s The Tanks.

On modern corporate innovation try Gillian Tett’s excellent The Silo EffectCreation Myth” by Malcolm Gladwell, Clay Christensen’s Innovator’s Dilemma and The Disruption Dilemma by Joshua Gans.


The original paper on architectural innovation is:

Henderson, Rebecca M., and Kim B. Clark. “Architectural Innovation: The Reconfiguration of Existing Product Technologies and The Failure of Established Firms.” Administrative Science Quarterly 35, no. 1 (March 1990): 9–30



Receive these posts by email

(You can unsubscribe at any time)

A guide to having an actually happy Christmas

Is Christmas a time of magic, generosity and conviviality? Or of overconsumption, stress, and social anxiety? It is easy to make a case either way: listen to Paul McCartney’s “Wonderful Christmas Time”, followed immediately by Tom McRae’s slow sighing cover of the song and hear the same lyrics convey backslapping cheer and solitary despair.

Messrs McCartney and McRae illustrate the dilemma, but they do not resolve it. For that, we need data, so I consulted some academics on the slippery subject of “subjective wellbeing”, or as you or I would call it, “happiness”.

Two years ago, wellbeing researchers at the London School of Economics surveyed a panel of experts, asking them: “Do you think that populations on average have higher wellbeing during major festive periods like Christmas?” None of the respondents was particularly confident, but the verdict was that Christmas is a time for good cheer: 54 per cent thought that Christmas increased average wellbeing, with 18 per cent disagreeing and the rest sitting on the fence.

Fence-sitting is perhaps the wise choice here: the topic has been rather sparsely researched, and what studies do exist provide viewpoints as contradictory as Messrs McCartney and McRae. One recent piece of research by Michael Mutz found that “the Christmas period is related to a decrease in life satisfaction and emotional wellbeing”. An older study by Tim Kasser and Kennon Sheldon found instead that “subjects are on the whole reasonably satisfied with their holiday experience” and that while many people found Christmas a bit stressful, the majority did not. One thing we can say with confidence is that, contrary to the popular myth, suicide rates don’t spike at Christmas; they fall.

It may be more fruitful to ask about how different people experience Christmas — and whether we can suggest ways to enhance the joy and reduce the anxiety. One plausible hypothesis is that Christmas is an amplifier of existing inequalities. Those who are relaxed, have no money worries and a good relationship with friends and family should find plenty to enjoy in Christmas; those who are anxious, isolated or financially stretched may find Christmas makes everything worse.

An alternative view is that how we feel about the festival depends on how we approach it. Mr Mutz found that Christians felt happier at Christmas, while others felt less happy. Similarly Messrs Kasser and Sheldon found that people who spent more time with their families or engaging in religious practices tended to have a better time of things. Consumerism fared less well, according to Messrs Kasser and Sheldon; for all the money and effort buying and wrapping gifts, the activity “apparently contributes little to holiday joy”.

I am not sure atheists would feel better if they headed to church, nor that people who dislike their relatives should seek them out anyway. But these findings do suggest that the syrupy advice of a thousand moralising television specials — that the true spirit of Christmas is friends, family and the little baby Jesus — has something going for it.

What, then, is an undercover economist to advise for a truly merry Christmas?

First, keep the crass spending under control. It is pointless to lament the commercialisation of Christmas, which is not new. Santa Claus appears in advertisements from the 1840s, Macy’s was open until midnight on Christmas Eve in 1867, and Rudolf the Red Nosed Reindeer was invented in 1939 by an advertising copywriter at Montgomery Ward who needed a free gift for shoppers.

So don’t get mad with the marketing men: get even. Commercial spaces such as shopping centres and Christmas markets lay on the sights, sounds and smells of Christmas free of charge. If you like that sort of thing, savour the atmosphere, and don’t bother with the flashily-packaged trash nobody really wants.

You can cite economist Joel Waldfogel, author of Scroogenomics (UK) (US) if you like: he has convincingly demonstrated that many Christmas presents are poorly chosen. Or you can quote Harriet Beecher Stowe, if you prefer: “There are worlds of money wasted, at this time of year, in getting things that nobody wants, and nobody cares for after they are got.” In any case, if it is the thought that counts, then think.

Second, make your own social rituals, whether it is a regular reunion with old friends, carol singing, or church on Christmas morning. There is plenty of evidence that both religious and secular Christmas rituals can improve your enjoyment of the holiday. The difficulty comes in wading through the coagulated expectations of everyone else in your social circle. Take the time to think about what you really value, discuss it with your family, and make it happen.

Third: share the chores. Women have tended to spend considerably more time on the task of shopping for and wrapping Christmas gifts, while men seem to enjoy Christmas more than women do. This may not be a coincidence.

Fourth: be grateful, and write your thank-you letters. More on this next week. Finally, don’t listen to too much Tom McRae. Merry Christmas.

Written for and first published in the Financial Times on 21 Dec 2018.

My new podcast is “Cautionary Tales” [Apple] [Spotify] [Stitcher]

Free email updates

(You can unsubscribe at any time)


My books of the year 2019

Not all of them published this year – and of course the list is subjective.

Book that did most to change the way I thought – Caroline Criado Perez, Invisible Women. My long-time producer, Charlotte McDonald, has been trying to get me to engage with the “gender data gap” for ages, but I never really felt I got the problem. Perez has delivered a much needed correction: full of persuasive examples and analysis of areas from public policy, medicine, economics and elsewhere in which data have been gathered in such a way as to obscure or omit matters of most concern to women. I learned a lot.

Best book about numbers – David Spiegelhalter’s deep yet very readable The Art Of StatisticsSir David is a superb explainer of statistical concepts, and here he delivers much of the material one might find in a first-year undergraduate course on statistics – yet while managing to avoid most of the technicalities. The book is full of memorable examples and crystal-clear explanations.

Best book about catastrophe Meltdown by Chris Clearfield and Andras Tilcik. This book was up against a lot of competition, because I’ve been reading a lot about catastrophe recently – but Meltdown is fun, wide-ranging, vivid and full of clever observations. Two episodes of Cautionary Tales owe a debt to Meltdown and I strongly recommend the book.

Best book about numbers and catastrophe – Humble Pi by Matt Parker. Very funny, terrific storytelling, and despite some hair-raising tales very few people actually die. You’ll also learn about the mathematics behind all sorts of everyday technologies from Excel to a jumbo jet.

Best science fiction  The Last Day by Andrew Hunter Murray. I don’t read enough fiction but when I was sent an early copy of this book I found myself drawn in. Very elegantly done – a post-apocalyptic thriller in which the apocalypse is superbly inventive and the Orwellian police state brilliantly low-rent. It’s about Brexit without being about Brexit, about climate change without being about climate change, and I very much enjoyed it.

Best picture book – Sandman by Neil Gaiman. I’ve never read this classic series and after an enjoyable but slightly schlocky first volume it quickly finds its epic, whimsical, endlessly inventive stride. I’ve been reading a book every couple of months all year; what a joy. Honourable mentions.

Best coffee-table book – The Brick: A World History. I don’t own a coffee table, but goodness me this book is gorgeous.

Best business book  Range by David Epstein. Epstein nails the difficult mix of argument, evidence and story. His book is a persuasive argument for not settling down or focusing too narrowly. I explored some of the same issues in Messy but even so I ended up learning much that I didn’t know.

Best computer science book – Hello World by Hannah Fry. An expert but highly accessible account of what algorithms do, how they work, and what they’ll do to the world around us. Great fun and a model of crisp explanation.

Best self-help book – Digital Minimalismby Cal Newport, by a long long way. I would never have found the time to read anything else if I hadn’t read this book last Christmas. Newport eschews tips and hacks and instead demands that we face up to our digital habit and make far more deliberate choices. I cannot recommend this book too strongly.


Receive these posts by email

(You can unsubscribe at any time)

8th of December, 2019ResourcesComments off

Cautionary Tales Ep 5 – Buried by the Wall Street crash

Two of the greatest economists who ever lived, Irving Fisher and John Maynard Keynes, thought they could predict the future and make a killing on the stock market. Both of them failed to see the Wall Street crash, the greatest financial disaster of the age – and arguably, of any age. Yet having made the same forecasting error, Fisher and Keynes went on to meet very different fates. What does it take to see into the future? And when you fail, what does it take to bounce back from ruin?

Featuring: Alan Cumming, Russell Tovey, Mircea Monroe, Rufus Wright, Ed Gaughan, and Melanie Gutteridge.

Producers: Ryan Dilley and Marilyn Rust. Sound design/mix/musical composition: Pascal Wyse. Fact checking: Joseph Fridman. Editor: Julia Barton. Recording: Wardour Studios, London. GSI Studios, New York. PR: Christine Ragasa.

Thanks to the team at Pushkin Industries, Heather Fain, Mia Lobel, Carly Migliori, Jacob Weisberg, and of course, the mighty Malcolm Gladwell.

[Apple] [Spotify] [Stitcher]


Further reading


Walter Friedman’s The Fortune Tellers is a key source on Fisher. It’s a history of all economic forecasting in the US. I loved it.

Sylvia Nasar’s excellent Grand Pursuit has much more on both Keynes and Fisher.

There are several fine journalist accounts of Keynes’s participation in the Degas auction. Try the BBC, the Wall Street Journal, or History Today.

On Keynes, the central source on his investment performances is David Chambers and Elroy Dimson. 2013. “Retrospectives: John Maynard Keynes, Investment Innovator.” Journal of Economic Perspectives27 (3): 213-28.DOI: 10.1257/jep.27.3.213. There’s more biographical detail in the more informal Keynes’s Way To Wealth by John Wasik.

Philip Tetlock’s original study is detailed in his subtle, scholarly and ground-breaking Expert Political JudgmentHis more recent book with Dan Gardner, Superforecasting is more journalistic and covers his recent discoveries. Both books are very good, but quite different in style.

The case of Dorothy Martin and the UFO cult is told first hand by Festinger and his colleagues in When Prophecy FailsThere’s further discussion in Mistakes Were Made (But Not By Me)an excellent guide to all the ways in which we can fail to notice we’re wrong, by Carol Tavris and Elliot Aronson.


Receive these posts by email

(You can unsubscribe at any time)

How to survive an election with your sanity intact

A week to go — or eleven months, if you’re a US voter — and the time has come to share with you my handy guide to surviving an election.

Step one: think about your goals. Mine are to keep my cool, keep my friends, learn a little about the world and cast my vote wisely. You might well share these goals — but bear in mind that most of the people you will encounter on the news or on social media have very different aims in mind: they would like you to be excited, if not downright angry. Therein lie the clicks, the views and sometimes the votes, too.

It follows that we need to be thoughtful about what sort of political news we watch and read. There is plenty of excellent analysis out there, but one needs to seek it out. Twitter has its merits as well as its faults, but it has been a while since I saw a really good explainer go viral on Twitter.

Step two: find out about the issues. I can think of a good newspaper that provides detailed analysis of policies, but it is not the only source. There is a rich seam of blogs run by academics, while think-tanks such as the Institute for Fiscal Studies, the Resolution Foundation, The King’s Fund and many others have claim — justifiably — to provide unbiased and dispassionate analysis. Others clearly lean to the political left or right, but even those are far more likely to educate you about a given policy issue than watching the television news or reading your Facebook feed.

I realise that nobody is going to plough through all those policy papers. Instead, pick a topic that matters to you — maybe climate change, maybe Brexit, maybe healthcare — and read a few pages of wonkery. I’d be surprised if you don’t learn something both interesting and valuable within five minutes.

Step three: don’t obsess about all the lies. The use of the lie in politics is mutating. Once politicians made questionable claims in the hope that the deceit would pass unchallenged. These days, one of the weapons of political warfare is to make a false claim in the full expectation that it will be rebutted, and the outrage about the lie will crowd out other stories. (See also: “£350m a week for the National Health Service”.) By all means shoot down the lie — but then move on.

There are people whose heroic task is to fact-check all the important claims made in media interviews, debates, election leaflets and on social media. Given that some fairly dark propaganda can be quietly circulated on social media, this is not an easy task. As Joseph O’Leary, senior fact-checker at the UK charity Full Fact observes: “fact checkers are only as good as the claims they notice”. There is also the “bullshit asymmetry” principle: it takes 10 times as much effort to refute bullshit as it does to produce it.

Professional fact-checkers could be forgiven for being outraged at the task our political discourse has handed them. Yet the best of them are rigorous, fair, transparent — and careful not to amplify false claims by endlessly repeating them as part of a fact-check.

Step four: vote tactically. Yes, it would be nice to have a rational electoral system, but we don’t. So check out how the votes went in your local area in 2017 and 2015. (For goodness’ sake, don’t trust the bar graphs on any election leaflets shoved through your door.) In most cases, the choice is simple: pick whichever you prefer of the two leading candidates in your area. For extra credit, you might try to find out whether the incumbent lies on the sane or insane wing of their own party — although many of the sane incumbents seem to be quitting politics, which is not encouraging.

I realise it might be tempting to exercise a protest vote, and that is every voter’s right. But bear in mind that the smug glow of ideological purity had better feel pretty good to make it worthwhile, because some of those protest votes are going to prove awfully counterproductive.

Step five: if you’re having a conversation about politics, try to learn something. There is no point in having a shouting match with friends and neighbours, and it is equally fruitless to sit around with like-minded people commiserating with each other about how terrible the other lot are. Why not, instead, ask what people have found most noteworthy about the campaign? An intriguing person, maybe, or a weird policy, a columnist or a podcast they’d recommend? When someone expresses an opinion, whether you agree or disagree, ask them to elaborate.

Be curious. You might learn something — and the psychological research suggests that they might learn something, too. In my own fond recollection — which is, no doubt, a nostalgic delusion — politics used to take the form of an argument between reasonable people about the best way to solve the country’s problems. If it is now evidence-free rather than evidence-based, insulting rather than respectful, destructive rather than constructive, then that’s something we need to change.

And since I can’t control what everyone else does, I suppose I’ll have to start by changing myself.

Written for and first published in the Financial Times on 15 November 2019.

My new podcast is “Cautionary Tales” [Apple] [Spotify] [Stitcher]

Receive these posts by email

(You can unsubscribe at any time)


  • 1 Twitter
  • 3 RSS
  • 5 Podcasts
  • 6 Facebook


  • Fifty Inventions That Shaped the Modern Economy
  • Messy
  • The Undercover Economist Strikes Back
  • Adapt
  • Dear Undercover Economist
  • The Logic of Life
  • The Undercover Economist

Free Email Updates

Enter your email address to receive notifications of new articles by email (you can unsubscribe at any time).

Join 188,947 other subscribers.