How the Brexit debate was flushed down the drain

On a scale of one to seven, how well do you understand how a flush lavatory works?

This was a question asked by two Yale psychologists, Leonid Rozenblit and Frank Keil, almost two decades ago. Before I explain why, here’s a follow-up exercise: write down your lavatory explanation in as much detail as you can. You may wish to draw a diagram, or explain it to a friend. Or not.

You may then reflect that you knew a little less than you realised. That was the experience of many of the study’s subjects — and not just for lavatories (why does all the water disappear down the U-bend?) but also for zips, quartz watches, helicopters, speedometers, cylinder locks, piano keys and sewing machines. People felt they understood the mechanisms that surrounded them, but their confidence was severely dented by the simple act of giving them pencil and paper and saying: “Show me.”

The same exercise can be performed with politics. In 2013, Steven Sloman and Philip Fernbach, authors of The Knowledge Illusion, were members of a research team that did just that, inviting people resident in the US to rate their understanding of American policy proposals such as introducing unilateral sanctions on Iran, a cap-and-trade system for carbon emissions and a national flat tax. They also asked people to rate their approval of each policy, which would have been unnecessary for lavatories and zips. (Lavatories are useful, zips self-evidently malevolent.)

Professors Sloman and Fernbach and their colleagues found that — just as with locks and speedometers — people tended to overrate their knowledge at first, and then discover some humility when asked to be more specific.

Perhaps British voters could use a dose of the same medicine when it comes to our understanding of Brexit. Leave or Remain, many of us came late to the realisation that there was a difference between the single market and the customs union. I am still not sure most people can explain what that difference is. Many people have strong views about Prime Minister Theresa May’s withdrawal agreement; rather fewer could give a convincing account of how it differs from the political declaration that accompanies it.

When Mrs May began her premiership with the statement that “Brexit means Brexit”, it dawned on most of us that the details of the whole project might need a little more work. But she wasn’t the only one who was vague.

I’d love to see the contenders for the Conservative party leadership quizzed a little less about their cocaine habits and instead forced to sit down and write a detailed explanation of what a no-deal Brexit actually is. While we wait, perhaps the same exercise could be given to the 160,000 Conservative party members who are about to select the country’s next prime minister.

How long, for example, will HM Revenue & Customs wave through imports without inspections? Will the French reciprocate? What are the implications of “trading under World Trade Organization rules” for the UK’s banking and insurance industries? How large are those industries?

How many other developed countries are content to rely solely on WTO arrangements in their trade with the EU? Is the WTO capable of enforcing the rules anyway, given the current crisis in its appellate body? How likely is the EU to grant permission to British farmers to sell meat, milk or cheese? Would any of these decisions be different if the UK refused to pay the “divorce bill” it had negotiated?

I don’t think it is especially shameful that we ordinary voters are incurious about the ins and outs of Brexit, any more than we should be obliged to understand the workings of a quartz watch. An ability to read the time is generally sufficient. But I am stunned by just how little we seem to demand of our political leaders.

We want tailors to understand sewing machines, locksmiths to understand locks and plumbers to know that a lavatory is basically a siphon. But our standards for politicians seem far lower. The next prime minister is likely to be a person who believes that if we demanded it with enough gusto, sewage would remove itself from our homes in some scatological remix of Mary Poppins — and that anyone who tells you otherwise is clearly a shill for Big Porcelain.

We should expect more of anyone who wants to lead the country. And since our politicians have grown so fond of punting the hard questions back to us, perhaps we should also demand more of ourselves.

Profs Sloman and Fernbach found that asking people to explain the workings of the policies they so fervently supported or opposed had a humbling effect. When people realised that they knew less than they had once believed, they quite reasonably wound their necks in as a result. It seems strange to die in a ditch for something we can’t clearly explain, even to ourselves.

Next time you find yourself in some heated political debate, perhaps you should suggest that both sides pause to explain the policy in question. You may find you understand less — and agree more — than you realised.

Written for and first published in the Financial Times on 21 June 2019.

My book “Fifty Things That Made the Modern Economy” (UK) / “Fifty Inventions That Shaped The Modern Economy” (US) is out now in paperback – feel free to order online or through your local bookshop.

How the US is weaponising the world economy

Back in 2002, serious people were worried about the possibility of a nuclear exchange between India and Pakistan. Millions might have died — and the prospect seemed real enough that both the US and the UK advised their citizens to flee the region. How, then, was the crisis defused?

Thomas Friedman, author of The World Is Flat, is fond of telling the story that US businesses (in particular Dell) told their Indian suppliers (in particular Wipro) to calm things down or get cut out of the loop. And things did indeed calm down, so perhaps it was the concerns over Dell’s supply chain that prevented catastrophe. Perhaps.

Mr Friedman duly coined the phrase “the Dell Theory Of Conflict Prevention”: no two countries will go to war if they are part of the same global supply chain. He was never entirely serious about that, but the question now arises: did he have it backwards? Rather than a line of defence against hostile action, might global supply chains be a line of attack?

One example is the recent executive order banning US companies from working with Huawei, effectively denying the Chinese telecoms company the use of Qualcomm’s chips and Google’s Android operating system. Another was Mr Trump’s crude — and fleeting — threat to slap tariffs on Mexico if it didn’t satisfy him on immigration policy.

It is tempting to view such actions as uniquely Trumpian. Would any other president threaten sanctions against one of its largest trading partners, via Twitter, with 10 days’ notice, at the very moment they were presenting the new Trump-championed trade agreement to the Mexican senate?

Yet while a different president might act with more subtlety, the US seems unlikely to abandon the aggressive tweaking of the nerves and sinews under the skin of the world economy. Henry Farrell and Abraham Newman — political scientists at George Washington and Georgetown Universities, respectively — have popularised the term “weaponised interdependence”, the title of a forthcoming article in the journal International Security.

Messrs Farrell and Newman point out that supply chains and digital networks can be used both as a “panopticon” to see everything that happens and as a “chokepoint”, denying access to some vital service. Both approaches require a certain bureaucratic apparatus — something that would be hard to disassemble once in operation. There is more going on here than the whim of “Tariff Man”.

Consider Swift, the international financial messaging system. Although Swift does not directly handle transfers of money between banks, it provides the secure service that makes those transfers possible. Swift is a private company based in Brussels, yet last summer it found itself on the receiving end of US demands to cut off Iranian banks. The EU, in turn, demanded that it did not comply. Forced to pick a side in this tug of war, late last year it picked the US.

That is an indication of just how much power the US can wield if it is determined to do so. And the temptation is strong: it seems far safer to attack Iranian interests through stern letters to a messaging service in Brussels than with a carrier strike group.

That very temptation, of course, risks over-reach. The US is not the first global superpower to ponder the use of financial and communication networks as a weapon of war. In the early 20th century, modern economies were increasingly underpinned by complex financing. Britain viewed the central role of the City of London in the world’s banking, telegraph and marine insurance system as potentially decisive when coupled with the power of the Royal Navy. Should war break out with Germany, these networks could be used to sustain the UK economy while crushing that of Germany.

The idea seemed plausible, but needless to say, these plans for economic shock and awe failed both to head off the first world war and to limit its duration and brutality.

Two questions arise: would the US be wise to use its economic leverage more sparingly? And should other nations be building alternative networks beyond the hegemon’s gaze and grip?

It is too easy to say that the US should restrain itself in its own enlightened self-interest. The logic of network effects is self-reinforcing. Having established a central position in finance through Wall Street and the mighty dollar, and in digital networks thanks to Silicon Valley and the Pentagon’s role in funding the early internet, the US advantage may endure many abuses.

Still, the more the US seeks to coerce others through its privileged position in banking and the internet, the greater the incentive to develop alternatives that cut the US out of the loop — for example, a Chinese-built operating system for smartphones, or Instex, the special purpose vehicle launched by France, Germany and the UK to allow companies to do business with Iran beyond the reach of US punishment.

Wolfgang Munchau described Instex as “a dysfunctional insurance vehicle for small carpet traders”; it turns out that these alternatives are not cheap or easy to develop. In ordinary circumstances, few would even bother to try. But if the US presses too painfully on the global economy’s nervous system, its rivals and even its allies will look for relief.

Written for and first published in the Financial Times on 14 June 2019.

My book “Fifty Things That Made the Modern Economy” (UK) / “Fifty Inventions That Shaped The Modern Economy” (US) is out now in paperback – feel free to order online or through your local bookshop.

What to do when blessings come well-disguised

Keith Jarrett’s 1975 concert in Cologne should have been a musical catastrophe. Owing to a string of mix-ups and bad luck, he was faced with the choice of attempting his widely admired improvisations on a beaten-up old piano with sticky keys and a harsh upper register — or walking out altogether. He was all for walking out, but felt sorry for the concert promoter and agreed to play the unplayable piano against his better judgment. The result was not a catastrophe but a masterpiece, and a bestselling one at that: The Köln Concert album.

I’m fond of that story, of the way that an obstacle can unleash a creative response — partly by concentrating the mind and partly by forcing the artist to explore fresh approaches. It’s not the only such tale. The great jazz guitarist Django Reinhardt developed his distinctive style after suffering severe burns that left him with only two fully functioning fingers on his fretting hand.

A quotidian parallel is the Tube strike of 2014, which partially shut down the London Underground, prodding several thousand commuters into finding better routes into work.

The three stories share another common element: nobody at the time would have predicted that anything good would come of the problem. Keith Jarrett could have sought out a bad piano; young Django could have tied two fingers together; commuters could have tried a new route any day. But they didn’t, until there was no choice. The blessings came well disguised.

David Epstein’s new book, Range, has brought other instances to my attention. Range is mostly about the benefits of being a generalist rather than a specialist, but a recurring theme is of teaching or training techniques that seem to fall flat but pay off in the long run.

For instance, in the US Air Force Academy, students are initially assigned at random to academic instructors, then later randomly reassigned to follow-on courses — a nice natural experiment. Scott Carrell and James West looked at the data and found snatches of unplayable piano in it: students assigned the “best” instructors for introductory courses, as measured by short-term pupil performance, earned the highest student evaluations but went on to produce the worst results in the long run.

In a nutshell, teachers who “teach to the test”, or otherwise provide simple problem-solving procedures, immediately improve grades and their students thank them for it. But in the long-term, the students suffer from not having been forced to think more deeply.

This isn’t just about pedagogical techniques. One study that stuck with me when I was researching my own book, Messy, was by Katherine Phillips and other psychologists. The researchers set small groups working on a problem before introducing a new team member to help, sometimes a stranger and sometimes a familiar face. The groups forced to work with the stranger were much more likely to solve the problem, but also enjoyed the experience less and sharply underrated their success. Groups of friends did much worse, but had fun and were under the illusion that they had done a good job.

Again the striking thing is not just that the obstacle turned out to be helpful, but that nobody thought so at the time. Only with hindsight did people realise.

Three more examples. Sports scientists now suggest that top endurance athletes should keep most of their training at low intensity, resisting the temptation to train too hard. Traffic planners are familiar with Braess’ Paradox: closing a road can and often does improve traffic flow, even if the number of journeys stays the same. The closure prevents drivers from choosing routes that make sense for them but cause congestion for others.

The inverse is even more common: a pleasing intervention that does no good. For instance, preliminary research suggests that using virtual reality to promote empathy — for example, for refugees, homeless people or those with disabilities — can promote a surge of emotional support in the short run without delivering any long-run insight.

We shouldn’t leap to the conclusion that every blessing comes well disguised, and that every problem is an opportunity. Sometimes blessings are perfectly apparent, and we should remember to count them. Sometimes a problem is simply a problem. But the prevalence of these counterintuitive results is a reminder that we know less about the world — and about our reaction to it — than we like to think.

I draw three lessons. The first is the need to gather solid evidence — for example, by running randomised trials of teaching techniques — and to ensure that the evidence looks at a good range of outcomes over a long enough time to be meaningful. Common sense can lead us astray, especially in situations where short-term pain leads to long-term gain.

The second lesson is to experiment with more variety in our own lives — whether choosing a holiday or a commuting route. It shouldn’t take industrial action to prompt us to try an alternative route to work.

The final lesson is the simplest. When life confronts us with an unplayable piano, perhaps we should sit down and try to play it.

Written for and first published in the Financial Times on 7 June 2019.

My book “Fifty Things That Made the Modern Economy” (UK) / “Fifty Inventions That Shaped The Modern Economy” (US) is out now in paperback – feel free to order online or through your local bookshop.

The dying art of compromise

I don’t often find myself agreeing with Esther McVey, but I wondered this week whether the candidate for leader of the UK Conservative party might accidentally have spoken the truth: “People saying we need a Brexit policy to bring people together are misreading the situation. That is clearly not possible.”

The British do indeed seem in no mood to compromise. The results of elections to the European Parliament produced a thunderous endorsement of parties that proudly reject an attempt to find common ground on Brexit. The Conservatives and Labour, each caught in an awkward straddle, were slaughtered. Labour offered the slogan “let’s bring our country together”. Ha! Voters preferred the Liberal Democrats (“Bollocks to Brexit”) and the Brexit party (“they’re absolutely terrified of us”).

Sometimes an extreme position is the correct one. When King Solomon proposed cutting the baby in half, it wasn’t because he was looking for the middle ground.

Yet a capacity to find compromises is a good thing to have. Positions may differ, but whether we live in the same home or on the opposite side of the planet, we benefit when we can find a way to get along.

If this new distaste for compromise is a problem, it is not the UK’s alone. Positions seem to be hardening everywhere, the sclerotic arteries that may lead to a heart attack for western democracies. Perhaps this is driven by personalities. For a man whose name adorns a book titled The Art of The Deal, Donald Trump is curiously uninterested in negotiating lasting agreements with anyone. Or maybe it is a function of an information ecosystem in which outrage sells.

Perhaps the problems themselves are more intractable. Some issues do not lend themselves to compromise. Brexit is one. Splitting the difference between Remainers and hard Brexiters is less like cutting a cake and more like splattering its ingredients everywhere. Egg on my face, flour on yours, and nobody even partially satisfied.

Abortion is another. There is a principled case to be made for a woman’s absolute right to control her body. There is also a principled case to be made for the absolute right to life of a foetus. But like the unstoppable cannonball and the immovable post, both rights cannot be absolute simultaneously.

In contrast, other complex and emotive problems may still allow for compromise. On climate change, we can shrug and do nothing, or we can turn our economic system upside down, but there is plenty of middle ground between those options. In a trade negotiation, a mutually advantageous outcome is almost always there to be discovered.

Roger Fisher and William Ury’s classic negotiation handbook Getting to Yes advises: focus on the problem rather than the personalities; explore underlying interests rather than explicit positions; and consider options that may open up scope for mutual benefit. We may find a much better way to split the cake if we discover that you scrape the icing into the bin, while I would happily eat it with a spoon. It is sometimes astonishing how far a principled negotiation can go towards giving both sides what they want.

It is clear that we British have failed to follow this advice. Our debate is driven by a bitter focus on personalities, from Theresa May to Nigel Farage to Jeremy Corbyn to the generic “Remoaner elite”. Each side knows what the other wants but has shown very little interest in why they want it. Without sincerely exploring the underlying aims and values of warring tribes there is no chance of finding an outcome everyone can accept.

The US debate also seems the antithesis of Fisher and Ury’s advice. Too many politically active people seek the humiliation of the other tribe. Dismissing compromise as craven appeasement seems to be a winning tactic, particularly in the primary elections that set the tone of US politics.

Compromise, however, is often possible even in unpromising situations. On abortion, for example, it emerges with a focus not on absolute rights but on practicalities. Many people can get behind policies to minimise unwanted pregnancies, and to make abortions safe and regulated rather than dangerous and illicit. It is a middle ground that many countries manage to find.

One can see politics as a competitive sport or a search for solutions. There’s truth in both views. However, a democratic election is far closer to a competition than to a principled negotiation. Do we not wish to see the opposite team soundly thrashed? Do we not boo their villainous antics and laugh at their mishaps? Who wants to play out a nil-nil draw?

I would not want to venerate compromise as the supreme good in politics. Sometimes it really is true that you and I, dear reader, are absolutely right and they are absolutely wrong. (It may even be true that we are absolutely wrong and they are absolutely right.) Either way, the merits of the case must be weighed against the merits of trying to respect everyone. It feels good to win, but this isn’t a fairytale: the losers won’t stamp their feet and vanish through the floor. They — or we — aren’t going anywhere.

Written for and first published in the Financial Times on 31 May 2019.

My book “Fifty Things That Made the Modern Economy” (UK) / “Fifty Inventions That Shaped The Modern Economy” (US) is out now in paperback – feel free to order online or through your local bookshop.

Why brilliant people lose their touch

It hasn’t been a great couple of years for Neil Woodford — and it has been just as miserable for the people who have entrusted money to his investment funds. Mr Woodford was probably the most celebrated stockpicker in the UK, but recently his funds have been languishing. Piling on the woes, Morningstar, a rating agency, downgraded his flagship fund this week. What has happened to the darling of the investment community?

Mr Woodford isn’t the only star to fade. Fund manager Anthony Bolton is an obvious parallel. He enjoyed almost three decades of superb performance, retired, then returned to blemish his record with a few miserable years investing in China.

The story of triumph followed by disappointment is not limited to investment. Think of Arsène Wenger, for a few years the most brilliant manager in football, and then an eternal runner-up. Or all the bands who have struggled with “difficult second-album syndrome”.

There is even a legend that athletes who appear on the cover of Sports Illustrated are doomed to suffer the “SI jinx”. The rise to the top is followed by the fall from grace.

There are three broad explanations for these tragic career arcs. Our instinct is to blame the individual. We assume that Mr Woodford lost his touch and that Mr Wenger stopped learning. That is possible. Successful people can become overconfident, or isolated from feedback, or lazy.

But an alternative possibility is that the world changed. Mr Wenger’s emphasis on diet, data and the global transfer market was once unusual, but when his rivals noticed and began to follow suit, his edge disappeared. In the investment world — and indeed, the business world more broadly — good ideas don’t work forever because the competition catches on.

The third explanation is the least satisfying: that luck was at play. This seems implausible at first glance. Could luck alone have brought Mr Wenger three Premier League titles? Or that Mr Bolton was simply lucky for 28 years? Do we really live in such an impossibly random universe?

Perhaps we do. Michael Blastland’s recent book, The Hidden Half, argues that much of the variation we see in the world around us is essentially mysterious. Mr Blastland’s opening example is the marmorkrebs, a kind of crayfish that reproduces parthenogenetically — that is, marmorkrebs lay eggs without mating and those eggs develop into clones of their mothers.

Place two clones into two identical fish tanks and feed them identical food. These genetically identical creatures raised in apparently identical environments produce genetically identical offspring who nevertheless vary dramatically in their size, form, lifespan, fecundity, and behaviour. Sometimes things turn out very differently for no reason that we can discern. We might as well call that reason “luck” as anything else.

This is not to say that skill doesn’t matter — merely that in a competition in which all the leaders are highly skilled, randomness may explain the difference between triumph and failure. Good luck plus skill beats bad luck plus skill any time.

It is easy to underestimate how much chance is at play all around us. The psychologist Daniel Kahneman has recently been studying what he calls “noise”: the variability of judgments for no obvious reason.

A wine expert blind-tasting two glasses from the same bottle will often rate them differently. Pathologists disagree with each other in their judgments of the same biopsy. More disconcertingly, they also disagree with their own prior judgments of the case.

We rarely appreciate just how much inconsistency there is in the judgments we and others make, argues Prof Kahneman. It can hardly be a surprise, then, if past performance is no guarantee of future success.

We should remember, too, that people often achieve outsized success by taking risks or being contrarian. When John Kay examined the forecasting record of economists in the 1990s, he noted that Patrick Minford, an idiosyncratic forecaster, would often produce the best forecast one year and the worst forecast the next. If the consensus is wrong, being an outlier gives you a high chance both of dramatic success and spectacular failure.

We perceive all this randomness through a particular filter, too. Few people make the cover of Sports Illustrated after a run of mediocre luck. They appear after things have been going well, and if the good luck fails to hold then it seems like the SI jinx. More likely it is “regression to the mean”, or in simple terms, a return to business as usual.

We begin paying attention only when someone is producing a remarkable performance. Genius followed by mediocrity is a story arc we all notice. Mediocrity followed by genius just looks like genius — assuming the mediocre performer gets a second chance. Not all do.

So I wish Mr Woodford well. Perhaps he has lost his touch, perhaps the world has changed, or perhaps he has simply been unlucky. It would be nice to know which, but in such matters the world does not always satisfy our curiosity.

Written for and first published in the Financial Times on 24 May 2019.

My book “Fifty Things That Made the Modern Economy” (UK) / “Fifty Inventions That Shaped The Modern Economy” (US) is out now in paperback – feel free to order online or through your local bookshop.

The Doris Day effect – when obstacles help us

She had hoped to become a ballet dancer. After her leg was shattered in an accident at the age of 15, she took singing lessons instead. It was a striking detail in the obituaries. If not for that painful setback, the star that was Doris Day would never have risen.

Was the car accident that redirected her career an extraordinary twist in the story of an extraordinary life? Or was it typical of some broader truth about life, that frustrations can actually help us? Perhaps it is true that what does not kill us makes us stronger. It may, in contrast, be that what does not kill us nevertheless slows us down.

The conventional wisdom is that initial advantages tend to snowball into an avalanche of privilege. Sometimes this reflects genuine achievements: a bit of luck with an early teacher sharpens a student’s skills, lifting her into a higher set, which in turn gets her into a better university, then a job with more stimulating peers, and so on.

An egregious example, made famous by Malcolm Gladwell’s book Outliers, is the tendency of elite athletes to be born early in their school year. Being a few months older at the age of five means you are stronger and faster, are more likely to be picked for school teams, get more practice and are still reaping the benefits as an adult athlete. The effect is particularly well-studied among boys playing ice hockey in Canada, and football in a variety of countries.

At other times, well-deserved acclaim is followed by unearned praise. In academia this tendency was named by the sociologist Robert K Merton as “the Matthew Effect” in reference to a biblical verse: “For to every one who has will more be given, and he will have abundance; but from him who has not, even what he has will be taken away.”

If three researchers collaborate on a problem, and one of them already has a Nobel Prize, the laureate tends to earn disproportionate recognition for the joint work. When a teacher and a student work together, the senior researcher is cited because that name is already recognised. The junior is easily forgotten.

In the wider workplace, we have evidence that the luck of graduating in a benign economic climate can lead to a lasting advantage. One researcher, Paul Oyer, found that young PhD and MBA students who started off in favourable job markets were employed in better places with smarter colleagues, and were still doing better a decade later than those who started out in tougher times.

Hannes Schwandt and Till Marco Von Wachter studied the other end of the US labour market to find the story is even worse there: entering the job market during a recession damages anyone’s prospects, but the harm is deeper and lasts longer for less-educated and otherwise disadvantaged groups.

All this suggests that setbacks are setbacks: they drag us down, perhaps disproportionately. Doris Day was an exception, not the rule.

Yet a striking new study suggests that the Doris Day effect is quite real in one particular group of people: young scientists applying for research grants. Yang Wang, Benjamin Jones and Dashun Wang looked at scientists applying for funding from the US National Institutes for Health, with grants averaging $1.3m. In particular, they focused on borderline decisions, comparing those who scraped through to get a grant with those who just missed out. The near-winners and the near-losers were otherwise indistinguishable before the decision point, but afterwards it was the losers who prospered, publishing substantially more highly cited research papers.

We should remember that anyone in a position to nearly secure a million-dollar research grant has presumably enjoyed a few successes along the way at school and university. Failure at this hurdle may be described as an “early career setback”, but it is not comparable to the setback suffered by an undernourished two-year-old with no books in her bedroom.

Still: this is a counterintuitive finding. Yet I was not entirely surprised to encounter it. It may be that many people respond to a setback by bouncing back with renewed determination. It may also be that the failure provokes a rethink and a fresh course of action. Doris Day, after all, did not respond to a shattered leg by trying even harder to become a dancer. She changed her goals and prospered as a result.

We don’t have to be promising young scientists — or aspiring starlets — to benefit from having obstacles placed in our way. Something as mundane as a strike disrupting regular commuting has been shown to push people towards new habits. Three economists who studied data from London’s public transport network found that after a 48-hour strike in London in February 2014, thousands of commuters changed routes and never switched back. They discovered that they’d been doing the commute wrong their entire working lives.

Often failure is simply failure, and a setback is exactly what it seems. But sometimes the obstacle that has been placed in our path might provoke us to look around, and perhaps to discover that a better route was there all along.

Written for and first published in the Financial Times on 17 May 2019.

My book Messy explores some of these ideas in more depth – feel free to order online or through your local bookshop.

Why we should favour second guesses over first instincts…

Tension is rising in the Harford household as exams approach and we try to persuade Miss Harford Sr to relax, and Miss Harford Jr to be slightly less relaxed. I’m sure many readers have vivid memories of the exam room, recent or otherwise. But here’s a question about exam technique that suggests a much wider lesson. In a multiple-choice test, you sometimes write down an answer and then have second thoughts. Is it wise to stay with your first instincts, or better to switch?

Most people would advise that the initial answer is usually better than the doubt-plagued second guess. Three-quarters of students think so, according to various surveys over the years. College instructors think so too, by a majority of 55 to 16 per cent. The 2000 edition of Barron’s How to Prepare for the GRE Test is very clear that students should be wary of switching. “Experience indicates that many students who change answers change to the wrong answer.”

This confidence would be reassuring, were it not utterly erroneous. Researchers have been studying this question since the 1920s. They have overwhelmingly concluded both that individual answer changes are more likely to be from wrong to right, and that students who change their answers tend to improve their scores. This gap between perception and reality is stark enough to have earned a name: the “first-instinct fallacy”. No doubt our first instincts are often right, but when we start to have second thoughts, the second thoughts are usually occurring for a reason. It is better to switch. So why don’t we?

Justin Kruger, a psychologist at New York University, has been studying this question. (Prof Kruger is more famous as co-discoverer of the Dunning-Kruger effect: people who are incompetent are too incompetent to realise how incompetent they are.) With his colleagues Derrick Wirtz and Dale Miller he replicated the longstanding findings that college students believe you should trust your first answer in a multiple choice question, and yet that switching to a second answer tends to improve your grades. Then the trio started to explore why.

In one study they showed subjects video based on the TV show Who Wants To Be A Millionaire? and asked them to imagine that they were watching a teammate play, accumulating cash for the team whenever he or she gave the correct answer. Some subjects were shown teammates who always switched on 50/50 questions, while others were shown teammates who always stuck with their first instinct. In the study, both strategies produced identical results, yet subjects watching a switching teammate were more frustrated and critical and had a good memory for the errors.

Another study by Prof Kruger and his colleagues showed that we also have a warped recollection of our own errors in multiple choice tests. We have a rosy memory of sticking to our first instincts, forgetting the failures and exaggerating the successes. We vividly recall switching to the wrong answer and overestimate how often we did so. In short, we remember sticking as having been the best tactic, when in fact switching was better. No wonder the Barron’s guide remarks that “experience” tells us switching is a bad idea. Our own experiences do indeed tell us that, but only because we misremember the lessons of previous switches.

If you — or a loved one — are about to enter exam season, perhaps this evidence-based strategy will be of use. But it’s hard not to see a broader lesson. How often in life do we make a choice and then stick to it despite mounting doubts?

In politics, such questions are aggravated by questions of partisanship and pride. Nobody wants to admit that they were wrong in the face of jeers from those on the opposite side of the political fence. The U-turn is one of the greatest sins in politics, if only because it is so easy to criticise. Either you were wrong before or you are wrong now.

But even in everyday life, we find ourselves clinging to bad choices. Steven Levitt, the co-author of Freakonomics, once conducted a study in which people hesitating over big choices — to leave a spouse, to adopt a child, to quit a job, to start a business — agreed to be guided by a coin toss. Those who had been nudged to act ended up being happier several months on than those who had been nudged to stick with the status quo.

We are prone to cling tightly to the devil we know. The likely explanation is that we are seeking to minimise regret. Forget the old cliché about regretting the things you didn’t do more than the things you did: we regret misfiring action more than misfiring inaction. We starkly remember the times we changed things for the worse, and we more easily forget the times when we failed to change things for the better. It’s not that impulsive action is always the best option, any more than your first answer on a test is always wrong. Instead, the lesson is that if you are hesitating over whether to leave things as they are, you probably needed to make a change some time ago.

Written for and first published in the Financial Times on 10 May 2019.

My book “Fifty Things That Made the Modern Economy” (UK) / “Fifty Inventions That Shaped The Modern Economy” (US) is out now in paperback – feel free to order online or through your local bookshop.

The clash of the two cultures and the challenge of collaboration

May 7 was the 60th anniversary of the delivery of CP Snow’s famous lecture, The Two Cultures and the Scientific Revolution. It is dimly remembered as a lament about the mutual incomprehension between arts and sciences, wrapped up with some pompous anecdotes about Oxbridge high table and airy generalisations about the dynamism of scientists. Some of it is absurd. Snow dismisses George Orwell’s 1984 as pure Luddism, “the strongest possible wish that the future should not exist”. (Orwell old chap, relax and enjoy the fruits of technological progress!) That Snow’s lecture is remembered at all is probably thanks to an acidic rebuttal by the literary critic FR Leavis.

Nevertheless, Snow was on to something important. His message was garbled, in fact, because he was on to several important things at once. The first is the challenge of collaboration. If anything, The Two Cultures understates that. Yes, the classicists need to work with the scientists, but the physicists also need to work with the biologists, the economists must work with the psychologists, and everyone has to work with the statisticians. And the need for collaboration between technical experts has grown over time because, as science advances and problems grow more complex, we increasingly live in a world of specialists.

The economist Benjamin Jones has been studying this issue by examining databases of patents and scientific papers. His data show that successful research now requires larger teams filled with more specialised researchers. Scientific and material progress demands complex collaboration.

Snow appreciated — in a way that many of us still do not — how profound that progress was. The scientist and writer Stephen Jay Gould once mocked Snow’s prediction that “once the trick of getting rich is known, as it now is, the world can’t survive half rich and half poor” and that division would not last to the year 2000. “One of the worst predictions ever printed,” scoffed Gould in a book published posthumously in 2003. Had Gould checked the numbers, he would have seen that between 1960 and 2000, the proportion of people living in extreme poverty had roughly halved, and it has continued to fall sharply since then. Snow’s 40-year forecast was more accurate than Gould’s 40 years of hindsight. Even when we fancy ourselves broadly educated, as Gould did, we may not know what we don’t know. That was one of Snow’s points.

But the deepest point of all — buried a little too deep, perhaps — is a practical problem that remains as pressing today as it was in 1959: how to reconcile technical expertise with the demands of policy and politics. In short — have we really had enough of experts?

The historian Lisa Jardine highlights this sentence in Snow’s argument: “It is the traditional culture, to an extent remarkably little diminished by the emergence of the scientific one, which manages the western world.” We didn’t decide we’d had enough of experts in 2016; we made that decision long ago.

There have never been many scientists in politics. In his 2012 book, The Geek Manifesto, Mark Henderson reckoned there was one scientist among 650 members of the UK parliament. The US Congress is packed with lawyers. We need a little more technical expertise close to the levers of power: pithy quotations from Cicero will not do the trick; nor those from Karl Marx.

As Snow pointed out in a series of lectures he gave at Harvard in 1960, published as Science and Government, grave mistakes can result not only from a vacuum of technical knowledge in politics, but from a monopoly — the single expert, unchallenged. He cited the allied bombing of dense urban areas in Germany during the war, which not only took a terrible toll on civilians, but failed in military terms by sparing industrial targets. The source of the problem was a flawed statistical study by Winston Churchill’s scientific adviser Frederick Lindemann that no one had both the technical skill and the political clout to challenge. It is not enough to give political influence to a physicist or an economist. The corridors of power must ring with scientifically informed debate.

Snow quotes another scientist about losing the argument over area-bombing. “I confess to a haunting sense of personal failure . . . if we had only been more persuasive and had forced people to believe our simple arithmetic . . . might we not have changed this decision?” That is precisely how many economists felt after the Brexit referendum. There is no moral equivalence, but there is an intellectual parallel: we felt that a serious mistake had been made and that it was partly our fault for not being more persuasive.

None of this is to assert the superiority of a technical education over a classical, literary or vocational one, although Snow sometimes seemed to yield to that temptation. We don’t need a parliament full of chemists any more than we need one full of classicists. We need a mix. Like scientific research, good policymaking is now a team effort. It requires different perspectives and a range of specialist expertise. We all must learn to work with people who see the world very differently. That wasn’t easy to do in 1959. It is no easier today.

Written for and first published in the Financial Times on 3 May 2019.

My book “Fifty Things That Made the Modern Economy” (UK) / “Fifty Inventions That Shaped The Modern Economy” (US) is out now in paperback – feel free to order online or through your local bookshop.

Range, misinformation and the fine line between stupid and clever

Over the past couple of week’s I’ve been enjoying David Epstein’s Range – about the value of being a generalist. As I may have mentioned it’s in sympathy with my latest TED talk, which cites Epstein. I had the pleasure of seeing Epstein dig up a lot of interesting new research that supports some of what I argued in Messy – about the virtues of moving between fields, switching contexts and improvising. (It felt a bit like an out-of-sample check of my expectations.) Epstein skillfully blends stories with argument and evidence, and he makes it look easy (it isn’t).

Also impressed by The Misinformation Age by Cailin O’Connor and James Owen Weatherall (he also wrote the very nice Physics of Wall Street). O’Connor and Weatherall cover the territory very well – again, some nice stories, many of which I hadn’t heard – and they use a lot of network analysis to show how true and false ideas can spread or be propagated by various devious pieces of propaganda. Fairly nerdy but easy and fun to read.

Next on the pile, The Intelligence Trap: Why Smart People Make Dumb Mistakes by David Robson. This comes recommended to me but I have not yet cracked the spine. Looks v. interesting, though.

(While I’m here, I should mention that I just bought a new Moto G7 Play phone for £130 and I’m delighted with it. Seems to be a heck of a lot of phone for not a lot of money.)

My book “Fifty Things That Made the Modern Economy” (UK) / “Fifty Inventions That Shaped The Modern Economy” (US) is out now in paperback – feel free to order online or through your local bookshop.

Why going on holiday gives us more memories

Lucky me. I’ve just returned from a family holiday in that most exotic of countries, Japan. So many fresh sights and strange tastes: from flower gardens, temples and communal baths to robots, bullet trains and the Kawaii Monster Café. Although we were there barely more than a week, it’s hard to believe we packed so much in.

While on an adventurous holiday, many people experience that strange sense of time having slowed down in the most pleasurable way, and of conversations that begin, “Was it really only yesterday that we . . . ?”

Ten days in a far-off land produces a richer treasury of detailed memories than 10 weeks back home. But what is behind this phenomenon? And does it teach us something about living a full life?

One answer comes from Claude Shannon, a titan of computer science still under-appreciated outside his field. In 1948, Shannon published one of his two profound contributions, A Mathematical Theory of Communication.

One of the implications of Shannon’s theory is that a message can be compressed to the extent that it is predictable. Anyone who has played the guessing game of Hangman knows that once a few letters are in place, the remainder are usually easy to guess. Similarly: sntnces wth vwls rmvd sty cmprhsbl. Ritualised conversations (“How are you?” “Very well, thank you. How are you?”) can be heavily compressed; poetry, perhaps, less so.

A movie can be compressed because, between cuts, each frame tends to resemble the previous one. A compression algorithm can start with the first frame after the cut and store a series of “diffs” — changes from the previous frame. The faster and more dramatic the movement or transitions, the harder a video is to compress, because the diffs are almost as big as the original frames.

Although the parallel is not exact, much the same thing seems to be going on with our memories of life. The brain is not a video recorder; we recall the gist. Sometimes the gist is very brief. If I get up in the morning at the usual time, eat my customary breakfast and catch my usual train to the office, why should my brain trouble itself to remember this day two weeks after the fact? The diffs are barely worth bothering with. In contrast, fresh experiences defy compression: the diffs are too big.

We could expand this idea to other aspects of life, maximising information content by keeping things unpredictable. Brian Christian, author of The Most Human Human, a book about conversations between humans and computers, speculates that if we’re seeking advice we should ask the person of whose answer we are least certain. If we want to understand a person, we should ask them the question to which we are least sure of their answer.

Perhaps the best newspaper columnist is the one whose columns are the least predictable? That may be going too far: too much novelty is exhausting, and we need an anchor for our expectations. But I well recall a friend saying that her favourite newspaper pandered to her preconceptions so much she didn’t really need to bother reading it; she could just think hard about everything she already held true. We scribblers should hope to do better.

One would not want a life of endless novelty, if only because that would mean a procession of superficial impressions and comparisons. Static, surprisingly, is impossible to compress without losing information, because its randomness makes prediction or interpolation impossible. Yet static is also meaningless. Too many random novelties, too fast, produces a lived experience not unlike static.

One of the virtues of experience is that it can attune us to subtle details and deep connections: “a world in a grain of sand, and a heaven in a wild flower”. (It may also allow us to draw out meaning from a brief allusion to a larger body of work.) But if repeated experience becomes humdrum, and we do not look for the depths, our days will be thin and forgettable.

So if you want a full life, rich with memories, keep searching for new experiences. That is far easier for the young than the old, but it should be possible for anyone. Surprising conversations are always there for the having and, while a holiday on the other side of the world is a costly (if reliable) source of vivid experiences, novelty is affordable for almost anyone in the form of new music, books, even walking an unfamiliar path through your own home town. It is always worth seeking out whatever is excellent — but for vivid memories, the same old excellence is not quite enough. Freshness matters, too.

I suspect that we all know this, yet we fall back on familiar routines. I often eat the same lunch in the same lunch spot, have a favourite library desk, and enjoy particular genres of film and book. In the moment, these habits are convenient and comforting. In the future, they will result in a life more easily compressed in the memory.

In Japan, it was hard to avoid eye-opening sights and attention-grabbing situations. Now I’m resolved to seek out the new and challenging in the UK, just as we did when far from home.

 
Written for and first published in the Financial Times on 26 April 2019.

My book “Fifty Things That Made the Modern Economy” (UK) / “Fifty Inventions That Shaped The Modern Economy” (US) is out now in paperback – feel free to order online or through your local bookshop.