Tim Harford The Undercover Economist
  • THE UNDERCOVER ECONOMIST STRIKES BACK
    “Every Tim Harford book is cause for celebration. He makes the ‘dismal science’ seem like an awful lot of fun.”
    – Malcolm Gladwell, author of “The Tipping Point”
  • Adapt – Why Success Always Starts with Failure


    “A highly readable argument... Very impressive”
    – Nassim N Taleb, author of “The Black Swan”
  • Dear Undercover Economist
    “Harford has a knack for explaining economic principles and problems in plain language and, even better, for making them fun”
    – The New York Times
  • The Logic of Life
    “Highly readable, funny and daringly contentious... a whopping good time.”
    – The San Francisco Chronicle
  • The Undercover Economist
    “Required reading”
    – Stephen J. Dubner, co-author of “Freakonomics”
Video

How to give a TED talk in a hurry

Free email updates

(You can unsubscribe at any time)

4th of May, 2016VideoComments off
Undercover Economist

Could an income for all provide the ultimate safety net?

‘Though the idea of a basic income is far from mainstream, it has had astonishingly broad support’

Last week, I pondered how society should protect or compensate people whose jobs have been lost to the forces of globalisation or technological change. I did not, however, discuss the most obvious idea of all: that we should simply give people money — a basic income for everyone, regardless of what they do or what they need. It’s the ultimate social safety net.

For an idea that is so far from mainstream political practice, the payment of a basic income has had astonishingly broad support, from Martin Luther King Jr to Milton Friedman. It’s on the lips of the policy wonk community too: the Freakonomics podcast recently devoted an episode to the case for a universal basic income. The Royal Society for Arts, a venerable British think-tank, has published a report enthusiastically supporting the idea. Dutch journalist Rutger Bregman is just as keen, as outlined in his recent, eloquent book Utopia for Realists.

Policy experiments are also on the way. The charity GiveDirectly has just announced plans to run a randomised trial in which 6,000 Kenyans will receive a basic income for more than a decade. Various Silicon Valley types — with one eye on the looming Robot Job Apocalypse — are making serious-sounding noises about running experiments too. Pilots are planned in Canada and Finland, and the Swiss have a referendum on the topic in June.

Could a basic income really work? The answer is yes. But the plan may be more painful than some of its advocates are willing to admit.

First, let’s establish what we’re talking about. A universal basic income is a cash payment from the state, paid to everyone unconditionally. For the sake of being concrete, let’s call it £10 a day. That seems like a lot of money to be giving to absolutely everyone, but it’s within the bounds of reason. Such a payment would cost £234bn a year across 64 million UK residents, so it could be largely paid for by scrapping all social security spending, which is £217bn.

There are lots of other proposals that one might call a basic income. Leftwing advocates might want far more than £10 a day but that would require a huge expansion of the state, with much higher taxes. The more libertarian proponents of the idea might also approve of a higher basic income, in exchange for a rolling back of state-provided services. Privatising the entire health and education system in the UK would free up £240bn, easily enough to double the basic income to £20 a day for every man, woman and child. But that money would need to cover school fees and medical bills.

All this is within the bounds of affordability. But is it desirable? Here are two big question marks over the idea.

The first is whether people would simply stop working. Several large experiments conducted in the US and Canada in the late 1970s and early 1980s suggest that a minimum income would encourage people to reduce their hours a little. If such slacking-off undermined the tax base, the entire project could become both economically and politically unsustainable.

But the tax base is probably safe enough, because the people who might be tempted to quit work and live on £10 a day are not the people whose taxes pay for most state spending. In the UK, the richest 15 per cent of taxpayers — people who pay at least some tax at the 40 per cent rate — supply about two-thirds of income tax revenue. Few of these people are likely to find the basic income a tempting inducement to leave the labour force.

In some cases, we might celebrate a decision to stop work. Some people volunteer; others care for children or relatives; some might use the income to fund themselves as they stay in education or retrain. Some, alas, might use the money to stay alive as they write poetry.

The second objection is more worrying: if the welfare state is to be replaced by a basic income, it will provide far too little for some. A tenner a day is less than half the new UK state pension, so it’s hard to imagine pensioners embracing the idea with much gusto.

On the other hand, if the basic income is to be supplemented by a raft of special cases — people with disabilities, people with expensive rent, people who are elderly — then it may become as complex as the tangle of benefit entitlements it aims to replace, or hugely expensive, or both.

Andrew Hood of the Institute for Fiscal Studies says that compared with current welfare benefits, a basic income would “either be a lot less generous or a lot more expensive”. Take your pick.

In the end, the idea appeals to three types of people: those who are comfortable with a dramatic increase in the size of the state, those who are willing to see needy people lose large sums relative to the status quo, and those who can’t add up.

A basic income makes perfect sense once we arrive at an economy where millions work for low wages while automation produces a bountiful economy all around them. The debate turns on whether that world has already arrived.

Written for and first published at ft.com.

Free email updates

(You can unsubscribe at any time)

3rd of May, 2016Undercover EconomistComments off
Undercover Economist

Tata Steel, Port Talbot and how to manage industrial decline

‘The wounds of a large industrial closure run deep. The entire economic ecosystem of an area can collapse’

The possible closure of Tata Steel’s operations in Port Talbot casts a deep shadow over the area. There’s something familiar about this depressing story. The shipyards of the Clyde and the cotton mills of Manchester have faded. The coalfields of Derbyshire, Nottinghamshire and South Yorkshire were all around me as I grew up in Chesterfield during the miners’ strike of 1984-85. Now the mining jobs are gone. Further afield, there are the job losses in the automobile production lines of Detroit, for the shoemakers of Kobe in Japan, or at Eastman Kodak in Rochester, New York.

So what should be done when communities are wounded by such blows? One tempting answer is “everything”; that the government should nationalise troubled operations or adopt similar big-bazooka tactics, such as high trade barriers or large subsidies. It’s easy enough to make the emotional case for this but the practical case isn’t so plausible. Would nationalisation have saved Kodak’s film business? Is Manchester the place for a 21st-century Cottonopolis?

Sometimes, government can help restructure a troubled business — as with the Obama administration’s interventions in the case of General Motors, or the long but ultimately successful nationalisation of Rolls-Royce in the 1970s.

However, taxpayers are always at risk of being saddled with the role of supporting industries in inescapable structural decline. The political economy of these cases is skewed towards preservation rather than creative destruction. Old industries under stress have much to gain from government support, and can point to people who need help. There is no constituency for jobs that have not yet been created.

So a different answer is that we should do nothing. This laissez-faire reasoning points out that economic change inevitably creates losers but, ultimately, society is better off. We cannot resist change, only adjust. Former autoworkers, steelworkers, and coalminers need to pick themselves up and move to where fresh jobs are available, or retrain.

There is a logic to this argument but it glosses over the deep wounds of a large industrial closure. It isn’t just that workers lose jobs. The entire economic ecosystem of an area can collapse. Newly jobless workers find that their homes are worthless, their pensions too sometimes.

And workers with the kind of skills that are under pressure from technology or trade may find that they move from one sinking lifeboat to another, with their new jobs under threat from the same forces that destroyed the old ones. More radically, retraining — maybe as a neurosurgeon or data scientist — would solve that problem, but then so would discovering a Rembrandt in the attic.

Between the ideologically pure answers of “do everything” and “do nothing”, we have the current consensus, “do something”. But what?

There are three broad approaches to looking after the losers from economic change: try to bring new jobs to people; try to help the people change to find new jobs; just send money.

Bringing new jobs to people is the most natural idea, but regional regeneration is difficult. Depressed communities often stay depressed. A Sheffield Hallam University study from 2014, The State of the Coalfields, found that 30 years after the miners’ strike, coalfield communities have lower employment rates and higher reliance on disability benefits. The track record of place-based regeneration policies is patchy and sobering.

If the jobs won’t move, perhaps the workers can? An influential 1992 study by economists Olivier Blanchard and Lawrence Katz found that the US labour market once worked this way. While a shock could have a lasting effect on a local economy, the unemployment rate itself would subside, “not because employment picks up, but because workers leave the state”.

But new research from Mai Dao, Davide Furceri and Prakash Loungani at the IMF finds that US workers move less than they did back in the 1980s. Instead of moving, they are more likely to stay put and stay jobless. (Mobility has improved in the European Union, albeit from much lower levels.)

We don’t really know why mobility is falling in the US. Maybe because dual-income households find it harder to move — but then the same pattern is seen for single people. Housing costs increasingly prevent poor people from moving to booming areas such as New York and London in search of work.

“My guess is that there’s no one reason for the fall in mobility,” says Betsey Stevenson of the University of Michigan, also formerly chief economist at the US labour department. Stevenson, like many economists, argues that education must be a huge part of the answer to economic shocks. The jobs have changed, and so must we.

Education is, indeed, a remarkable thing. Lawrence Katz has observed that between 1979 and 2012, the wage gap between a US household of two college graduates and a household of two high school graduates grew by around $30,000 — a sum that dwarfs most shifts in the economic landscape. But it is easy to be glib about retraining: governments are tempted to use training programmes as a way to make work and shift people off the welfare rolls.

So a final answer as to how to compensate the losers is the simplest: give them money. That is a strategy that offers both more, and less, than it might seem at first glance. But that is a topic for next week.

Written for and first published at ft.com.

Free email updates

(You can unsubscribe at any time)

Marginalia

What I’ve been reading in April

Utopia for Realists by Rutger Bregman.  A short book arguing in favour of a universal basic income (and a few other things, but the UBI stuff is easily the most interesting). I’m not totally convinced by the proposal – the arithmetic of a UBI calls for some painful choices – but it’s an excellent read and full of well-told stories and details I didn’t know. (UK) (US)

The Art of Travel by Alain de Botton – re-reading my favourite de Botton book. A fascinating and original reflection on how and why we travel – and whether we might find what we seek closer to home. (UK) (US)

The Creative Habit by Twyla Tharp. A practical guide to living a creative life, by a brilliant, driven and entrepreneurial choreographer. I love the story of Twyla’s near-disastrous collaboration with Billy Joel, and often retell the story myself. (UK) (US)

You Are Not So Smart by David McRaney – a fun guide to cognitive errors and logical fallacies. I’ve been adoring the podcast of the same name. (UK) (US)

Out on the Wire by Jessica Abel. Okay, here’s the deal: everyone I know in radio is reading this book to learn how to be the next Ira Glass, Jad Abumrad or Roman Mars. And it’s good – very good. (UK) (US)

Or you may fancy one of my own books – for example, Adapt.

 

25th of April, 2016MarginaliaComments off
Highlights

How Politicians Poisoned Statistics

We have more data — and the tools to analyse and share them — than ever before. So why is the truth so hard to pin down?

In January 2015, a few months before the British general election, a proud newspaper resigned itself to the view that little good could come from the use of statistics by politicians. An editorial in the Guardian argued that in a campaign that would be “the most fact-blitzed in history”, numerical claims would settle no arguments and persuade no voters. Not only were numbers useless for winning power, it added, they were useless for wielding it, too. Numbers could tell us little. “The project of replacing a clash of ideas with a policy calculus was always dubious,” concluded the newspaper. “Anyone still hankering for it should admit their number’s up.”

This statistical capitulation was a dismaying read for anyone still wedded to the idea — apparently a quaint one — that gathering statistical information might help us understand and improve our world. But the Guardian’s cynicism can hardly be a surprise. It is a natural response to the rise of “statistical bullshit” — the casual slinging around of numbers not because they are true, or false, but to sell a message.

Politicians weren’t always so ready to use numbers as part of the sales pitch. Recall Ronald Reagan’s famous suggestion to voters on the eve of his landslide defeat of President Carter: “Ask yourself, ‘Are you better off now than you were four years ago?’” Reagan didn’t add any statistical garnish. He knew that voters would reach their own conclusions.

The British election campaign of spring last year, by contrast, was characterised by a relentless statistical crossfire. The shadow chancellor of the day, Ed Balls, declared that a couple with children (he didn’t say which couple) had lost £1,800 thanks to the government’s increase in value added tax. David Cameron, the prime minister, countered that 94 per cent of working households were better off thanks to recent tax changes, while the then deputy prime minister Nick Clegg was proud to say that 27 million people were £825 better off in terms of the income tax they paid.

Could any of this be true? Yes — all three claims were. But Ed Balls had reached his figure by summing up extra VAT payments over several years, a strange method. If you offer to hire someone for £100,000, and then later admit you meant £25,000 a year for a four-year contract, you haven’t really lied — but neither have you really told the truth. And Balls had looked only at one tax. Why not also consider income tax, which the government had cut? Clegg boasted about income-tax cuts but ignored the larger rise in VAT. And Cameron asked to be evaluated only on his pre-election giveaway budget rather than the tax rises he had introduced earlier in the parliament — the equivalent of punching someone on the nose, then giving them a bunch of flowers and pointing out that, in floral terms, they were ahead on the deal.

Each claim was narrowly true but broadly misleading. Not only did the clashing numbers confuse but none of them helped answer the crucial question of whether Cameron and Clegg had made good decisions in office.

To ask whether the claims were true is to fall into a trap. None of these politicians had any interest in playing that game. They were engaged in another pastime entirely.

Thirty years ago, the Princeton philosopher Harry Frankfurt published an essay in an obscure academic journal, Raritan. The essay’s title was “On Bullshit”. (Much later, it was republished as a slim volume that became a bestseller.) Frankfurt was on a quest to understand the meaning of bullshit — what was it, how did it differ from lies, and why was there so much of it about?

Frankfurt concluded that the difference between the liar and the bullshitter was that the liar cared about the truth — cared so much that he wanted to obscure it — while the bullshitter did not. The bullshitter, said Frankfurt, was indifferent to whether the statements he uttered were true or not. “He just picks them out, or makes them up, to suit his purpose.”

Statistical bullshit is a special case of bullshit in general, and it appears to be on the rise. This is partly because social media — a natural vector for statements made purely for effect — are also on the rise. On Instagram and Twitter we like to share attention-grabbing graphics, surprising headlines and figures that resonate with how we already see the world. Unfortunately, very few claims are eye-catching, surprising or emotionally resonant because they are true and fair. Statistical bullshit spreads easily these days; all it takes is a click.

Consider a widely shared list of homicide “statistics” attributed to the “Crime Statistics Bureau — San Francisco”, asserting that 81 per cent of white homicide victims were killed by “blacks”. It takes little effort to establish that the Crime Statistics Bureau of San Francisco does not exist, and not much more digging to discover that the data are utterly false. Most murder victims in the United States are killed by people of their own race; the FBI’s crime statistics from 2014 suggest that more than 80 per cent of white murder victims were killed by other white people.

Somebody, somewhere, invented the image in the hope that it would spread, and spread it did, helped by a tweet from Donald Trump, the current frontrunner for the Republican presidential nomination, that was retweeted more than 8,000 times. One can only speculate as to why Trump lent his megaphone to bogus statistics, but when challenged on Fox News by the political commentator Bill O’Reilly, he replied, “Hey, Bill, Bill, am I gonna check every statistic?”

Harry Frankfurt’s description of the bullshitter would seem to fit Trump perfectly: “He does not care whether the things he says describe reality correctly.”

While we can’t rule out the possibility that Trump knew the truth and was actively trying to deceive his followers, a simpler explanation is that he wanted to win attention and to say something that would resonate with them. One might also guess that he did not check whether the numbers were true because he did not much care one way or the other. This is not a game of true and false. This is a game of politics.

While much statistical bullshit is careless, it can also be finely crafted. “The notion of carefully wrought bullshit involves … a certain inner strain,” wrote Harry Frankfurt but, nevertheless, the bullshit produced by spin-doctors can be meticulous. More conventional politicians than Trump may not much care about the truth but they do care about being caught lying.

Carefully wrought bullshit was much in evidence during last year’s British general election campaign. I needed to stick my nose in and take a good sniff on a regular basis because I was fact-checking on behalf of the BBC’s More or Less programme. Again and again I would find myself being asked on air, “Is that claim true?” and finding that the only reasonable answer began with “It’s complicated”.

Take Ed Miliband’s claim before the last election that “people are £1,600 a year worse off” than they were when the coalition government came to power. Was that claim true? Arguably, yes.

But we need to be clear that by “people”, the then Labour leader was excluding half the adult population. He was not referring to pensioners, benefit recipients, part-time workers or the self-employed. He meant only full-time employees, and, more specifically, only their earnings before taxes and benefits.

Even this narrower question of what was happening to full-time earnings is a surprisingly slippery one. We need to take an average, of course. But what kind of average? Labour looked at the change in median wages, which were stagnating in nominal terms and falling after inflation was taken into account.

That seems reasonable — but the median is a problematic measure in this case. Imagine nine people, the lowest-paid with a wage of £1, the next with a wage of £2, up to the highest-paid person with a wage of £9. The median wage is the wage of the person in the middle: it’s £5.

Now imagine that everyone receives a promotion and a pay rise of £1. The lowly worker with a wage of £1 sees his pay packet double to £2. The next worker up was earning £2 and now she gets £3. And so on. But there’s also a change in the composition of the workforce: the best-paid worker retires and a new apprentice is hired at a wage of £1. What’s happened to people’s pay? In a sense, it has stagnated. The pattern of wages hasn’t changed at all and the median is still £5.

But if you asked the individual workers about their experiences, they would all tell you that they had received a generous pay rise. (The exceptions are the newly hired apprentice and the recent retiree.) While this example is hypothetical, at the time Miliband made his comments something similar was happening in the real labour market. The median wage was stagnating — but among people who had worked for the same employer for at least a year, the median worker was receiving a pay rise, albeit a modest one.

Another source of confusion: if wages for the low-paid and the high-paid are rising but wages in the middle are sagging, then the median wage can fall, even though the median wage increase is healthy. The UK labour market has long been prone to this kind of “job polarisation”, where demand for jobs is strongest for the highest and lowest-paid in the economy. Job polarisation means that the median pay rise can be sizeable even if median pay has not risen.

Confused? Good. The world is a complicated place; it defies description by sound bite statistics. No single number could ever answer Ronald Reagan’s question — “Are you better off now than you were four years ago?” — for everyone in a country.

So, to produce Labour’s figure of “£1,600 worse off”, the party’s press office had to ignore the self-employed, the part-timers, the non-workers, compositional effects and job polarisation. They even changed the basis of their calculation over time, switching between different measures of wages and different measures of inflation, yet miraculously managing to produce a consistent answer of £1,600. Sometimes it’s easier to make the calculation produce the number you want than it is to reprint all your election flyers.

Very few claims are eye-catching, surprising or emotionally resonant because they are true and fair

Such careful statistical spin-doctoring might seem a world away from Trump’s reckless retweeting of racially charged lies. But in one sense they were very similar: a political use of statistics conducted with little interest in understanding or describing reality. Miliband’s project was not “What is the truth?” but “What can I say without being shown up as a liar?”

Unlike the state of the UK job market, his incentives were easy to understand. Miliband needed to hammer home a talking point that made the government look bad. As Harry Frankfurt wrote back in the 1980s, the bullshitter “is neither on the side of the true nor on the side of the false. His eye is not on the facts at all … except insofar as they may be pertinent to his interest in getting away with what he says.”

Such complexities put fact-checkers in an awkward position. Should they say that Ed Miliband had lied? No: he had not. Should they say, instead, that he had been deceptive or misleading? Again, no: it was reasonable to say that living standards had indeed been disappointing under the coalition government.

Nevertheless, there was a lot going on in the British economy that the figure omitted — much of it rather more flattering to the government. Full Fact, an independent fact-checking organisation, carefully worked through the paper trail and linked to all the relevant claims. But it was powerless to produce a fair and representative snapshot of the British labour market that had as much power as Ed Miliband’s seven-word sound bite. No such snapshot exists. Truth is usually a lot more complicated than statistical bullshit.

On July 16 2015, the UK health secretary Jeremy Hunt declared: “Around 6,000 people lose their lives every year because we do not have a proper seven-day service in hospitals. You are 15 per cent more likely to die if you are admitted on a Sunday compared to being admitted on a Wednesday.”

This was a statistic with a purpose. Hunt wanted to change doctors’ contracts with the aim of getting more weekend work out of them, and bluntly declared that the doctors’ union, the British Medical Association, was out of touch and that he would not let it block his plans: “I can give them 6,000 reasons why.”

Despite bitter opposition and strike action from doctors, Hunt’s policy remained firm over the following months. Yet the numbers he cited to support it did not. In parliament in October, Hunt was sticking to the 15 per cent figure, but the 6,000 deaths had almost doubled: “According to an independent study conducted by the BMJ, there are 11,000 excess deaths because we do not staff our hospitals properly at weekends.”

Arithmetically, this was puzzling: how could the elevated risk of death stay the same but the number of deaths double? To add to the suspicions about Hunt’s mathematics, the editor-in-chief of the British Medical Journal, Fiona Godlee, promptly responded that the health secretary had publicly misrepresented the BMJ research.

Undaunted, the health secretary bounced back in January with the same policy and some fresh facts: “At the moment we have an NHS where if you have a stroke at the weekends, you’re 20 per cent more likely to die. That can’t be acceptable.”

All this is finely wrought bullshit — a series of ever-shifting claims that can be easily repeated but are difficult to unpick. As Hunt jumped from one form of words to another, he skipped lightly ahead of fact-checkers as they tried to pin him down. Full Fact concluded that Hunt’s statement about 11,000 excess deaths had been untrue, and asked him to correct the parliamentary record. His office responded with a spectacular piece of bullshit, saying (I paraphrase) that whether or not the claim about 11,000 excess deaths was true, similar claims could be made that were.

So, is it true? Do 6,000 people — or 11,000 — die needlessly in NHS hospitals because of poor weekend care? Nobody knows for sure; Jeremy Hunt certainly does not. It’s not enough to show that people admitted to hospital at the weekend are at an increased risk of dying there. We need to understand why — a question that is essential for good policy but inconvenient for politicians.

One possible explanation for the elevated death rate for weekend admissions is that the NHS provides patchy care and people die as a result. That is the interpretation presented as bald fact by Jeremy Hunt. But a more straightforward explanation is that people are only admitted to hospital at the weekend if they are seriously ill. Less urgent cases wait until weekdays. If weekend patients are sicker, it is hardly a surprise that they are more likely to die. Allowing non-urgent cases into NHS hospitals at weekends wouldn’t save any lives, but it would certainly make the statistics look more flattering. Of course, epidemiologists try to correct for the fact that weekend patients tend to be more seriously ill, but few experts have any confidence that they have succeeded.

A more subtle explanation is that shortfalls in the palliative care system may create the illusion that hospitals are dangerous. Sometimes a patient is certain to die, but the question is where — in a hospital or a palliative hospice? If hospice care is patchy at weekends then a patient may instead be admitted to hospital and die there. That would certainly reflect poor weekend care. It would also add to the tally of excess weekend hospital deaths, because during the week that patient would have been admitted to, and died in, a palliative hospice. But it is not true that the death was avoidable.

Does it seem like we’re getting stuck in the details? Well, yes, perhaps we are. But improving NHS care requires an interest in the details. If there is a problem in palliative care hospices, it will not be fixed by improving staffing in hospitals.

“Even if you accept that there’s a difference in death rates,” says John Appleby, the chief economist of the King’s Fund health think-tank, “nobody is able to say why it is. Is it lack of diagnostic services? Lack of consultants? We’re jumping too quickly from a statistic to a solution.”

When one claim is discredited, Jeremy Hunt’s office simply asserts that another one can be found to take its place

This matters — the NHS has a limited budget. There are many things we might want to spend money on, which is why we have the National Institute for Health and Care Excellence (Nice) to weigh up the likely benefits of new treatments and decide which offer the best value for money.

Would Jeremy Hunt’s push towards a seven-day NHS pass the Nice cost-benefit threshold? Probably not. Our best guess comes from a 2015 study by health economists Rachel Meacock, Tim Doran and Matt Sutton, which estimates that the NHS has many cheaper ways to save lives. A more comprehensive assessment might reach a different conclusion but we don’t have one because the Department for Health, oddly, hasn’t carried out a formal health impact assessment of the policy it is trying to implement.

This is a depressing situation. The government has devoted considerable effort to producing a killer number: Jeremy Hunt’s “6,000 reasons” why he won’t let the British Medical Association stand in his way. It continues to produce statistical claims that spring up like hydra heads: when one claim is discredited, Hunt’s office simply asserts that another one can be found to take its place. Yet the government doesn’t seem to have bothered to gather the statistics that would actually answer the question of how the NHS could work better.

This is the real tragedy. It’s not that politicians spin things their way — of course they do. That is politics. It’s that politicians have grown so used to misusing numbers as weapons that they have forgotten that used properly, they are tools.

You complain that your report would be dry. The dryer the better. Statistics should be the dryest of all reading,” wrote the great medical statistician William Farr in a letter in 1861. Farr sounds like a caricature of a statistician, and his prescription — convey the greatest possible volume of information with the smallest possible amount of editorial colour — seems absurdly ill-suited to the modern world.

But there is a middle ground between the statistical bullshitter, who pays no attention to the truth, and William Farr, for whom the truth must be presented without adornment. That middle ground is embodied by the recipient of William Farr’s letter advising dryness. She was the first woman to be elected to the Royal Statistical Society: Florence Nightingale.

Nightingale is the most celebrated nurse in British history, famous for her lamplit patrols of the Barrack Hospital in Scutari, now a district of Istanbul. The hospital was a death trap, with thousands of soldiers from the Crimean front succumbing to typhus, cholera and dysentery as they tried to recover from their wounds in cramped conditions next to the sewers. Nightingale, who did her best, initially believed that the death toll was due to lack of food and supplies. Then, in the spring of 1855, a sanitary commission sent from London cleaned up the hospital, whitewashing the walls, carting away filth and dead animals and flushing out the sewers. The death rate fell sharply.

Nightingale returned to Britain and reviewed the statistics, concluding that she had paid too little attention to sanitation and that most military and medical professions were making the same mistake, leading to hundreds of thousands of deaths. She began to campaign for better public health measures, tighter laws on hygiene in rented properties, and improvements to sanitation in barracks and hospitals across the country. In doing so, a mere nurse had to convince the country’s medical and military establishments, led by England’s chief medical officer, John Simon, that they had been doing things wrong all their lives.

A key weapon in this lopsided battle was statistical evidence. But Nightingale disagreed with Farr on how that evidence should be presented. “The dryer the better” would not serve her purposes. Instead, in 1857, she crafted what has become known as the Rose Diagram, a beautiful array of coloured wedges showing the deaths from infectious diseases before and after the sanitary improvements at Scutari.

When challenged by Bill O’Reilly on Fox News, Trump replied, ‘Hey Bill, Bill, am I gonna check every statistic?’

The Rose Diagram isn’t a dry presentation of statistical truth. It tells a story. Its structure divides the death toll into two periods — before the sanitary improvements, and after. In doing so, it highlights a sharp break that is less than clear in the raw data. And the Rose Diagram also gently obscures other possible interpretations of the numbers — that, for example, the death toll dropped not because of improved hygiene but because winter was over. The Rose Diagram is a marketing pitch for an idea. The idea was true and vital, and Nightingale’s campaign was successful. One of her biographers, Hugh Small, argues that the Rose Diagram ushered in health improvements that raised life expectancy in the UK by 20 years and saved millions of lives.

What makes Nightingale’s story so striking is that she was able to see that statistics could be tools and weapons at the same time. She educated herself using the data, before giving it the makeover it required to convince others. Though the Rose Diagram is a long way from “the dryest of all reading”, it is also a long way from bullshit. Florence Nightingale realised that the truth about public health was so vital that it could not simply be recited in a monotone. It needed to sing.

The idea that a graph could change the world seems hard to imagine today. Cynicism has set in about statistics. Many journalists draw no distinction between a systematic review of peer-reviewed evidence and a survey whipped up in an afternoon to sell biscuits or package holidays: it’s all described as “new research”. Politicians treat statistics not as the foundation of their argument but as decoration — “spray-on evidence” is the phrase used by jaded civil servants. But a freshly painted policy without foundations will not last long before the cracks show through.

“Politicians need to remember: there is a real world and you want to try to change it,” says Will Moy, the director of Full Fact. “At some stage you need to engage with the real world — and that is where the statistics come in handy.”

That should be no problem, because it has never been easier to gather and analyse informative statistics. Nightingale and Farr could not have imagined the data that modern medical researchers have at their fingertips. The gold standard of statistical evidence is the randomised controlled trial, because using a randomly chosen control group protects against biased or optimistic interpretations of the evidence. Hundreds of thousands of such trials have been published, most of them within the past 25 years. In non-medical areas such as education, development aid and prison reform, randomised trials are rapidly catching on: thousands have been conducted. The British government, too, has been supporting policy trials — for example, the Education Endowment Foundation, set up with £125m of government funds just five years ago, has already backed more than 100 evaluations of educational approaches in English schools. It favours randomised trials wherever possible.

The frustrating thing is that politicians seem quite happy to ignore evidence — even when they have helped to support the researchers who produced it. For example, when the chancellor George Osborne announced in his budget last month that all English schools were to become academies, making them independent of the local government, he did so on the basis of faith alone. The Sutton Trust, an educational charity which funds numerous research projects, warned that on the question of whether academies had fulfilled their original mission of improving failing schools in poorer areas, “our evidence suggests a mixed picture”. Researchers at the LSE’s Centre for Economic Performance had a blunter description of Osborne’s new policy: “a non-evidence based shot in the dark”.

This should be no surprise. Politicians typically use statistics like a stage magician uses smoke and mirrors. Over time, they can come to view numbers with contempt. Voters and journalists will do likewise. No wonder the Guardian gave up on the idea that political arguments might be settled by anything so mundane as evidence. The spin-doctors have poisoned the statistical well.

But despite all this despair, the facts still matter. There isn’t a policy question in the world that can be settled by statistics alone but, in almost every case, understanding the statistical background is a tremendous help. Hetan Shah, the executive director of the Royal Statistical Society, has lost count of the number of times someone has teased him with the old saying about “lies, damned lies and statistics”. He points out that while it’s easy to lie with statistics, it’s even easier to lie without them.

Perhaps the lies aren’t the real enemy here. Lies can be refuted; liars can be exposed. But bullshit? Bullshit is a stickier problem. Bullshit corrodes the very idea that the truth is out there, waiting to be discovered by a careful mind. It undermines the notion that the truth matters. As Harry Frankfurt himself wrote, the bullshitter “does not reject the authority of the truth, as the liar does, and oppose himself to it. He pays no attention to it at all. By virtue of this, bullshit is a greater enemy of the truth than lies are.”

 

Written for and first published in the FT Magazine

Free email updates

(You can unsubscribe at any time)

20th of April, 2016HighlightsOther WritingComments off
Marginalia

Three pieces of Brexit Bullshit

A referendum on UK membership of the European Union is scheduled for June 23: dodgy statistics ahoy.

“Ten Commandments — 179 words. Gettysburg address — 286 words. US Declaration of Independence — 1,300 words. EU regulations on the sale of cabbage — 26,911 words”

Variants of this claim have been circulating online and in print. It turns out that the “cabbage memo” is a longstanding urban myth that can be traced back to the US during the second world war. Variants have been used to berate bureaucrats on both sides of the Atlantic ever since.

Part of the bullshit here is that nobody ever stops to ask how many words might be appropriate for rules on fresh produce. Red Tractor Assurance, the British farm and food standards scheme, publishes 56 different protocols on fresh produce alone. The cabbage protocol is 28 pages long; there is a separate 28-page protocol on pak choi and choi sum. None of this has anything to do with the EU.

Three million jobs depend on the EU

This claim is popular among “Remain” advocates — most famously the former deputy prime minister Nick Clegg. What makes this claim bullshit is that it could easily be true, or utterly false, and it all hangs on the definition of “depend”.

The claim is that “up to 3.2 million jobs” were directly linked to exports of goods and services to other EU countries. That number passes a quick reality check: it’s about 10 per cent of UK jobs, and UK exports to the EU are about 10 per cent of the UK economy.

But even if “up to” 3.2 million jobs depend on trade with the EU, that does not mean they depend on membership of the EU. Nobody proposes — or expects — that trade with the EU will just stop. Three million jobs might well be destroyed if continental Europe was to sink beneath the waves like Atlantis, but that is not what the referendum is about.

EU membership costs £55m a day

This one is from Ukip leader Nigel Farage, who says membership amounts to more than £20bn a year. In fact, the UK paid £14.3bn to the EU in 2014 and got £6bn back. The net membership fee, then, was £8.3bn, less than half Farage’s number.

But even the correct number is little use without context. It is, for example, just over 1 per cent of UK public spending. Not nothing, but not everything either. And non-member states such as Norway and Switzerland pay large sums to the EU to retain access to the single market, so Brexit would not make this bill disappear.

The membership fee is small relative to the plausible costs and benefits of EU membership, positive or negative. If EU membership is good for Britain then £8.3bn is cheap. And if the EU is holding Britain back, then a few billion on membership is the least of our worries.

 

Written as a sidebar for “How Politicians Poisoned Statistics“, and first published in the FT Magazine.

Free email updates

(You can unsubscribe at any time)

16th of April, 2016MarginaliaOther WritingComments off
Undercover Economist

Why one size doesn’t fit all

‘The most notorious example of this “compromise effect” is our tendency to plump for the second-cheapest bottle on the wine list’

Most Tuesday afternoons, in the lazy late hour between the end of school and the start of Brownies, you can find me ensconced in the café of the local Marks and Spencer, sipping a hot chocolate with the younger Miss Harford.

Recently, the café has taken the unusual step of radically simplifying the drinks menu. All the standards are there, of course: tea, cappuccino, hot chocolate. But the size options have been covered over with masking tape. Gone are the “small” and the “large”. Now you can have any size of hot beverage you like, as long as it’s medium.

I did ponder registering a note of protest, since a medium hot chocolate contains more sugar than any Brownie can handle. (For reasons that escape me, hot chocolate dodged George Osborne’s much-vaunted new tax on sugary drinks.) But when I perused the comments book, I realised that my own complaint would barely register amid pages of objections from the café’s octogenarian customers.

Perhaps M&S hopes to save money by rationalising the crockery. Or perhaps they simply guess that simplicity is attractive. A few years ago, Debenhams, another British household name with a frumpy image, tried the slogan “Say goodbye to coffee confusion”. Debenhams rebranded cappuccino as “frothy coffee” and caffè latte as “really, really milky coffee”, winning headlines for the most patronising publicity stunt in living memory.

But simplicity can sell. A famous study by psychologists Sheena Iyengar and Mark Lepper offered samples of speciality jam to customers of a high-end supermarket. (Loyal readers may recall the column I wrote about this in November 2009.) Offered a choice of six types of jam, a third of customers went on to make a purchase. Offered a choice of 24, almost nobody did. Iyengar and Lepper concluded that choice can overwhelm and discourage us.

Yet it is unclear how widespread this “choice demotivates” effect is. The original Iyengar-Lepper results, like many in psychology, seem to be fragile. Several follow-up studies have failed to find evidence for the effect. (There is no shame in this; that’s science at work.)

Many successful businesses, from supermarkets to Starbucks, offer a vast range without scaring away their customers. A first-time visitor to Starbucks might be confused, but regulars work it out. And clever design can prevent choice seeming overwhelming. Starbucks offers about 100,000 drink combinations — millions, once the syrups are taken into account — but the menu seems much simpler than that.

What surprised me about M&S’s decision to serve only one size of drink was that it was giving up three advantages. The first is the simplest: offering more choice lets people get closer to exactly what they desire. I want a small drink for a small child — why not sell it to me?

The second advantage is subtler: a company can offer drinks with different margins, in the hope that they can present a bargain to price-sensitive customers, while hoovering up more from customers who are more carefree with their cash. Much of the input cost of a pricey large hot chocolate — staff time, rent, space in the dishwasher — is the same for the cheaper small hot chocolate. By offering more profitable larger drinks alongside the small ones, a café can attract bargain-hunters while still profiting from lavish spenders.

There’s a third advantage to offering a wide number of choices: the extreme choices frame the central ones and can influence what customers buy. The late Amos Tversky, an influential psychologist, once pointed to a Williams-Sonoma bread maker that went on sale for $279 and was a flop. When another bread maker hit the shelves, priced at $429, sales of the original model surged. Why? Well, customers weren’t sure that they wanted to buy a bread maker, nor what price might be reasonable, until they had a choice of two. Suddenly the choice became clear: if they did want to buy a bread maker, they wanted the cheaper one.

The pattern behind this behaviour was demonstrated more rigorously in research by Tversky and Itamar Simonson: you can boost demand for a £400 camera by placing it next to a £4,000 camera, or a £200 handbag next to a £1,000 handbag. Offer three options and people choose the middle one. Offer several and people avoid the extremes. The most notorious example of this “compromise effect” is our tendency to plump for the second-cheapest bottle on the wine list.

Starbucks seems to understand the compromise effect very well. Its menu includes the colossal “trente” — that’s 30 US ounces, more than one-and-a-half British pints or nearly a litre. It omits the far more practical 8oz “short” cappuccino. The effect is to change our sense of what the compromise choice is. A16oz “grande”, three times the size of a classic Italian cappuccino, no longer seems like a milky behemoth but a happy medium.

The curious thing is that the short cappuccino is available on request. Starbucks want to be able to offer a drink that is something close to a properly sized cappuccino, but they don’t want that option cluttering up their menu and pulling customers towards smaller, less profitable products.

M&S tells me that medium is the most popular size of drink. Of course it is. They’re experimenting with the simplification in a few locations to see how it works out. Good for them; you never know until you try. But the complaints book suggests the experiment may be short-lived.

Written for and first published at ft.com.

Free email updates

(You can unsubscribe at any time)

Undercover Economist

Delusions of objectivity

‘“Naive realism” is the seductive sense that we’re seeing the world as it truly is, without bias or error’

“Have you ever noticed when you’re driving,” the comedian George Carlin commented, “that anybody driving slower than you is an idiot, and anyone going faster than you is a maniac?”

True enough. But when you think for a moment about Carlin’s quip, how could it be otherwise? You’ve made a decision about the appropriate speed for the driving conditions, so by definition everybody else is driving at a speed that you regard as inappropriate.

If I am driving at 70 and pass a car doing 60, perhaps my view should be, “Hmm, the average opinion on this road is that the right speed is 65.” Almost nobody actually thinks like this, however. Why not?

Lee Ross, a psychologist at Stanford University and co-author of a new book, The Wisest One in the Room, describes the problem as “naive realism”. By this he means the seductive sense that we’re seeing the world as it truly is, without bias or error. This is such a powerful illusion that whenever we meet someone whose views conflict with our own, we instinctively believe we’ve met someone who is deluded, rather than realising that perhaps we’re the ones who could learn something.

The truth is that we all have biases that shape what we see. One early demonstration of this was a 1954 study of the way people perceived a college-football game between Dartmouth and Princeton. The researchers, Albert Hastorf and Hadley Cantril, showed a recording of the game to Dartmouth students and to Princeton students, and found that their perceptions of it varied so wildly that it is hard to believe they actually saw the same footage: the Princeton students, for example, counted twice as many fouls by Dartmouth as the Dartmouth students did.

A more recent investigation by a team including Dan Kahan of Yale showed students footage of a demonstration and spun a yarn about what it was about. Some students were told it was an anti-abortion protest in front of an abortion clinic; others were told it was a protest outside an army recruitment office against the military’s (then) policy of “don’t ask, don’t tell”.

Despite looking at exactly the same footage, the experimental subjects drew sharply different conclusions about how aggressive the protesters were being. Liberal students were relaxed about the behaviour of people they thought were gay-rights protesters but worried about what the pro-life protesters were doing; conservative students took the opposite view. This was despite the fact that the researchers were asking not about the general acceptability of the protest but about specifics: did the protesters scream at bystanders? Did they block access to the building?

We see what we want to see. We also tend to think the worst of the “idiots” and “maniacs” who think or act differently. One study by Emily Pronin and others asked people to fill in a survey about various political issues. The researchers then redistributed the surveys, so that each participant was shown the survey responses of someone else. Then the participants were asked to describe their own reasoning and speculate about the reasoning of the other person.

People tended to say that they were influenced by rational reasons such as “attention to fact”, and that people who agreed with them had similar motives. Those who disagreed were thought to be seeking “peer approval”, or acting out of “political correctness”. I pay attention to facts but you’re a slave to the approval of your peers. I weigh up the pros and cons but you’re in the pocket of the lobbyists.

Even when we take a tolerant view of those who disagree with us, our empathy only goes so far. For example, we might allow that someone takes a different view because of their cultural upbringing — but we would tend to feel that they might learn the error of their ways, rather than that we will learn the error of ours.

Pity the BBC’s attempts to deliver objective and neutral coverage of a politicised issue such as the British referendum on leaving the EU. Eurosceptics will perceive a pro-Brussels slant, Europhiles will see the opposite. Both sides will assume corruption, knavery or stupidity is at play. That is always possible, of course, but it is also possible that passionate advocates simply don’t recognise objectivity when they see it.

But then how could the situation be otherwise? If any media outlet criticises a political position that you personally admire, there is a contradiction to be resolved, and an easy way to explain the disagreement is to conclude either that the media are biased, or that you are. You can guess which choice people instinctively make. Small wonder that careful studies of media bias in the US show that most newspapers and radio or TV stations don’t try to persuade their readers and viewers; instead, they pander to the biases of their audience.

It is hard to combat naive realism because the illusion that we see the world objectively is such a powerful one. At least I’ve not had to worry about it too much myself. Fortunately, my own perspective is based on a careful analysis of the facts, and my political views reflect a cool assessment of reality rather than self-interest, groupthink or cultural bias. Of course, there are people to the left of my position. They’re idiots. And the people on my right? Maniacs.

Written for and first published at ft.com.

Free email updates

(You can unsubscribe at any time)

Marginalia

What I’ve been reading in March

Uncle Petros and Goldbach’s Conjecture by Apostolos Doxiadis – a remarkable short novel about a mysterious uncle who tries to solve a great mathematical mystery. Apparently he failed – or did he? What does “failure” look like? A lovely book about how we make the big choices in life. (Also, you’ll learn some maths.) (UK) (US)

Getting Organised in the Google Era by Douglas Merrill. Good sense about how to use email, calendars, cloud backups etc. Personally I didn’t learn that much from this book, but only because I’m already following much of the advice. I think Merrill knows his stuff. This book is a few years old now (2011) so some of the specific topics about software are dated. Still, recommended. (UK) (US)

Logicomix by Doxiadis and Papadimitriou. How did I not know that this book existed? It’s a masterpiece – a graphic novel about logic, mathematics and the life of Bertrand Russell. I learned a lot even though this is an area I’ve studied. The art is excellent and so is the storytelling. Way, way above the usual “Comic Book Guide to XXX” format – a great achievement. (UK) (US)

Naked Statistics by Charles Wheelan. Wheelan wrote the excellent “Naked Economics” (UK) (US) – he really knows his stuff and is a witty writer. Excellent. (UK) (US)

And if you’re in the mood for reading, there’s always my latest: The Undercover Economist Strikes Back.

Free email updates

(You can unsubscribe at any time)

4th of April, 2016MarginaliaComments off
Undercover Economist

Trump, trade and ‘the China shock’

‘Freer trade has inflicted a more grievous toll than economists, myself included, had expected’

It hasn’t escaped the notice of pundits that the political iconoclasts Bernie Sanders and Donald Trump have something in common: they’re sceptical about trade. Trump, for example, has riffed expansively: “We don’t win any more. We don’t beat China in trade. We don’t beat Japan . . . We can’t beat Mexico, at the border or in trade.” Sanders expressed his concerns with a little more precision: “While bad trade agreements are not the only reason why manufacturing jobs in the US have declined, they are an important factor.”

Both men have vastly outperformed expectations in the primary campaigns. There are many reasons for that but perhaps the simplest explanation is that freer trade has inflicted a more grievous toll than economists, myself included, had expected.

Fifteen years ago, the conventional economic wisdom was that free trade was almost unambiguously a good idea. Here’s the basic logic. There are two ways for the British to get hold of wine. We can grow and press our own grapes, or we can make something that the French want and trade with them. If we’re good at making, say, computer games and the French are good at making wine, then trading is the better way to get what we want.

The idea that we might, Trumpishly, “beat the French in trade” sounds appealing but is incoherent. And while a British Sanders might point to the loss of jobs in the UK wine industry, that would miss the gains in the software industry. There is little economic difference between a tariff on the import of French wine and a tariff on the export of British software.

Here’s a parable beloved of economists. An entrepreneur announces a technological breakthrough: he has a machine that can disintegrate computer game discs and reconstitute the atoms into fine wine. He sets up a factory on the coast of Kent with the machine inside. Computer games go in, and cases of wine emerge. But then an investigative reporter from the Financial Times gains access to the factory and finds that there is no machine — just a dock where a forklift truck operator busily unloads French wine from a boat, replacing it with computer games for export to the French market. Should we care? From the point of view of the British, isn’t France merely a technology for converting computer games into wine?

With formal models to back up this sort of story, most economists took the view that when countries lower their trade barriers, even unilaterally, they prosper. What the British wine industry loses, the UK computer games industry gains. Meanwhile, consumers get better and cheaper wine into the bargain.

It was always clear that, despite the win-win nature of trade at the national level, freer trade could create losers — such as British vineyards and French computer game studios. But the conventional wisdom was that these losses were both small and fixable with the right policies of retraining or redistribution. Most importantly, people who lost their jobs could find new ones in booming export industries.

Admittedly, it was evident even 20 years ago that median household incomes were stagnating in the US, inequality was rising in anglophone countries, and manufacturing employment was steadily falling. But these trends seemed to owe more to technological change than to globalisation.

I’ve been phrasing all this “conventional wisdom” in the past tense but, for the most part, it stands up. However, it is acquiring an important and depressing footnote. A new research paper, “The China Shock”, from David Autor, David Dorn and Gordon Hanson, is part of a rethink under way in the economics profession.

Autor and his colleagues try to zoom in on the impact of China’s emergence as a trading power. China’s rise has been dramatic, driven almost entirely by internal policy changes inside China, and has had a differential effect on different regions and industries. For example, Tennessee and Alabama are both US manufacturing centres exposed to global competition. But Tennessee’s furniture manufacturing industry is much more exposed to China in particular than is Alabama’s heavier manufacturing industries. This helps the researchers to figure out with more confidence what the impact of the China shock has been.

Autor, Dorn and Hanson conclude that the American workers who have been hurt by competition with China have been hurt more deeply, and for a longer period, than many economists predicted. Employment has fallen in industries exposed to trade competition, as expected. But it has not shown much signs of rising in export-oriented sectors.

The US labour market is less flexible than we thought, it seems. In a simplified economic model, workers move smoothly to a new home, a new industry, even a new level of education. In practice, Autor and his colleagues find that communities hit by Chinese competition often do not adapt; they wither. It may take a generation or two, rather than a few years, to adjust.

In the long run, of course, that adjustment will happen — just as we have adjusted to the decline of agricultural labour or the need for typewriter repairs. But the long run is longer than many economists feared. It is easy to see why supporters of Trump and Sanders have run out of patience.

Written for and first published at ft.com.

Free email updates

(You can unsubscribe at any time)

Previous

Elsewhere

  • 1 Twitter
  • 2 Flickr
  • 3 RSS
  • 4 YouTube
  • 5 Podcasts
  • 6 Facebook

Books

  • Messy
  • The Undercover Economist Strikes Back
  • Adapt
  • Dear Undercover Economist
  • The Logic of Life
  • The Undercover Economist

About Tim

Tim Harford is an author, columnist for the Financial Times and presenter of Radio 4's "More or Less".
Learn More ›

Subscribe to TimHarford.com

Enter your email address to receive notifications of new articles by email.

Tim’s Tweets

Do NOT follow this link or you will be banned from the site!