Tim Harford The Undercover Economist

Undercover EconomistUndercover Economist

My weekly column in the FT Magazine on Saturday’s, explaining the economic ideas around us every day. This column was inspired by my book and began in 2005.

Undercover Economist

The truth about our norm core

‘Social pressure matters but it is not the only thing that matters. Facts can trump groupthink’

While not quite as infamous as Philip Zimbardo’s Stanford prison simulation, or Stanley Milgram’s obedience research, Solomon Asch’s conformity experiments remain among the most celebrated in psychology. In 1951, Asch’s research showed that our judgments about simple factual matters can be swayed by what people around us say. The finding echoed down the decades. Milgram found in 1961 that people were willing to administer apparently dangerous electric shocks when ordered to do so by an experimenter. In 1971, Zimbardo set up an imitation prison in a Stanford University basement with subjects given the role of guards and prisoners, then observed as the guards humiliated the prisoners.

Between them, the three academic psychologists taught us that in order to fit in with others, we are willing to do almost anything. That, at least, is what we are told. The truth, as so often, is more interesting.

Asch gave his subjects the following task: identify which of three different lines, A, B or C, was the same length as a “standard” line. The task was easy in its own right but there was a twist. Each individual was in a group of seven to nine people, and everyone else in the group was a confederate of Asch’s. For 12 out of 18 questions they had been told to choose, unanimously, a specific incorrect answer. Would the experimental subject respond by fitting in with the group or by contradicting them? Many of us know the answer: we are swayed by group pressure. Offered a choice between speaking the truth and saying something socially convenient, we opt for social convenience every time.

But wait — “every time”? In popular accounts of Asch’s work, conformity tends to be taken for granted. I often describe his research myself in speeches as an example of how easily groupthink can set in and silence dissent. And this is what students of psychology are themselves told by their own textbooks. A survey of these textbooks by three psychologists, Ronald Friend, Yvonne Rafferty and Dana Bramel, found that the texts typically emphasised Asch’s findings of conformity. That was in 1990 but when Friend recently updated his work, he found that today’s textbooks stressed conformity more than ever.

This is odd, because the experiments found something more subtle. It is true that most experimental subjects were somewhat swayed by the group. Fewer than a quarter of experimental subjects resolutely chose the correct line every time. (In a control group, unaffected by social pressure, errors were rare.) However, the experiment found that total conformity was scarcer than total independence. Only six out of 123 subjects conformed on all 12 occasions. More than half of the experimental subjects defied the group and gave the correct answer at least nine times out of 12. A conformity effect certainly existed but it was partial.

This surprised me, and it may surprise others who have read popular accounts of the so-called conformity studies. I doubt that it surprised Asch. Conformity was already a well-established finding by 1951, and his experiments were designed to contrast with earlier research on social norms. This previous research showed that people conformed to social pressure in situations where there was no clear correct answer — for instance, when asked to identify which of two ungrammatical sentences was the most ungrammatical. But Asch wanted to know if peer pressure would also wield influence when the crowd was unambiguously wrong. His research provided an answer: social pressure is persuasive but, for most people, the facts are more persuasive still.

Myths about famous experiments have always grown in the telling. It seems most unlikely that Archimedes ran naked through the streets of Syracuse yelling “Eureka!”, and an apple probably did not strike Newton’s head. But there seems to be something particularly attractive about these famous psychology experiments that paint us all as sheep — even when the experiments may have been flawed, impossible to replicate or (as with Asch’s work) have simply found something much more subtle than the myth would have us believe.

The psychologist Christian Jarrett comments, “the resistance to tyranny shown by many participants in Zimbardo’s prison study has largely been ignored, and so, too, has the disobedience shown by many participants in Milgram’s seminal work.”

Zimbardo’s Stanford prison experiment was shocking stuff, and raised serious questions about research ethics. But we should also ask questions about what Zimbardo really found. By his own admission he gave a strong steer to the guards, and cast himself as their ally in a quest to dehumanise the prisoners. “We’re going to take away their individuality in various ways,” he told them. Other psychologists have suggested that this was more a test of obedience to Zimbardo than a demonstration that sadism blooms given the opportunity.

Few textbook accounts of the study mention Zimbardo’s attempt to influence the guards; nor do they point out that two-thirds of the guards refrained from sadism.

Social pressure matters but it is not the only thing that matters. Solomon Asch showed that facts can trump groupthink. It would be ironic if our own biased recollections of his finding proved him wrong.

Written for and first published at ft.com.

Undercover Economist

Down with mathiness!

‘In the recent UK election campaign, a diet of numbers was stuffed into voters like feed into French ducks’

The American Economic Review isn’t usually the place for trash talk but a brief new article by Paul Romer is the closest academic economics is likely to come to a spat between boxers at a pre-fight weigh-in. Romer, a professor at New York University, is worried that a new trend in economics — “mathiness” — is polluting the discipline. And he names names — including Robert Lucas and Edward Prescott, both Nobel laureates, and inequality guru Thomas Piketty.

In a follow-up comment, “Protecting the Norms of Science in Economics”, Romer says: “I point to specific papers that deserve careful scrutiny because I think they provide objective, verifiable evidence that the authors are not committed to the norms of science.”

Romer adds that if his suspicions are confirmed, such people should be ostracised — suggesting that Nobel Prize winners should be ejected from academic discussion because of their intellectual bad faith. This is strong stuff.

Romer, though, has rarely stuck to the academic script. In the late 1980s he developed a new approach to thinking about economic growth that mathematically modelled the development and spread of ideas, an achievement that many regard as worthy of the Nobel memorial prize in economics. But Romer then drifted away from academia, first founding an online learning platform called Aplia, and then campaigning for a radical development idea, “charter cities”.

Does economics have a mathiness problem? Many casual observers would say, “of course”. Economics has a reputation for producing rigorous nonsense.

But Romer’s attack is much more focused. He doesn’t mean that economics uses too much mathematics but that some economic theorists are pushing an ideological agenda and using fancy mathematics to disguise their intentions. They can redefine familiar words to mean unfamiliar things. They can make unrealistic assumptions. They can take hypothetical conclusions and suggest they have practical significance. And they can do all these things with little fear of detection, behind a smokescreen of equations. If Romer is right, some economics papers are Orwellian Newspeak dressed up as calculus.

In his short essay “Politics and the English Language”, Orwell argued that there was a “special connection between politics and the debasement of language”. While some people wish to communicate clearly, the political writer prefers a rhetorical fog. And the fog can spread. Writers who should know better imitate sloppy writing habits. Readers become jaded and stop hoping that anyone will tell them the truth.

Romer fears a similar rot at the heart of economics. As some academics hide nonsense amid the maths, others will conclude that there is little reward in taking any of the mathematics seriously. It is hard work, after all, to understand a formal economic model. If the model turns out to be more of a party trick than a good-faith effort to clarify thought, then why bother?

Romer focuses his criticism on a small corner of academic economics, and professional economists differ over whether his targets truly deserve such scorn. Regardless, I am convinced that the malaise Romer and Orwell describe is infecting the way we use statistics in politics and public life.

There being more statistics around than ever, it has never been easier to make a statistical claim in service of a political argument.

In the recent election campaign in the UK, a diet of numbers was stuffed into voters like feed into French ducks. A fact-checking industry sprang up to scrutinise these numbers — I was part of it — but the truth is that most of the numbers were not false, unhelpful. Instead of simply verifying or debunking the latest number, fact checkers found themselves spending much effort attempting to purify muddied waters.

This is infuriating — for the public, for the fact checkers, and for the scientists and statisticians who take such pains to gather evidence. Imagine their dismay when the politicians seize that evidence and hold it up for protection like a human shield. Good statistics matter; without them it is almost impossible to understand the modern world. Yet when statistics are dragged into political arguments, it is not the reputation of politics that suffers but the reputation of statistics. The endgame isn’t pretty: it becomes too much trouble to check statistical claims, and so they are by default assumed to be empty, misleading or false.

Just as the antidote to Newspeak isn’t to stop using language, the antidote to mathiness isn’t to stop using mathematics. It is to use better maths. Orwell wanted language to be short, simple, active and direct. Romer wants economists to use maths with “clarity, precision and rigour”. Statistical claims should be robust, match everyday language as much as possible, and be transparent about methods.

Some critics believe that economics should conduct itself in plain English at all times. This is, I think, unreasonable. Mathematics offers precision that English cannot. But it also offers a cloak for the muddle-headed and the unscrupulous. There is a profound difference between good maths and bad maths, between careful statistics and junk statistics. Alas, on the surface, the good and the bad can look very much the same.

Written for and first published at ft.com.

Undercover Economist

Mind the fair trade gap

‘If fair trade does deliver higher incomes for farmers, it may prove too successful for its own good’

In 2001, the world price of coffee sank to its lowest ebb for decades, threatening dreadful hardship for the often-poor farmers who grow the sainted berry. It was also around that time that fair trade coffee seemed to come of age, with a common certification mark launched in 2002, and the product becoming a familiar sight in supermarkets and coffee chains.

The premise of fair trade is that the disparity between poor coffee farmers and prosperous drinkers presents both a problem and an opportunity. The problem is that farmers often live a precarious existence: geographically isolated and growing a crop with a volatile price. The opportunity is that many western consumers care about the earnings and conditions of the people who grow their coffee, and have some money to spare if only it might reach those people.

Unlike a taxi driver or a waiter, you can’t just tip the guy who grew your coffee. The fair trade answer to the conundrum is a labelling scheme: an inspector verifies that all is well on the farm, with good conditions and a higher price paid for coffee; this information is conveyed to consumers by way of a recognisable trademark, the most famous of which is the Fairtrade logo. It’s an appealing idea — a voluntary scheme that helps people who want to help people. (Or rather, several voluntary schemes: there is more than one fair trade label, alongside diverse certification schemes such as Organic or Rainforest Alliance.) Who wouldn’t want a better deal for farmers who are poor and work hard? But there are problems with the idea too.

The most obvious problem is that this labelling scheme costs money. Flocert, a certification body set up by the Fairtrade Labelling Organization, charges farmer co-operatives €538 merely to apply for certification, plus an initial audit fee of €1,466 even for a small co-op. Cynics might suspect bureaucratic bloat but the costs may well be real. It cannot be cheap to check pay and conditions in some remote Peruvian coffee plantation. But every euro spent on certification is a euro that the farmer cannot spend on his family. And larger co-operatives from richer, better-connected countries are more likely to find it worthwhile to pay for certification. For this reason, economist and fair trade critic Ndongo Sylla says that fair trade benefits “the rich”. That seems too strong; but it is certainly a challenge for the fair trade model to reach the poorest.

A second problem is that fair trade certification cannot guarantee fair trade sales. If coffee importers want to put the Fairtrade mark on their coffee, they must find a Fairtrade certified producer and pay the Fairtrade price, which reflects both a modest premium and a guaranteed minimum price. But importers are not obliged to buy fair trade coffee and may avoid it when it gets too expensive, exactly when the premium is most needed. A study by Christopher Bacon found that during the price slump of 2000 and 2001, Nicaraguan coffee farmers were earning twice as much per pound when selling fair trade coffee as when selling the uncertified stuff. But much of their coffee could not find a buyer at such rates and was sold at market rates instead; as a result, the average price premium, while substantial, was much lower at around a third.

Another study, by Tina Beuchelt and Manfred Zeller, found the fair trade certified farmers in Nicaragua started at a similar income level to conventional farmers and, if anything, slipped backwards. A recent survey by Raluca Dragusanu, Nathan Nunn, and Daniele Giovannucci was more upbeat but still found the evidence in favour of fair trade “mixed and incomplete”.

A final irony is that if fair trade does deliver higher incomes for farmers, it may prove too successful for its own good. If coffee farmers are able to sell more coffee at a premium price, more people will want to become coffee farmers. One possible result is that the market price for uncertified coffee falls and, on balance, coffee farmers are no better off.

As the development economist Paul Collier once wrote, fair trade certified farmers “get charity as long as they stay producing the crops that have locked them into poverty”. It is a telling point. For all the good I may wish the people who make my coffee, a globalised tip jar makes a precarious foundation for their future prosperity.

Written for and first published at ft.com.

Undercover Economist

Why democratic elections are always flawed

I sometimes wonder if we expect more than we should from democracy

Most Britons are unhappy with the result of the UK general election. That is the logical conclusion, given that 63 per cent of voters cast their vote for someone other than David Cameron’s Conservatives. Nevertheless, the Conservatives surprised even themselves by winning more seats than every other party put together.

From the point of view of the losers, this state of affairs seems outrageous. It is sometimes said that splits on the left of British politics have prevented what should have been a solid leftwing majority and allowed the rightwing views of a minority to prevail. Yet the right is also split: the Conservatives and the UK Independence Party attracted more than 50 per cent of the vote between them. Leftwingers frustrated by the election result should blame the voters before they blame the voting system.

Critics of the British voting system do have a point. An analysis by Jack Blumenau and Simon Hix of the London School of Economics suggests that the disparity between votes cast and seats won has been widening for many decades, the consequence of the large number of votes now cast for smaller parties.

Since each seat is decided separately, votes cast for losing candidates simply do not count. It is possible to stack up a hefty pile of such votes while winning only a single seat — just ask Ukip’s Nigel Farage, who failed to be elected despite leading a party that attracted 3.9 million votes. The Conservatives earned about 34,000 votes per seat won, and Labour about 40,000 votes. The Scottish Nationalists needed just 26,000 votes per seat. Nick Clegg’s derided Liberal Democrats required more than 300,000 votes for each of their eight seats. Supporters of both Ukip and the Liberal Democrats might well feel disenfranchised, as might the Greens and the many Scots who voted for parties other than the SNP. Still, rules are rules and everyone knew the rules before they started to play the game.

Yet rules can be changed. And perhaps they should be. But to what? Clever schemes abound: the D’Hondt method offers something close to proportional representation while maintaining a link to local constituencies; the Borda count attempts to measure the strength of preferences; the alternative vote, AV, is designed to allow people to cast a conditional vote for whichever of several parties might find itself with a chance of winning.

AV was decisively rejected by the British in a referendum in 2011. But perhaps referendums themselves need looking at. Consider a referendum on an issue such as gay marriage. A small number of people — gay people who might wish to get married — have a tremendous interest in liberalisation. But no matter how strongly they feel, they get just a single vote each and so they have had to wait while the weakly held views of the majority slowly move in a tolerant direction.

Glen Weyl, an economist, argues that in such cases we might want to hold a referendum that allows people to express their strongly held beliefs by buying multiple votes at increasing cost: one vote costs $1; two votes cost $4; 1,000 votes cost $1m. Weyl calls this idea “quadratic voting”. It has some appealing theoretical properties but to the layperson it looks alarming. Expect to see it used in TV talent shows.

I am all in favour of improving institutions when we can but I sometimes wonder if we expect more than we should from democracy. There are two deep reasons why democratic elections are always flawed.

The first is that voters are, quite rationally, rather ignorant about politics. Sensible people vote to express themselves or out of a sense of duty, not because they harbour the illusion that it might be their vote that swings the entire election. Quite sensibly, then, people who devote hours to researching a new phone will not waste time researching which party to support.

The second reason is Nobel laureate Ken Arrow’s “impossibility theorem”, one of the most celebrated and misunderstood results in economics. Arrow’s theorem is often described as showing that there is no voting system that will reflect what society truly prefers. Arrow actually showed something more profound: that it makes little sense to speak of what “society truly prefers”. That very idea is incoherent. And those who expect that a democratic election will ever give society what it “truly prefers” will have to get used to disappointment.

Written for and first published at ft.com.

Undercover Economist

Tax: a Scandinavian solution

‘With tax, our politicians seem determined to make the process as clumsy and painful as possible’

If a politician was a surgeon, faced with the task of amputating your leg, we can well imagine how it would go. First he’d deny that he planned to amputate the leg. Then he’d pass a law making it illegal to amputate the leg. Then he’d say that he’d amputate an investment banker’s leg instead. Finally, he would blame the mess handed to him by the previous surgeon and would begin to rub away at your toes with a cheese grater.

So it is with taxes. It’s no fun paying them but public spending must be paid for somehow. Yet our politicians seem determined to make the process as clumsy, painful and disingenuous as possible.

This may be because politicians see taxes purely in political terms. They believe that the deep problem with taxes is that people do not like paying them, which is why they say, instead, that the taxes will be levied only on multinational corporations, investment bankers and tax dodgers of all stripes. Politicians placate angry voters with tax exemptions and deductions. All this is politically understandable but has the effect of making the taxes much more damaging than they need to be.

The true problem with taxes is quite different. It is that in an effort to pay less tax, people do some extraordinary things. Most obviously and controversially, they’ll adopt odd legal labels that have the effect of reducing their tax bill. Some are fiendishly complex international schemes, playing different tax treaties off against each other and generating corporate profit that each tax authority deems is someone else’s problem. Others are quite simple. All of them are unfair, and all of them generate paperwork.

A second problem, less fussed-about but probably more serious, is that people will change their behaviour rather than just the legal description of that behaviour. For example, some new mothers who want to work will stay at home rather than hire childcare out of heavily taxed income. The mother doesn’t get the career she wanted, and the taxman doesn’t get the tax revenue. Nobody wins.

Two articles in last year’s Journal of Economic Perspectives explore how governments might get more serious about raising taxes. One, by Gabriel Zucman, emphasises that the complexity and inconsistency of different tax systems allow wealthy individuals and multinational companies to exploit cross-border loopholes and avoid tax. Zucman’s calculations suggest that US companies are increasingly booking their profits in what he calls “the main tax havens”, jurisdictions that housed about 2 per cent of US corporate profits in 1984 but 18 per cent in 2013.

But one way to look at the problem of levying high taxes is to ask who has solved it. The answer: Denmark, Norway and Sweden. US tax revenue is about 25 per cent of GDP, the UK and Germany at about 35 per cent, and the Scandinavians at about 45 per cent, according to economist Henrik Jacobsen Kleven. Somehow the Scandinavians have managed to raise large sums from their citizens without destroying their economies. How?

That’s the question that Kleven sets out to answer and, of course, the answer is partly cultural. It is also partly about the comprehensive tax reporting in Scandinavia, which makes outright evasion very difficult. Norwegian tax returns are published for all to examine. (No wonder Gabriel Zucman dreams — perhaps implausibly — of a global financial registry to help track down tax dodgers.)

Not everyone will feel delighted about an all-seeing government determined to invade privacy in the name of higher taxes. But there are other elements of Scandinavian taxation that any government might want to emulate: Scandinavian countries minimise the distortions of their tax system by avoiding the bad habits of politicians in other countries.

Chief among these habits is targeting a narrow tax base. The US tax system is full of ad hoc deductions and exemptions. The UK system needlessly excludes swaths of the economy from tax. Rather than charge a 10 per cent rate of VAT on everything, the UK government charges a 20 per cent rate of VAT on roughly half of what consumers spend. The Danes have a much broader VAT base, and a higher rate too.

The simplest way to broaden the tax base is to dismantle barriers to getting a job. Scandinavian governments subsidise education, transport and care for children and the elderly, all of which help people to work who might otherwise find themselves stuck at home. As a result, even high taxes do not keep them out of the labour market.

That makes sense. If the surgeon really is going to amputate your leg, having a prosthetic replacement would be wise.

Written for and first published at ft.com.

Solve this puzzle and win a Dom Reilly bag

Tim Harford will be interviewing University of Chicago economist Richard Thaler about his new book Misbehaving on stage in London on June 10. Please visit live.ft.com/richard-thaler for more details.

In anticipation of this event, Professor Thaler is setting FT readers a challenge, revisiting a puzzle he set them once before, in 1997.

The task is simple: choose a number between 0 and 100, and supply a short justification for your choice. The winner is the person whose number is closest to two-thirds of the average of all the entries. For example, three entries are submitted: 20, 30 and 40. The average is then 30 and the winning entry is 20, being exactly two-thirds of the average.

In the event of a tie, the prize will go to the person who submits the best justification. Prof Thaler’s decision is final.

The prize for the winning entry is a luxurious weekend bag designed for the FT by Dom Reilly — lightweight, elegant and exquisitely handcrafted in brown full-grain leather with subtle FT branding.

Please send your guess and your justification to: email hidden; JavaScript is required

Competition ends May 31. T&Cs apply. ft.com/thalerconditions

Undercover Economist

The problem with sexed-up statistics

‘Statistics tell us nothing until we understand what is being counted in the first place’

Men think about sex every seven seconds. Eighty-four per cent of women are emotionally unsatisfied with their relationships. Single people in the United States have more sex than married people do. Sixty-nine per cent of people over the age of 35 have had extramarital affairs. People have 40 per cent less sex now than they did 20 years ago. Truth, or myth?

A new book by statistician David Spiegelhalter, Sex by Numbers, runs a statistical comb through our collective sex lives. His book is largely designed to teach us about sexual behaviour — who is doing what with whom and how often — but along the way he manages to impart some important statistical lessons too.

Lesson one is that statistics tell us nothing until we understand what is being counted in the first place. To ask how old people are when they start having sex, or when they stop having sex, or how many sexual partners people typically have, we need a generally agreed definition of “having sex”.

We should not take for granted that we all mean the same thing when we talk about sex. Just ask Bill Clinton, who notoriously declared “I did not have sexual relations with that woman, Miss Lewinsky.” When it later became clear that he had received oral sex from her, he apologised for giving a misleading impression but maintained that “my answers were legally accurate.”

Clinton’s carefully chosen words were in tune with the way most people used language. A survey of US college students conducted in 1991 found that only 40 per cent of them reckoned that oral sex counted as “sex”. (The US Senate implicitly reached a similar conclusion in clearing President Clinton of perjury.)

While Clinton exploited ambiguity, modern scientific surveys of sexual behaviour try to eliminate it. According to definitions used by the UK’s well-regarded National Survey of Sexual Attitudes and Lifestyles, Natsal, Clinton did have sexual relations with Miss Lewinsky.

A second lesson is that we should pay attention to whether statistical work has been done carefully or casually. Consider Time Out magazine’s finding that people have sex 10 times a month if they are in a relationship, though only five times a month if they are married. This is twice as much as more credible surveys have found. As Spiegelhalter observes, Time Out’s method can only tell us about the sexual claims of people who go out of their way to fill in sex surveys. Spiegelhalter is similarly dismissive of the “Trojan US Sex Census”, which announced that Los Angeles was the most sexually active city in America with 135 sex acts per person per year. While a great source of publicity for a manufacturer of condoms, the people who fill in Trojan’s survey don’t represent the rest of us.

Some of the most famous sex researchers are also limited by a lack of representative sampling. Alfred Kinsey found that 37 per cent of men had a homosexual experience resulting in orgasm; Shere Hite reported that 95 per cent of women experienced “emotional and psychological harassment” from their male partners. The underlying research here was politically groundbreaking but we cannot have too much confidence that these numbers are correct.

Hite, for example, distributed questionnaires through university women’s centres, abortion rights groups and other women’s groups; the response rates were less than 5 per cent, making it unclear whether respondents were typical of women as a whole. Kinsey was on the lookout for interesting sexual case histories and so sent his researchers to prisons and to bars famous for being gay meeting places. He may well have captured a broader range of sexual behaviour as a result but at the cost of a representative sample. As the great mathematician John Tukey once told Kinsey, “I would trade all your 18,000 case histories for 400 in a probability sample.” If the aim is to judge what is going on in the population as a whole, Tukey was right.

To revisit the factoids in the first paragraph: most are unproven, the results of unrepresentative surveys. The “seven seconds” claim is an urban myth and provably nonsense. But the final discovery — that we are having 40 per cent less sex — is true. According to the rigorously collected Natsal survey, heterosexually active people aged 16-44 typically had sex five times in the past month back in 1990. By 2010, the number had fallen steadily to three times. Perhaps the next Natsal survey will be able to figure out why.

Written for and first published at ft.com.

Undercover Economist

What a radical Conservative government could do

‘Scrap all mainstream benefit payments — jobseeker’s allowance, child benefit, housing benefit and even the state pension’

Last week I described Anthony Atkinson’s proposals for reducing inequality. Atkinson — a professor of economics at Nuffield College, Oxford — proposes substantially higher income tax rates for everyone earning more than £65,000, a much higher minimum wage, guaranteed public employment, an expansion of universal benefits and much else. It is the agenda one would expect of a courageous Labour party, which of course places it a long way from the agenda that the actual Labour party is proposing.

It seems only fair, then, to offer the same service to the Conservatives: on the off chance that we ever see an economically radical Tory party, what policies might I suggest they embrace?

Step one is to replace the benefit system with a more libertarian form of redistribution. Scrap all the mainstream benefit payments — for example, child benefit, jobseeker’s allowance, housing benefit, winter fuel allowance and even the state pension. Scrap the income tax allowance too. Give all long-term UK residents a taxable basic income of £8,000 a year and charge a flat 40 per cent income tax rate on every penny. The basic income can be phased in on a residency basis over 10 years, ensuring that recent immigrants pay a larger net contribution to the exchequer.

This policy targets poverty rather than inequality. It abolishes much of the bureaucracy that surrounds benefit eligibility, promotes individual responsibility and reduces the stigma of collecting money from the state. It gives everyone, rich and poor, a clear incentive to work. Compared to the current system, it redistributes to the working poor and to the highest earners — both groups of people who are likely to produce more taxable income in response. It is simple, discouraging tax avoidance. And, despite the flat headline rate, the average income tax contribution is progressive: negative for those on low incomes, 10-25 per cent for those on average incomes and approaching 40 per cent for the rich.

People with unusual needs — the severely disabled, for instance — would be helped by a multibillion-pound fund with considerable discretion to make direct cash payments or commission assistance from charities.
A second policy is to privatise the entire school system. Children would receive a £10,000 basic income in a tax-sheltered educational account controlled by parents but usable only for childcare, school or university fees. Compulsory schooling would end at the age of 14 and educational institutions would be competing to attract these pots of tax-free cash with engaging and practical training courses. Any unspent money would be taxed and handed over to the child at the age of 21.

Third, scrap the personal pension system. Both the logic for and the reputation of the existing system is in tatters. With the new flat tax and universal basic income it would also be superfluous. People can save for their retirement in more flexible Isa-style savings accounts and could be nudged into doing so by a default payroll deduction.

A fourth policy must involve the housing market. The current cluster of housing policies (“cluster” is a polite abbreviation for a more appropriate term) ensures slow growth, resentment of immigrants, a crippling housing-benefit bill, inequality growing through luck rather than hard work and innovation, and the direction of potentially productive savings into accumulating unproductive housing wealth. This is a multifunctional policy indeed.

Given that housing benefit is to be abolished by this radical government, there is an urgent need to build large numbers of houses. This would boost the economy and reduce the price of new homes. One possibility, proposed by the Centre for Policy Studies, is the establishment of “pink zones” with lighter planning regulation (the colour represents a dilution of red tape). In these zones, substantial increases in housing could be achieved by a coalition of local authorities, community groups and developers.

However the trick is pulled off, the government must create the conditions for a housing boom — 400,000 new homes a year for five years would do to begin with. It’s ambitious, but necessary after decades of insufficient building.

A final idea: look to broaden the tax base and lower tax rates. Abolishing all VAT exemptions would be a good start, and would provide substantial revenue. A carbon tax would also be well worth introducing, as would more proportionate taxation of housing wealth. The proceeds of these taxes would be needed at first to pay for the universal basic income but the aim would be to reduce universal income tax rate too. A future leftwing government could redistribute within the same framework by increasing the basic income.

That should do the trick for the first term but a Conservative government should also commit to staying in the European Union, which stands in favour of trade, business and hard money; and to leaving the National Health Service alone for a few years just to see how it performs when not being incessantly prodded by politicians.

There you have it: a smaller, less bureaucratic state, innovation in education, redistribution to the poorest, a lower but more transparent income tax to attract the rich, an economic boom on the back of much-needed home-building and affordable housing for all.

Conservative Central Office can thank me later.

Written for and first published at ft.com.

5th of May, 2015Undercover EconomistComments off
Undercover Economist

The truth about inequality

‘One myth is that inequality in the UK has risen since the financial crisis. In fact it has fallen quite sharply’

How serious a problem is inequality? And if it is serious, what can be done about it?

Myths abound. Many people seem to believe that Thomas Piketty’s Capital in the Twenty-First Century showed that wealth inequality is at an all-time high; instead, his data show that wealth inequality has risen only slowly since the 1970s, after falling during the 20th century. In Europe we are thankfully nowhere near the wealth inequality of the past.

Another common belief is that the richest 1 per cent of the world’s population own half the world’s wealth (almost true) and that their share is inexorably increasing (not true). The richest 1 per cent had 48.2 per cent of the world’s wealth in the year 2014, according to widely cited research from Credit Suisse, but that share has fallen and risen over the past 15 years. It is lower now than in 2000 and 2001.

Neither is it clear that global inequality is rising. Average incomes in China and India have risen much faster than those in richer countries; this is a powerful push towards equality of income. But inequality within many countries is rising. Research from Branko Milanović, author of The Haves and the Have-Nots, suggests that the two forces have tended to balance out roughly over the past generation.

One final myth is that inequality in the UK has risen since the financial crisis. In fact, it has fallen quite sharply. “Inequality remains significantly lower than in 2007-08,” said the Institute for Fiscal Studies last summer. That conclusion is based on data through April 2013. The IFS did add, though, that “there is good reason to think that the falls in income inequality since 2007-08 are currently being reversed.”

Given all this, why the sudden anxiety about inequality? The answer is partly political: incomes fell and then stagnated after the financial crisis, and the crisis also made it seem risible to claim that the entrepreneurial activities of the rich would indirectly help the poor. None of this is directly connected to rising inequality but it certainly changes the conversation.

Yet there is more going on than a change in the political wind. By most reasonable measures, inequality of incomes has risen substantially over the past 40 years in both the US and the UK, with a particular surge in the 1980s. That should clarify the issue: the problem is most clearly seen within boundaries of nation states rather than globally; in income rather than wealth; and over the past few decades rather than the past few years. And it is stark enough to need no exaggeration.

I recently attended the launch of Inequality: What Can Be Done?, a book by Anthony Atkinson. Professor Atkinson is the economist who set the stage for younger stars such as Piketty and Emmanuel Saez; his first major paper on the subject of inequality was published in 1970, before either of them was born.

One thing that can be done, says Atkinson, is to use the same old redistributive tools with more vigour. The UK already redistributes income extensively. As Gabriel Zucman of the London School of Economics points out, the UK’s richest fifth had 15 times the pre-tax income of the poorest fifth, but after taxes and benefits they had just four times as much.

For some people that will seem more than enough redistribution. Others will disagree, and Atkinson is one of them. He would like to see the current 45 per cent top rate of tax levied at a much lower level (about £65,000), a new 65 per cent top rate for those earning more than £200,000, a substantially higher minimum wage, a “minimum inheritance” paid to every 18-year-old, guaranteed public employment, more comprehensive taxation of inheritance and property and an expansion of universal benefits.

Like it or loathe it, this is ambitious stuff. I don’t know if a 65 per cent top rate of tax is likely to be counterproductively high and neither does Tony Atkinson. I suspect that it is, and the available evidence provides some support for that suspicion. However, there is a wide margin of uncertainty so Atkinson is right to say that the evidence doesn’t conclusively rule it out.

Atkinson also wants to make market incomes themselves more egalitarian, leaving the welfare state with less to do. Ed Miliband, the Labour leader, once talked of “pre-distribution”, which is an ugly word for the same idea. But neither Miliband nor Atkinson is entirely persuasive about how this might work. Atkinson suggests that competition policy, vetting mergers and breaking up or regulating monopolies, should be used to reduce inequality. Or possibly the state’s support for science and innovation — always important — could favour innovations that complement labour rather than replacing it? In theory all this is possible. But my imagination is not up to the task of figuring out what these labour-complementing innovations might be, nor how the government might help produce them.

The UK general election on May 7 might well produce a Labour-led government but it will be astonishing if that government embraces a redistributive agenda half as ambitious as Atkinson’s. The conversation about inequality has changed quickly — but what mainstream politicians are willing to countenance has not.

Written for and first published at ft.com.

Undercover Economist

The economists’ manifesto

If Britain’s top economists were in charge, what policies would they implement? Tim Harford sets the challenge

It’s often said that economists have too much influence on policy. A critic might say that politicians are dazzled by data-driven arguments and infatuated with the free-market-fetishising practitioners of the dismal science. As a card-carrying economist, I have never been convinced that politicians are the puppets of economists. Still, the idea seemed worth exploring, so I called up some of the country’s most respected economists and presented them with this scenario: after the election, the new prime minister promises to throw his weight behind any policy you choose. What would you suggest?

My selection of economists was mainstream — no Marxists or libertarians — but arbitrary. There is no pretence of a representative survey here. But there were common threads, some of which may surprise.

Let’s start with the deficit which, if we are to judge by column inches alone, is the single most important economic issue facing the country. Yet with the chance to push any policy they wished, none of my economic advisers expressed any concern about it. Indeed several wanted some form of increased spending and were happy to see that financed through borrowing or even printing money.

Economists have a reputation for being low-tax, free-market champions. Yet none of my panel fretted about red tape, proposed any tax cuts or mentioned free trade. Other untouched issues included the National Health Service, immigration and membership of the EU. Nobody suggested any changes to the way banks are regulated or taxed.

Less surprising is that several economists suggested structures that would put decision making at arm’s length from politicians, delegating it to technocrats with the expertise and incentives to do what is right for Britain. The technocracy already has several citadels: the Bank of England’s Monetary Policy Committee, the National Institute for Health and Care Excellence, the Competition Commission and the regulators of privatised utilities. My advisers wanted more of this. That is economics for you: when a political genie offers you whatever policy wish you desire, why not simply wish to have more wishes?

Nick Stern
Former chief economist of the World Bank, professor at the London School of Economics

Nick Stern will forever be associated with the Stern Review, a report into the economics of climate change published in 2006. He hasn’t stopped banging this drum but these days he is reframing the problem as an opportunity.

“I would launch a strategy for UK cities to be the most attractive, productive and cleanest in the world,” he says. Cities hold out the hope of being productive and desirable places to live as well as environmentally efficient ones. Consider Manhattan: it is rich, iconic and, with small apartments and a subway, it boasts a much smaller environmental footprint than most of sprawling, car-loving America.

That is the aim. But what is the policy? Lord Stern offers what he calls a “collection of policies”, including an expanded green infrastructure bank and more funding for green technology. His broadest stroke is to change the governance of British cities, devolving the power to raise taxes and borrow money but imposing strong national standards on energy efficiency.

Stern would introduce a platform for congestion charging to enable cities to develop areas connected by public transport and walking/cycling routes. He’d also raise the price of emitting carbon via a direct tax or an emissions trading system. Stern suggests £25/ton of CO2, and rising. That should add a penny to the price of 100g of airfreighted vegetables and £100-£200 to a household energy bill. It would raise £10bn, less than income tax or VAT but enough to narrow the deficit or allow other tax cuts.

But Stern doesn’t dwell on taxation. His policies are “long on UK strengths such as entrepreneurship, architecture and planning”, he says. While warning of the “deep deep dangers” of climate change, he claims his package “is attractive in its own right”.

. . .

Tim’s verdict Developing these new green city centres is a challenge. Are our urban planners up to it?

  • Political feasibility 3 out of 5
  • Economic radicalism 3 out of 5

Jonathan Haskel
Professor of economics at Imperial College, London

“Some people think that scientists have their heads in the sky, and if you gave them more government money they would simply do weirder research,” says Jonathan Haskel. Science enthusiasts, however, would say that weird research can help: Sir Andre Geim of the University of Manchester won the Nobel Prize for his discovery of the revolutionary material graphene — but not before receiving the Ig Nobel Prize for levitating a live frog.

Supporting scientific innovation has long been an easy sell for politicians. Who could be against technological progress, after all? The more difficult question is how to encourage this innovation. For Haskel, the answer is straightforward: the government should simply spend more money directly funding scientific research. At the moment the government gives about £3bn to research councils and more than £2bn to the Higher Education Funding Council. For the sake of being specific, Haskel was happy to accept my suggestion of simply doubling this funding over the course of a five-year parliament.

Haskel’s research finds that government funding of science is the perfect complement to private, practically minded research funding. “This is an example of crowding in,” he says, meaning that if the government spends more on scientific research it is likely to draw in private funding too. There is a high correlation between the research scientists who receive government grants and those collaborating with or being funded by private sector companies. Haskel has found that sectors that attract government funding are also sectors with high productivity growth.

According to Haskel’s estimates, the rate of return on basic scientific research is around 20 per cent at current funding levels — a level that would not displease Warren Buffett himself. This would probably be less if science funding dramatically increased but, even at 15 or 10 per cent return, the case for spending more would be persuasive.

An extra £5bn is not trivial. Increasing all income tax rates by one percentage point, or raising VAT to 21 per cent, would cover the cost. But given current ultra-low interest rates, Haskel says he is happy for the government to borrow to fund this spending instead. “I would regard borrowing to fund the science base as a form of infrastructure investment,” he says. It may not be the traditional Keynesian infrastructure of roads and runways but it is investment for the future nonetheless.

. . .

Tim’s verdict It’s hard to object to scientific progress and Haskel’s evidence is persuasive. Leave a bit of cash for the social scientists, please.

  • Political feasibility 4 out of 5
  • Economic radicalism 2 out of 5

Gemma Tetlow
Top pensions expert at the Institute for Fiscal Studies

A confession: I may have led Gemma Tetlow astray. We begin by discussing how employers can avoid national insurance contributions by diverting some of their workers’ salaries into a pension. This, she says, is an unwarranted subsidy for the comfortably off, and abolishing the rule is “not a bad way to raise £11bn”.

As we talk, a bolder thought forms in my mind: why not just abolish national insurance entirely and replace it with higher rates of income tax? That would close Tetlow’s pension loophole and many other inconsistencies besides. I wonder if I have missed something obvious. Apparently not. Tetlow is perfectly happy to endorse the idea of a merger.

In some ways this would be a huge change: national insurance raises more than half as much as income tax does, so merging the two would mean huge increases in headline income tax rates. But while the policy would make things simpler and more transparent, it would not greatly alter the tax that most people pay.

The idea is tempting to an economist because successive governments have discovered a feat of political arbitrage. They reduce the basic rate of income tax, which gets a lot of attention, while increasing national insurance rates, which do not. Since 1979 the basic rate of income tax has fallen from 30 to 20 per cent but national insurance rates have risen and so the marginal tax rate on much employment income is still above 45 per cent, much the same as ever. A bit more honesty about this would be welcome.

As Tetlow explains, national insurance was once a contributory system, designed to cope with a male workforce in conditions of near full employment. Now national insurance is more like an income tax, where people pay if they can afford to and receive the benefits in any case.

So Tetlow and I agree that the system could comfortably be scrapped — even if we might be looking to replace it with a basic rate of income tax at 40 per cent or so. The benefits? Transparency, administrative simplicity and the end of a few unwelcome loopholes. The risks are chiefly administrative, although a decision would need to be made about whether to have a special income tax rate for people above the state pension age, who currently pay no national insurance.

Could it happen? Tetlow says she would be “astonished” if it did. Perhaps governments are too fond of pretending that the true basic rate of income tax is just 20 per cent.

. . .

Tim’s verdict I bounced Tetlow into this so can hardly object. But for our politicians, the confusion over national insurance isn’t a problem, it’s an opportunity.

  • Political feasibility 2 out of 5
  • Economic radicalism 1 out of 5

Simon Wren-Lewis
Macroeconomist at Merton College, Oxford

Simon Wren-Lewis has a growing audience as a trenchant critic of George Osborne’s fiscal contraction. I had expected him to make the case that the incoming government should spend more but he has something more radical in mind.

“We’re passing the period when the damage was done,” he says. For Wren-Lewis, the policy error was to tighten the fiscal screws in 2010 and 2011 — he estimates that with lower taxes and higher spending the economy today would be about 4 per cent larger, while deficit reduction could wait until the economy was stronger.

Cutting spending in a severe but temporary downturn is macroeconomically perverse but makes good sense to voters, so Wren-Lewis feels that a future government would make a similar mistake in similar circumstances. What to do?

Economists have faced a related problem before. When politicians controlled interest rates they were always tempted to cut rates before elections, overheating the economy and leading to inflation once the election was safely gone. The solution was to delegate control of monetary policy to the technocrats, as when Gordon Brown gave the Bank of England this power in 1997.

“That was a good idea,” says Wren-Lewis. But, he adds, “it was always incomplete.” The missing piece of the puzzle was what the bank should do when interest rates are nearly zero — as now — and cannot be cut further to stimulate the economy. The usual solution is a fiscal expansion — cutting taxes and increasing spending, just what George Osborne has shied away from. Wren-Lewis’s response: in future, the Bank of England should print the money and hand it to the government on condition it be used for a fiscal expansion.

This is radical — but not without precedent. Economists from Adair Turner to Ben Bernanke (in 2003) and Milton Friedman (1948) have argued that deficits could be financed by printing money rather than issuing government debt. Funding real spending from paper money might seem like nonsense: if the economy is working well, creating too much money will produce inflation. But when the economy is slack, judicious money printing can turn the waste of a depressed economy into useful output, without dangerous inflation. This is a rare free lunch.

The radicalism of Wren-Lewis’s proposal lies less in the economics than the politics: the idea that the Bank of England would decide a fiscal expansion was needed, and shove a reluctant, democratically elected government into it.

Wren-Lewis calls his idea “democratic helicopter money”. He feels the government should decide whether the stimulus takes the form of tax cuts, increased benefits or new infrastructure. But the actual decision to cut taxes and raise spending to stimulate the economy? Not something one should leave to the politicians.

. . .

Tim’s verdict I sympathise with Wren-Lewis’s wish for more stimulus spending in the last parliament but outsourcing such a basic democratic responsibility feels too bold to me.

  • Political feasibility 1 out of 5
  • Economic radicalism 5 out of 5

Diane Coyle
Professor of economics at the University of Manchester

“My starting point is that the extent of income inequality has got too big,” says Coyle. She points to median annual full-time earnings of just over £27,000, while the average pay of FTSE 100 chief executives is — according to Manifest, a proxy voting service — about £4.7m. “If you were to ask me whether the productivity of chief executives is really that much higher, my answer is no. Something has gone wrong with the way the market is operating here.”

That market failure is easy to diagnose: it is hard for dispersed shareholders to monitor what is going on and to insist on a more rigorous approach. So Coyle would give them a little help.

The most eye-catching suggestion is that companies should publish the ratio between what the chief executive is paid and what the median worker in the company is paid. A review body would give a strong steer as to how high that ratio could reasonably go (“I don’t know what the right number is,” says Coyle) and companies who did not comply would face unwanted scrutiny from shareholders, employees, unions and politicians.

“Just talking about this much more would start to shift the social norm,” says Coyle, who in her term on the BBC Trust (soon to end) has seen the BBC start to publish these pay ratios, which have been falling. Coyle wants a binding rule, too, that companies should not be able to pay performance bonuses linked to share price. That is too easily manipulated. Instead, these must be linked to indicators such as customer satisfaction, sales or profits.

. . .

Tim’s verdict Shareholders and citizens alike should welcome pay that is tied more closely to good management decisions. But can any of this be legislated effectively?

  • Political feasibility 4 out of 5
  • Economic radicalism 4 out of 5

John van Reenen
Professor at the London School of Economics

“Low productivity is the number one problem Britain faces,” says Van Reenen. Even before the crisis, it lagged behind other rich countries. The latest data suggest UK output per hour worked is 30 per cent below US levels, and 17 per cent below the G7 average (at purchasing power parity).

Such a problem has no single solution but Van Reenen wants to focus on a lack of investment in the UK’s core infrastructure — housing, energy and transport. As the FT reported recently, government capital investment has fallen by a third between 2009-10 and 2013-14, despite repeated statements by the chancellor that infrastructure is at the heart of plans for growth.

Milton Keynes, the last of the “new towns”, harks back to 1967 and has 100,000 dwellings. That gives some perspective on recent proposals to build a “garden city” at Ebbsfleet of a mere 15,000 homes. If the Barker Review’s headline number of 245,000 new homes a year is to be achieved, we need an Ebbsfleet every three weeks and have done for the past 12 years.

The HS2 high-speed rail line was first examined in 1999 and is still unlikely to be finished 30 years after that date. An observer might feel the project should either have been cancelled or completed by now. And let’s not dwell on London’s airports: in 1971 the Roskill Commission proposed a major new airport north of Aylesbury after rejecting the idea of building one in the Thames estuary. We are still weighing up the issues.

“I would propose to radically change the whole way we deliver infrastructure projects,” says Van Reenen, “with a new institutional architecture for making decisions.” There are three elements to this. First, an infrastructure strategy board to recommend long-term priorities, which would be voted up or down by parliament. Second, an infrastructure planning commission to meet those priorities and arrange compensation to those affected by the march of progress. Third, an infrastructure bank to help finance projects by borrowing from capital markets and investing alongside private-sector banks.

If this seems anti-democratic, Van Reenen’s defence is that his approach “puts politics in the right place”. MPs are concerned with the short-term and the local, which causes problems with long-term investments of national significance. Like Simon Wren-Lewis, John van Reenen has more faith in technocrats than politicians.

. . .

Tim’s verdict An approach that seems justified when facing such a chronic and serious problem. Sign me up.

  • Political feasibility 3 out of 5
  • Economic radicalism 1 out of 5

Kate Barker
Author of the 2004 Barker Review of Housing Supply

I am expecting Dame Kate Barker to propose something controversial but straightforward: that we should build more houses. It is, after all, her report that policy wonks have been citing for the past decade whenever they want a number for how many houses England needs. Instead, some of her solutions “are so unpopular I can hardly bring myself to suggest them to you”. This is music to my ears.

In 2002/2003, the private sector completed 125,000 houses in England; the Barker Review argued that number needed to almost double to reduce the growth in real house prices to the EU’s long-term average. But the number of private-sector housing completions in England has fallen to below 100,000 a year from 2009 through 2014. The trickle of new houses is manifestly failing to accommodate population growth.

So: more houses? Not necessarily. Barker lays out three options. The first is the status quo. It is not attractive. There will be an increasing divide between the housing haves (who enjoy capital appreciation) and the housing have-nots (who find it ever harder to buy a home).

Option two is a dramatic programme of house building, which seems logical. “We’ve built much less than the top-line number associated with my name,” says Barker. “I haven’t changed my view that we need to do more.” But she is sceptical about how feasible it is to expect house building on the scale needed, given the strength of opposition to development. She has had the ear of prime ministers before, after all, and not much changed.

And so to option three: resign ourselves to not building enough houses to meet demand, and use the tax system to soften the blow. Meaning what, exactly?

Consider someone with the finance and good fortune to buy a home in London in 1992. That person has enjoyed an enormous increase in the real value of her house. But she has paid surprisingly little tax on the windfall. Council tax is proportionately lower on expensive homes. Capital gains tax does not apply to people living in their own homes. If you become a millionaire through skill, effort or entrepreneurial spirit, you will be taxed. If you do it by buying a house in Islington at the right moment, your bounty is yours to keep in its entirety. That’s inequitable and the inequity is likely to last from one generation to the next.

Barker suggests two thrusts to the tax reform, and “ideally we would do both”. The first is to replace council tax with a land value tax. This would tax expensive homes more heavily, in line with their value, and encourage valuable land to be used intensively. But it would also weigh heavily on elderly widows living alone in large houses. The second is to charge capital gains tax on people’s principal residency. If you live in your own home and its price starts to soar, you will be taxed.

But both these reforms are complicated. A land tax would require frequent revaluations. The capital gains tax reform would require some sort of system for postponing the bill until death or entry into a retirement home. That is fiddly but the alternative might make it impossible to move house without a punitive tax charge.

As Baker admits, this is dramatic and unpopular stuff. The people who lose out are clearly identifiable and politically influential. But the same is true of the straightforward proposal to build many more houses. The UK’s housing problem seems to be the toughest of political tough nuts.

Tim’s verdict This makes sense despite the difficulties. But Barker identified the cure for unaffordable housing more than 10 years ago — build more houses. It’s depressing that she now has to advocate palliative measures instead.

  • Political feasibility 1 out of 5
  • Economic radicalism 2 out of 5

. . .

Last word

So what would the UK look like with my board of economists in charge? We’d have more borrowing and considerably more investment — in housing, in big infrastructure, in science and in green cities. Taxes seem unlikely to fall but they would be rationalised, with a focus on energy efficiency and a transparent taxation of income and housing wealth. Inequality would be in the spotlight.

The economists seem happy to leave the politicians to their usual arguments about the EU, immigration, the price of beer and the problem of tax-dodging. Noting that every party makes similar promises about funding the National Health Service, the economists have let it be.

Perhaps that is for the best because if the economists have their way, one big thing will change after the election: politicians will be kept at a safe distance from the decisions that matter.

Written for and first published at ft.com.

Undercover Economist

Cigarettes, damn cigarettes and statistics

We cannot rely on correlation alone. But insisting on absolute proof of causation is too exacting a standard

It is said that there is a correlation between the number of storks’ nests found on Danish houses and the number of children born in those houses. Could the old story about babies being delivered by storks really be true? No. Correlation is not causation. Storks do not deliver children but larger houses have more room both for children and for storks.

This much-loved statistical anecdote seems less amusing when you consider how it was used in a US Senate committee hearing in 1965. The expert witness giving testimony was arguing that while smoking may be correlated with lung cancer, a causal relationship was unproven and implausible. Pressed on the statistical parallels between storks and cigarettes, he replied that they “seem to me the same”.

The witness’s name was Darrell Huff, a freelance journalist beloved by generations of geeks for his wonderful and hugely successful 1954 book How to Lie with Statistics. His reputation today might be rather different had the proposed sequel made it to print. How to Lie with Smoking Statistics used a variety of stork-style arguments to throw doubt on the connection between smoking and cancer, and it was supported by a grant from the Tobacco Institute. It was never published, for reasons that remain unclear. (The story of Huff’s career as a tobacco consultant was brought to the attention of statisticians in articles by Andrew Gelman in Chance in 2012 and by Alex Reinhart in Significance in 2014.)

Indisputably, smoking causes lung cancer and various other deadly conditions. But the problematic relationship between correlation and causation in general remains an active area of debate and confusion. The “spurious correlations” compiled by Harvard law student Tyler Vigen and displayed on his website (tylervigen.com) should be a warning. Did you realise that consumption of margarine is strongly correlated with the divorce rate in Maine?

We cannot rely on correlation alone, then. But insisting on absolute proof of causation is too exacting a standard (arguably, an impossible one). Between those two extremes, where does the right balance lie between trusting correlations and looking for evidence of causation?

Scientists, economists and statisticians have tended to demand causal explanations for the patterns they see. It’s not enough to know that college graduates earn more money — we want to know whether the college education boosted their earnings, or if they were smart people who would have done well anyway. Merely looking for correlations was not the stuff of rigorous science.

But with the advent of “big data” this argument has started to shift. Large data sets can throw up intriguing correlations that may be good enough for some purposes. (Who cares why price cuts are most effective on a Tuesday? If it’s Tuesday, cut the price.) Andy Haldane, chief economist of the Bank of England, recently argued that economists might want to take mere correlations more seriously. He is not the first big-data enthusiast to say so.

This brings us back to smoking and cancer. When the British epidemiologist Richard Doll first began to suspect the link in the late 1940s, his analysis was based on a mere correlation. The causal mechanism was unclear, as most of the carcinogens in tobacco had not been identified; Doll himself suspected that lung cancer was caused by fumes from tarmac roads, or possibly cars themselves.

Doll’s early work on smoking and cancer with Austin Bradford Hill, published in 1950, was duly criticised in its day as nothing more than a correlation. The great statistician Ronald Fisher repeatedly weighed into the argument in the 1950s, pointing out that it was quite possible that cancer caused smoking — after all, precancerous growths irritated the lung. People might smoke to soothe that irritation. Fisher also observed that some genetic predisposition might cause both lung cancer and a tendency to smoke. (Another statistician, Joseph Berkson, observed that people who were tough enough to resist adverts and peer pressure were also tough enough to resist lung cancer.)

Hill and Doll showed us that correlation should not be dismissed too easily. But they also showed that we shouldn’t give up on the search for causal explanations. The pair painstakingly continued their research, and evidence of a causal association soon mounted.

Hill and Doll took a pragmatic approach in the search for causation. For example, is there a dose-response relationship? Yes: heavy smokers are more likely to suffer from lung cancer. Does the timing make sense? Again, yes: smokers develop cancer long after they begin to smoke. This contradicts Fisher’s alternative hypothesis that people self-medicate with cigarettes in the early stages of lung cancer. Do multiple sources of evidence add up to a coherent picture? Yes: when doctors heard about what Hill and Doll were finding, many of them quit smoking, and it became possible to see that the quitters were at lower risk of lung cancer. We should respect correlation but it is a clue to a deeper truth, not the end of our investigations.

It’s not clear why Huff and Fisher were so fixated on the idea that the growing evidence on smoking was a mere correlation. Both of them were paid as consultants by the tobacco industry and some will believe that the consulting fees caused their scepticism. It seems just as likely that their scepticism caused the consulting fees. We may never know.

Written for and first published at ft.com.

Previous Next


  • 1 Twitter
  • 2 Flickr
  • 3 RSS
  • 4 YouTube
  • 5 Podcasts
  • 6 Facebook


  • The Undercover Economist Strikes Back
  • Adapt
  • Dear Undercover Economist
  • The Logic of Life
  • The Undercover Economist

Search by Keyword

Subscribe to TimHarford.com

Enter your email address to receive notifications of new articles by email.

Tim’s Tweets

Do NOT follow this link or you will be banned from the site!