Tim Harford The Undercover Economist
  • THE UNDERCOVER ECONOMIST STRIKES BACK
    “Every Tim Harford book is cause for celebration. He makes the ‘dismal science’ seem like an awful lot of fun.”
    – Malcolm Gladwell, author of “The Tipping Point”
  • Adapt – Why Success Always Starts with Failure


    “A highly readable argument... Very impressive”
    – Nassim N Taleb, author of “The Black Swan”
  • Dear Undercover Economist
    “Harford has a knack for explaining economic principles and problems in plain language and, even better, for making them fun”
    – The New York Times
  • The Logic of Life
    “Highly readable, funny and daringly contentious... a whopping good time.”
    – The San Francisco Chronicle
  • The Undercover Economist
    “Required reading”
    – Stephen J. Dubner, co-author of “Freakonomics”
Highlights

How to see into the future

Billions of dollars are spent on experts who claim they can forecast what’s around the corner, in business, finance and economics. Most of them get it wrong. Now a groundbreaking study has unlocked the secret: it IS possible to predict the future – and a new breed of ‘superforecasters’ knows how to do it

Irving Fisher was once the most famous economist in the world. Some would say he was the greatest economist who ever lived. “Anywhere from a decade to two generations ahead of his time,” opined the first Nobel laureate economist Ragnar Frisch, in the late 1940s, more than half a century after Fisher’s genius first lit up his subject. But while Fisher’s approach to economics is firmly embedded in the modern discipline, many of those who remember him now know just one thing about him: that two weeks before the great Wall Street crash of 1929, Fisher announced, “Stocks have reached what looks like a permanently high plateau.”

In the 1920s, Fisher had two great rivals. One was a British academic: John Maynard Keynes, a rising star and Fisher’s equal as an economic theorist and policy adviser. The other was a commercial competitor, an American like Fisher. Roger Babson was a serial entrepreneur with no serious academic credentials, inspired to sell economic forecasts by the banking crisis of 1907. As Babson and Fisher locked horns over the following quarter-century, they laid the foundations of the modern economic forecasting industry.

Fisher’s rivals fared better than he did. Babson foretold the crash and made a fortune, enough to endow the well-respected Babson College. Keynes was caught out by the crisis but recovered and became rich anyway. Fisher died in poverty, ruined by the failure of his forecasts.

If Fisher and Babson could see the modern forecasting industry, it would have astonished them in its scale, range and hyperactivity. In his acerbic book The Fortune Sellers, former consultant William Sherden reckoned in 1998 that forecasting was a $200bn industry – $300bn in today’s terms – and the bulk of the money was being made in business, economic and financial forecasting.

It is true that forecasting now seems ubiquitous. Data analysts forecast demand for new products, or the impact of a discount or special offer; scenario planners (I used to be one) produce broad-based narratives with the aim of provoking fresh thinking; nowcasters look at Twitter or Google to track epidemics, actual or metaphorical, in real time; intelligence agencies look for clues about where the next geopolitical crisis will emerge; and banks, finance ministries, consultants and international agencies release regular prophecies covering dozens, even hundreds, of macroeconomic variables.

Real breakthroughs have been achieved in certain areas, especially where rich datasets have become available – for example, weather forecasting, online retailing and supply-chain management. Yet when it comes to the headline-grabbing business of geopolitical or macroeconomic forecasting, it is not clear that we are any better at the fundamental task that the industry claims to fulfil – seeing into the future.

So why is forecasting so difficult – and is there hope for improvement? And why did Babson and Keynes prosper while Fisher suffered? What did they understand that Fisher, for all his prodigious talents, did not?

In 1987, a young Canadian-born psychologist, Philip Tetlock, planted a time bomb under the forecasting industry that would not explode for 18 years. Tetlock had been trying to figure out what, if anything, the social sciences could contribute to the fundamental problem of the day, which was preventing a nuclear apocalypse. He soon found himself frustrated: frustrated by the fact that the leading political scientists, Sovietologists, historians and policy wonks took such contradictory positions about the state of the cold war; frustrated by their refusal to change their minds in the face of contradictory evidence; and frustrated by the many ways in which even failed forecasts could be justified. “I was nearly right but fortunately it was Gorbachev rather than some neo-Stalinist who took over the reins.” “I made the right mistake: far more dangerous to underestimate the Soviet threat than overestimate it.” Or, of course, the get-out for all failed stock market forecasts, “Only my timing was wrong.”

Tetlock’s response was patient, painstaking and quietly brilliant. He began to collect forecasts from almost 300 experts, eventually accumulating 27,500. The main focus was on politics and geopolitics, with a selection of questions from other areas such as economics thrown in. Tetlock sought clearly defined questions, enabling him with the benefit of hindsight to pronounce each forecast right or wrong. Then Tetlock simply waited while the results rolled in – for 18 years.

Tetlock published his conclusions in 2005, in a subtle and scholarly book, Expert Political Judgment. He found that his experts were terrible forecasters. This was true in both the simple sense that the forecasts failed to materialise and in the deeper sense that the experts had little idea of how confident they should be in making forecasts in different contexts. It is easier to make forecasts about the territorial integrity of Canada than about the territorial integrity of Syria but, beyond the most obvious cases, the experts Tetlock consulted failed to distinguish the Canadas from the Syrias.

Adding to the appeal of this tale of expert hubris, Tetlock found that the most famous experts fared somewhat worse than those outside the media spotlight. Other than that, the humiliation was evenly distributed. Regardless of political ideology, profession and academic training, experts failed to see into the future.

Most people, hearing about Tetlock’s research, simply conclude that either the world is too complex to forecast, or that experts are too stupid to forecast it, or both. Tetlock himself refused to embrace cynicism so easily. He wanted to leave open the possibility that even for these intractable human questions of macroeconomics and geopolitics, a forecasting approach might exist that would bear fruit.

. . .

In 2013, on the auspicious date of April 1, I received an email from Tetlock inviting me to join what he described as “a major new research programme funded in part by Intelligence Advanced Research Projects Activity, an agency within the US intelligence community.”

The core of the programme, which had been running since 2011, was a collection of quantifiable forecasts much like Tetlock’s long-running study. The forecasts would be of economic and geopolitical events, “real and pressing matters of the sort that concern the intelligence community – whether Greece will default, whether there will be a military strike on Iran, etc”. These forecasts took the form of a tournament with thousands of contestants; it is now at the start of its fourth and final annual season.

“You would simply log on to a website,” Tetlock’s email continued, “give your best judgment about matters you may be following anyway, and update that judgment if and when you feel it should be. When time passes and forecasts are judged, you could compare your results with those of others.”

I elected not to participate but 20,000 others have embraced the idea. Some could reasonably be described as having some professional standing, with experience in intelligence analysis, think-tanks or academia. Others are pure amateurs. Tetlock and two other psychologists, Don Moore and Barbara Mellers, have been running experiments with the co-operation of this army of volunteers. (Mellers and Tetlock are married.) Some were given training in how to turn knowledge about the world into a probabilistic forecast; some were assembled into teams; some were given information about other forecasts while others operated in isolation. The entire exercise was given the name of the Good Judgment Project, and the aim was to find better ways to see into the future.

The early years of the forecasting tournament have, wrote Tetlock, “already yielded exciting results”.

A first insight is that even brief training works: a 20-minute course about how to put a probability on a forecast, correcting for well-known biases, provides lasting improvements to performance. This might seem extraordinary – and the benefits were surprisingly large – but even experienced geopolitical seers tend to have expertise in a subject, such as Europe’s economies or Chinese foreign policy, rather than training in the task of forecasting itself.

“For people with the right talents or the right tactics, it is possible to see into the future after all”

A second insight is that teamwork helps. When the project assembled the most successful forecasters into teams who were able to discuss and argue, they produced better predictions.

But ultimately one might expect the same basic finding as always: that forecasting events is basically impossible. Wrong. To connoisseurs of the frailties of futurology, the results of the Good Judgment Project are quite astonishing. Forecasting is possible, and some people – call them “superforecasters”– can predict geopolitical events with an accuracy far outstripping chance. The superforecasters have been able to sustain and even improve their performance.

The cynics were too hasty: for people with the right talents or the right tactics, it is possible to see into the future after all.

Roger Babson, Irving Fisher’s competitor, would always have claimed as much. A serial entrepreneur, Babson made his fortune selling economic forecasts alongside information about business conditions. In 1920, the Babson Statistical Organization had 12,000 subscribers and revenue of $1.35m – almost $16m in today’s money.

“After Babson, the forecaster was an instantly recognisable figure in American business,” writes Walter Friedman, the author of Fortune Tellers, a history of Babson, Fisher and other early economic forecasters. Babson certainly understood how to sell himself and his services. He advertised heavily and wrote prolifically. He gave a complimentary subscription to Thomas Edison, hoping for a celebrity endorsement. After contracting tuberculosis, Babson turned his management of the disease into an inspirational business story. He even employed stonecutters to carve inspirational slogans into large rocks in Massachusetts (the “Babson Boulders” are still there).

On September 5 1929, Babson made a speech at a business conference in Wellesley, Massachusetts. He predicted trouble: “Sooner or later a crash is coming which will take in the leading stocks and cause a decline of from 60 to 80 points in the Dow-Jones barometer.” This would have been a fall of around 20 per cent.

So famous had Babson become that his warning was briefly a self-fulfilling prophecy. When the news tickers of New York reported Babson’s comments at around 2pm, the markets erupted into what The New York Times described as “a storm of selling”. Shares lurched down by 3 per cent. This became known as the “Babson break”.

The next day, shares bounced back and Babson, for a few weeks, appeared ridiculous. On October 29, the great crash began, and within a fortnight the market had fallen almost 50 per cent. By then, Babson had an advertisement in the New York Times pointing out, reasonably, that “Babson clients were prepared”. Subway cars were decorated with the slogan, “Be Right with Babson”. For Babson, his forecasting triumph was a great opportunity to sell more subscriptions.

But his true skill was marketing, not forecasting. His key product, the “Babson chart”, looked scientific and was inspired by the discoveries of Isaac Newton, his idol. The Babson chart operated on the Newtonian assumption that any economic expansion would be matched by an equal and opposite contraction. But for all its apparent sophistication, the Babson chart offered a simple and usually contrarian message.

“Babson offered an up-arrow or a down-arrow. People loved that,” says Walter Friedman. Whether or not Babson’s forecasts were accurate was not a matter that seemed to concern many people. When he was right, he advertised the fact heavily. When he was wrong, few noticed. And Babson had indeed been wrong for many years during the long boom of the 1920s. People taking his advice would have missed out on lucrative opportunities to invest. That simply didn’t matter: his services were popular, and his most spectacularly successful prophecy was also his most famous.

Babson’s triumph suggests an important lesson: commercial success as a forecaster has little to do with whether you are any good at seeing into the future. No doubt it helped his case when his forecasts were correct but nobody gathered systematic information about how accurate he was. The Babson Statistical Organization compiled business and economic indicators that were, in all probability, of substantial value in their own right. Babson’s prognostications were the peacock’s plumage; their effect was simply to attract attention to the services his company provided.

. . .

When Barbara Mellers, Don Moore and Philip Tetlock established the Good Judgment Project, the basic principle was to collect specific predictions about the future and then check to see if they came true. That is not the world Roger Babson inhabited and neither does it describe the task of modern pundits.

When we talk about the future, we often aren’t talking about the future at all but about the problems of today. A newspaper columnist who offers a view on the future of North Korea, or the European Union, is trying to catch the eye, support an argument, or convey in a couple of sentences a worldview that would otherwise be impossibly unwieldy to explain. A talking head in a TV studio offers predictions by way of making conversation. A government analyst or corporate planner may be trying to justify earlier decisions, engaging in bureaucratic defensiveness. And many election forecasts are simple acts of cheerleading for one side or the other.

“Some people – call them ‘superforecasters’– can predict geopolitical events with an accuracy far outstripping chance”

Unlike the predictions collected by the Good Judgment Project, many forecasts are vague enough in their details to allow the mistaken seer off the hook. Even if it was possible to pronounce that a forecast had come true or not, only in a few hotly disputed cases would anybody bother to check.

All this suggests that among the various strategies employed by the superforecasters of the Good Judgment Project, the most basic explanation of their success is that they have the single uncompromised objective of seeing into the future – and this is rare. They receive continual feedback about the success and failure of every forecast, and there are no points for radicalism, originality, boldness, conventional pieties, contrarianism or wit. The project manager of the Good Judgment Project, Terry Murray, says simply, “The only thing that matters is the right answer.”

I asked Murray for her tips on how to be a good forecaster. Her reply was, “Keep score.”

. . .

An intriguing footnote to Philip Tetlock’s original humbling of the experts was that the forecasters who did best were what Tetlock calls “foxes” rather than “hedgehogs”. He used the term to refer to a particular style of thinking: broad rather than deep, intuitive rather than logical, self-critical rather than assured, and ad hoc rather than systematic. The “foxy” thinking style is now much in vogue. Nate Silver, the data journalist most famous for his successful forecasts of US elections, adopted the fox as the mascot of his website as a symbol of “a pluralistic approach”.

The trouble is that Tetlock’s original foxes weren’t actually very good at forecasting. They were merely less awful than the hedgehogs, who deployed a methodical, logical train of thought that proved useless for predicting world affairs. That world, apparently, is too complex for any single logical framework to encompass.

More recent research by the Good Judgment Project investigators leaves foxes and hedgehogs behind but develops this idea that personality matters. Barbara Mellers told me that the thinking style most associated with making better forecasts was something psychologists call “actively open-minded thinking”. A questionnaire to diagnose this trait invites people to rate their agreement or disagreement with statements such as, “Changing your mind is a sign of weakness.” The project found that successful forecasters aren’t afraid to change their minds, are happy to seek out conflicting views and are comfortable with the notion that fresh evidence might force them to abandon an old view of the world and embrace something new.

Which brings us to the strange, sad story of Irving Fisher and John Maynard Keynes. The two men had much in common: both giants in the field of economics; both best-selling authors; both, alas, enthusiastic and prominent eugenicists. Both had immense charisma as public speakers.

Fisher and Keynes also shared a fascination with financial markets, and a conviction that their expertise in macroeconomics and in economic statistics should lead to success as an investor. Both of them, ultimately, were wrong about this. The stock market crashes of 1929 – in September in the UK and late October in the US – caught each of them by surprise, and both lost heavily.

Yet Keynes is remembered today as a successful investor. This is not unreasonable. A study by David Chambers and Elroy Dimson, two financial economists, concluded that Keynes’s track record over a quarter century running the discretionary portfolio of King’s College Cambridge was excellent, outperforming market benchmarks by an average of six percentage points a year, an impressive margin.

This wasn’t because Keynes was a great economic forecaster. His original approach had been predicated on timing the business cycle, moving into and out of different investment classes depending on which way the economy itself was moving. This investment strategy was not a success, and after several years Keynes’s portfolio was almost 20 per cent behind the market as a whole.

The secret to Keynes’s eventual profits is that he changed his approach. He abandoned macroeconomic forecasting entirely. Instead, he sought out well-managed companies with strong dividend yields, and held on to them for the long term. This approach is now associated with Warren Buffett, who quotes Keynes’s investment maxims with approval. But the key insight is that the strategy does not require macroeconomic predictions. Keynes, the most influential macroeconomist in history, realised not only that such forecasts were beyond his skill but that they were unnecessary.

Irving Fisher’s mistake was not that his forecasts were any worse than Keynes’s but that he depended on them to be right, and they weren’t. Fisher’s investments were leveraged by the use of borrowed money. This magnified his gains during the boom, his confidence, and then his losses in the crash.

But there is more to Fisher’s undoing than leverage. His pre-crash gains were large enough that he could easily have cut his losses and lived comfortably. Instead, he was convinced the market would turn again. He made several comments about how the crash was “largely psychological”, or “panic”, and how recovery was imminent. It was not.

One of Fisher’s major investments was in Remington Rand – he was on the stationery company’s board after selling them his “Index Visible” invention, a type of Rolodex. The share price tells the story: $58 before the crash, $28 by 1930. Fisher topped up his investments – and the price soon dropped to $1.

Fisher became deeper and deeper in debt to the taxman and to his brokers. Towards the end of his life, he was a marginalised figure living alone in modest circumstances, an easy target for scam artists. Sylvia Nasar writes in Grand Pursuit, a history of economic thought, “His optimism, overconfidence and stubbornness betrayed him.”

. . .

So what is the secret of looking into the future? Initial results from the Good Judgment Project suggest the following approaches. First, some basic training in probabilistic reasoning helps to produce better forecasts. Second, teams of good forecasters produce better results than good forecasters working alone. Third, actively open-minded people prosper as forecasters.

But the Good Judgment Project also hints at why so many experts are such terrible forecasters. It’s not so much that they lack training, teamwork and open-mindedness – although some of these qualities are in shorter supply than others. It’s that most forecasters aren’t actually seriously and single-mindedly trying to see into the future. If they were, they’d keep score and try to improve their predictions based on past errors. They don’t.

“Successful forecasters aren’t afraid to change their minds and are comfortable with the notion that fresh evidence might mean abandoning an old view”

This is because our predictions are about the future only in the most superficial way. They are really advertisements, conversation pieces, declarations of tribal loyalty – or, as with Irving Fisher, statements of profound conviction about the logical structure of the world. As Roger Babson explained, not without sympathy, Fisher had failed because “he thinks the world is ruled by figures instead of feelings, or by theories instead of styles”.

Poor Fisher was trapped by his own logic, his unrelenting optimism and his repeated public declarations that stocks would recover. And he was bankrupted by an investment strategy in which he could not afford to be wrong.

Babson was perhaps wrong as often as he was right – nobody was keeping track closely enough to be sure either way – but that did not stop him making a fortune. And Keynes prospered when he moved to an investment strategy in which forecasts simply did not matter much.

Fisher once declared that “the sagacious businessman is constantly forecasting”. But Keynes famously wrote of long-term forecasts, “About these matters there is no scientific basis on which to form any calculable probability whatever. We simply do not know.”

Perhaps even more famous is a remark often attributed to Keynes. “When my information changes, I alter my conclusions. What do you do, sir?”

If only he had taught that lesson to Irving Fisher.

Also published at ft.com.

Undercover Economist

Ice bucket challenge: the cold facts

In a world of limited generosity, who is to say which cause should be at the head of the queue?

Last week I finally succumbed to social pressure and invited some colleagues at the BBC to film me having a bucket of iced water tipped over my head. As surely nobody needs telling by now, the deal is that people film themselves being drenched, donate money to the US-based ALS Association or its British equivalent, the Motor Neurone Disease Association, and then nominate three further people for the same treatment.

The challenge is an infectious plague, humiliation TV and pyramid scheme all rolled into one, and it’s fundraising genius. Lady Gaga’s done it; Mark Zuckerberg has done it; George W Bush has done it. By the time this column is in print, I imagine everyone on the planet will have done it.

Social pressure is a powerful thing, and it’s refreshing to see it being used to spread smiles and encourage a generous spirit. This is not new, of course. Charities have long sought celebrity endorsements, and seeing famous people have liquids poured on them is a venerable tradition. As for seeking sponsorship to run a marathon or climb Kilimanjaro, we all know that shamelessly pressuring friends and colleagues to give money is the very essence of the exercise.

Peer pressure can also produce reluctant givers. Adriaan Soetevent, an economist at the University of Groningen, studied church collections in an open basket versus a closed collection bag. The open basket elicited larger donations. And in another clever field experiment run by three economists, Stefano DellaVigna, John List and Ulrike Malmendier, fundraisers went door to door raising money. Some households, chosen randomly, had received a flyer warning them exactly when the fundraisers would be around: this warning dramatically increased the chance that the door would not be opened. Not all of us welcome the opportunity to give money to randomly selected charities, it seems.

This time, the social element seems to be a source of no small joy: at a family gathering recently, people were gleefully ice-bucketing each other until the garden had become a swamp. Surely the ice bucket challenge is a good thing, raising money for a worthy cause while giving us a good chuckle into the bargain.

But any good economist has to ask – and I do apologise about this – “a good thing compared to what?” Some critics have suggested that charitable donations are a zero-sum game: more money for the ALS and MND associations means less money for other charities. The evidence for that proposition is thin, as it happens, but even if the many tens of millions raised by the ice bucket challenge are brand-new charitable giving, we could still ask where that money would best be spent.

The strength of a viral giving campaign is also its weakness: people join in for a laugh because their friends have put them up to it, rather than because of a logical analysis of the most worthy cause. Motor neurone disease traps people in their own bodies as they lose the ability to move, speak, eat and, eventually, even to breathe. It is a truly dreadful condition – but so is bowel cancer, fatal diarrhoea or simply starving to death. In a world of limited generosity and finite resources, who is to say which cause should be at the head of the queue?

The fact that ice-bucketeers are donating to the ALS Association feels entirely arbitrary. If the Red Cross or the American Cancer Society had happened to be the beneficiaries instead, very little else about the viral campaign would have changed. Would that have been a better situation?

GiveWell is an organisation which seems well placed to answer such questions: it aims to give donors the information they need to make the most effective donations. It sounds like an impossible job. GiveWell’s approach is to find cost-effective, evidence-based approaches such as distributing antimalarial bednets, and then search for transparent, efficient charities pursuing that approach. One of their top recommendations, for example, is the Schistosomiasis Control Initiative – a charity that could use a catchier name. It organises treatment for parasitic worms, a very unsexy cause indeed. But the worms can do a lot of harm and are absurdly inexpensive to treat – hence the finding that the SCI offers value for your donated money.

In the end, I sent a few pounds of my ice bucket donation to the Motor Neurone Disease Association. It would have felt wrong, somehow, to do otherwise. I sent a more substantial donation to SCI, surely one of the least media-friendly charities on the planet. All lives are equally valuable but some lives may be saved far more cheaply than others. It seems strange not to respond to a philanthropic bargain.

No doubt some will find this line of reasoning colder than a bucket full of iced water. But the truth is that whenever we give money to one cause rather than another, we’re making a decision about how deserving that cause is. When a social media campaign gathers momentum, it is human nature to make that decision spontaneously and without a moment’s reflection. It feels good. But feeling good and doing good are not the same thing.

You can donate to the SCI at www3.imperial.ac.uk/schisto

Also published at ft.com.

Undercover Economist

Here today, gone tomorrow

Don’t draw up your task list in the morning – do it the evening before, when you will have a more distant perspective

What does going on a diet have in common with time management? Here’s a musical clue: Little Orphan Annie sings: “Tomorrow, tomorrow, I love you, tomorrow – you’re always a day away.” Sheila Hancock’s song “My Last Cigarette” has a more cynical bent: “I’ll give up the habit, I will even yet, when I’ve had just one more cigarette.”

The songs could hardly be more different but the common thread is the way that the promise of tomorrow is transformed overnight into something altogether different: today. Strange things happen to us when tomorrow turns into today. Tomorrow we’ll eat fruit rather than candy bars. Tomorrow we’ll watch Krzysztof Kieślowski’s Blue rather than Sleepless in Seattle. And yet curiously when tomorrow arrives, we eat chocolate and watch romcoms. Our preferences flip.

This isn’t just my whimsical summary of human nature: the psychologist Daniel Read of Warwick Business School and his colleagues have conducted experiments finding pretty much exactly this behaviour. Experimental participants, given the opportunity to select food or movies in advance, are more likely to choose the highbrow film or the healthy snack. When the moment of truth arrives, they often change their minds if given the option.

Economists give this tendency the charmless name of hyperbolic discounting. Hyperbolic discounting poses some obvious and well-understood problems for those of us going on a diet or saving for a pension. The problem of personal productivity, however, is far thornier.

On any typical day – indeed, from moment to moment – we have to decide how to spend our time. We have a choice of long-term and short-term projects, big and small tasks/jobs, fixed commitments and free time, all within a daily rhythm of productive moments and postprandial slumps. To add to the challenge, unexpected tasks are always arriving in the inbox.

Armed with traditional tools of to-do list and calendar, this already looks like a tough enough optimisation problem. Add hyperbolic discounting and it looks vicious.

“Managing time is almost inhumane in its requirements,” says Dan Ariely, a behavioural scientist at Duke University. He’s right. While trying to figure out the wisest way to spend our time, we are constantly tempted to surf around on YouTube. Or perhaps we engage in busy-work, reorganising the filing cabinet and kidding ourselves that just because it’s work, it’s worth doing. Tomorrow’s priorities – applying for a promotion, starting the next big project, learning a new language – keep evaporating whenever tomorrow turns into today.

What are the solutions?

One possibility is to schedule tasks ahead of time in the calendar. The big presentation, the Japanese revision, the washing-up, all of it gets a diary slot. There’s promise in this approach. It still requires willpower but putting long-term priorities firmly in the calendar helps deal with the hyperbolic discounting problem. But an overstuffed diary is inflexible and one missed target means an entire calendar must be reworked. The system is unlikely to work for all but the most predictable lists of tasks.

Perhaps technology can save us. Ariely is part of a team producing a new smartphone app, Timeful, which aims to deliver the diary-stuffing approach more intelligently. The idealised form of the software would know everything you wanted to get done – from writing a novel to having a drink with old friends to doing the laundry. It would know how long each task would take, by when it had to be done, how important it was and when might be a productive time to do it. The software would also have access to your calendar, and it would tentatively schedule your tasks wherever it found free space. Over time it would learn about your productivity.

This seems enormously useful, although much depends on how close the algorithm comes to this idealised vision – and how much fuss it is to interact with it. (Users of iPhones can give it a try right now.)

. . .

For those who prefer a pen-and-paper approach to productivity, what to do about the hyperbolic discounting problem? I have two suggestions. The first helps bring a long-term perspective to the daily to-do list. Don’t draw up your list of tasks first thing in the morning – do it the previous evening, when you will have a slightly more distant perspective. When you do so, think about the two or three tasks you would feel most satisfied to have ticked off. Put those at the top of the list and make them your priority.

The second suggestion flips the telescope around and brings today’s perspective to tomorrow’s commitments. When being invited to do things months in advance, the diary usually looks pretty clear and it’s tempting to say “yes”. But whenever a new invitation arrives, ask yourself not, “should I accept the invitation in March?” but, “would I accept the invitation if it was for this week?”

The fundamental insight of hyperbolic discounting is that while tomorrow always looks different, eventually tomorrow will be today. If the flattering invitation would be impossible to accept for this week, what on earth makes you think the first week of March will look any different once it arrives? Tomorrow is always a day away – but your rash commitments are not.

Also published at ft.com.

Video

The secrets of superforecasting

Newsnight videos aren’t embeddable, it seems, but you may enjoy this four minutes of fun, in which I discuss the latest research on how to see into the future.

I discuss the research in much more detail in my FT article, “How to See Into the Future” – no paywall, so enjoy.

8th of September, 2014VideoComments off
Other Writing

Why inflation remains best way to avoid stagnation

The prospect is that central banks will find themselves helpless, writes Tim Harford

People who were not born when the financial crisis began are now old enough to read about it. We have been able to distract ourselves with two Olympics, two World Cups and two US presidential elections. Yet no matter how stale our economic troubles feel, they manage to linger.

Given the severity of the crisis and the inadequacy of the policy response, it should be no surprise that recovery has been slow and anaemic: that is what economic history always suggested. Yet some economists are growing disheartened. The talk is of “secular stagnation” – a phrase which could mean two things, neither of them good.

One fear has been well-aired: that future growth possibilities will be limited by an ageing population or perhaps even technological stagnation.

The second meaning of secular stagnation is altogether stranger: it is that regardless of their potential for growth, modern economies may suffer from a persistent tendency to slip below that potential, sliding into stubborn recessions. The west’s lost decade of economic growth may be a taste of things to come.

This view was put forward most forcefully by Lawrence Summers, who was Treasury secretary under Bill Clinton and a senior adviser to President Barack Obama. It has been discussed at length in a collection of essays published last week by the Centre for Economic Policy Research. But what could it mean?

Normally, when an economy slips into recession, the standard response is to cut interest rates. This encourages us to spend, rather than save, giving the economy an immediate boost.

Things become more difficult if nominal interest rates are already low. Central banks have to employ radical tactics of uncertain effectiveness, such as quantitative easing. Governments could and should borrow and spend to support the economy. In practice they have proved politically gridlocked (in the US), institutionally hamstrung (in the EU) or ideologically blinkered (in the UK). There is not much reason to think the politics of fiscal stimulus would be very different in the future, so the zero-interest rate boundary is a problem.

The awful prospect of secular stagnation is that this is the new normal. Interest rates will be very low as a matter of course, and central banks will routinely find themselves nearly helpless.
“A cut in interest rates encourages us to spend, rather than save, giving the economy an immediate boost”

Before we startle ourselves at shadows, let us ask why Prof Summers might be right. Real interest rates – the rates paid after adjusting for inflation – have been falling. In the US, real rates averaged about 5 per cent in the 1980s, 2 per cent in the 1990s and 1 per cent in the Noughties. (Since Lehman Brothers failed they have been negative, but the long-term trend speaks more eloquently.) Real interest rates have also been declining in the EU for 20 years. The International Monetary Fund’s estimate of global real interest rates has been declining for 30 years.

This does not look good, so why is it happening? The background level of real interest rates is set not by central banks but by supply and demand. Low real rates suggest lots of people are trying to save, and particularly in safe assets, while few people are trying to borrow and invest. Only with rates at a very low level can enough borrowers be found to mop up all the savings.

If secular stagnation is a real risk, we need policies to address it. One approach is to try to change the forces of supply and demand to boost the demand for cash to invest, while stemming the supply of savings, and reducing the bias towards super-safe assets.

This looks tricky. Much policy has pushed in the opposite direction. Consider the austerity drive and long-term goals to reduce government debt burdens; this reduces the supply of safe assets and pushes down real rates. Or the tendency in the UK to push pension risk away from companies and the government, and towards individuals; this encourages extra saving, just in case. Or the way in which (understandably) regulators insist that banks and pension funds hold more safe assets; again, this increases the demand for safe assets and pushes down real interest rates. To reverse all these policies, sacrificing microeconomic particulars for a rather abstract macroeconomic hunch, looks like a hard sell.

There is a simple alternative, albeit one that carries risks. Central bank targets for inflation should be raised to 4 per cent. A credible higher inflation target would provide immediate stimulus (who wants to squirrel away money that is eroding at 4 per cent a year?) and would give central banks more leeway to cut real rates in future. If equilibrium real interest rates are zero, that might not matter when central banks can produce real rates of minus 4 per cent.

If all that makes you feel queasy, it should. As Prof Summers argues, unpleasant things have a tendency to happen when real interest rates are very low. Bubbles inflate, Ponzi schemes prosper and investors are reckless in their scrabble for yield.

One thing that need not worry anyone, though, is the prospect of an inflation target of 4 per cent. It will not happen. That is particularly true in the place where the world economy most needs more inflation: in the eurozone. The German folk memory of hyperinflation in 1923 is just too strong. That economic catastrophe, which helped lay the foundations for Nazism and ruin much of the 20th century, continues to resonate today.

What practical policy options remain? That is easy to see. We must cross our fingers and hope that Prof Summers is mistaken.

Also published at ft.com.

5th of September, 2014Other WritingComments off
Video

In which I take the ice-bucket challenge the nerdy way…

On “More or Less” this week we discuss the ice bucket challenge: are viral memes a good way to get people giving to charity, or does it make us careless about what the charities are doing with the money? Is it possible to use data to identify the very best charity in the world? Listen on bbc.co.uk/moreorless.

You can donate to the Schistosomiasis Control Initiative here: http://www3.imperial.ac.uk/schisto

 

 

I’d just like to clarify that I an not sitting on the loo – it’s a shower with a seat. Oh, why am I even bothering to try to maintain my dignity?

5th of September, 2014VideoComments off
Other Writing

Monopoly is a bureaucrat’s friend but a democrat’s foe

The challenges from smaller competitors spur the innovations that matter

“It takes a heap of Harberger triangles to fill an Okun gap,” wrote James Tobin in 1977, four years before winning the Nobel Prize in economics. He meant that the big issue in economics was not battling against monopolists but preventing recessions and promoting recovery.

After the misery of recent years, nobody can doubt that preventing recessions and promoting recovery would have been a very good idea. But economists should be able to think about more than one thing at once. What if monopoly matters, too?

The Harberger triangle is the loss to society as monopolists raise their prices, and it is named after Arnold Harberger, who 60 years ago discovered that the costs of monopoly were about 0.1 per cent of US gross domestic product – a few billion dollars these days, much less than expected and much less than a recession.

Professor Harberger’s discovery helped build a consensus that competition authorities could relax about the power of big business. But have we relaxed too much?

Large companies are all around us. We buy our mid-morning coffee from global brands such as Starbucks, use petrol from Exxon or Shell, listen to music purchased from a conglomerate such as Sony (via Apple’s iTunes), boot up a computer that runs Microsoft on an Intel processor. Crucial utilities – water, power, heating, internet and telephone – are supplied by a few dominant groups, with baffling contracts damping any competition.

Of course, not all large businesses have monopoly power. Tesco, the monarch of British food retailing, has found discount competitors chopping up its throne to use as kindling. Apple and Google are supplanting Microsoft. And even where market power is real, Prof Harberger’s point was that it may matter less than we think. But his influential analysis focused on monopoly pricing. We now know there are many other ways in which dominant businesses can harm us.

In 1989 the Beer Orders shook up a British pub industry controlled by six brewers. The hope was that more competition would lead to more and cheaper beer. It did not. The price of beer rose. Yet so did the quality of pubs. Where once every pub had offered rubbery sandwiches and stinking urinals, suddenly there were sports bars, candlelit gastropubs and other options. There is more to competition than lower prices.

Monopolists can sometimes use their scale and cash flow to produce real innovations – the glory years of Bell Labs come to mind. But the ferocious cut and thrust of smaller competitors seems a more reliable way to produce many of the everyday innovations that matter.

That cut and thrust is no longer so cutting or thrusting as once it was. “The business sector of the US economy is ageing,” says a Brookings research paper. It is a trend found across regions and industries, as incumbent players enjoy entrenched advantages. “The rate of business start-ups and the pace of employment dynamism in the US economy has fallen over recent decades . . . This downward trend accelerated after 2000,” adds a survey in the Journal of Economic Perspectives.

That means higher prices and less innovation, but perhaps the game is broader still. The continuing debate in the US over “net neutrality” is really an argument about the least damaging way to regulate the conduct of cable companies that hold local monopolies. If customers had real choice over their internet service provider, net neutrality rules would be needed only as a backstop.

As the debate reminds us, large companies enjoy power as lobbyists. When they are monopolists, the incentive to lobby increases because the gains from convenient new rules and laws accrue solely to them. Monopolies are no friend of a healthy democracy.

They are, alas, often the friend of government bureaucracies. This is not just a case of corruption but also about what is convenient and comprehensible to a politician or civil servant. If they want something done about climate change, they have a chat with the oil companies. Obesity is a problem to be discussed with the likes of McDonald’s. If anything on the internet makes a politician feel sad, from alleged copyright infringement to “the right to be forgotten”, there is now a one-stop shop to sort it all out: Google.

Politicians feel this is a sensible, almost convivial, way to do business – but neither the problems in question nor the goal of vigorous competition are resolved as a result.

One has only to consider the way the financial crisis has played out. The emergency response involved propping up big institutions and ramming through mergers; hardly a long-term solution to the problem of “too big to fail”. Even if smaller banks do not guarantee a more stable financial system, entrepreneurs and consumers would profit from more pluralistic competition for their business.

No policy can guarantee innovation, financial stability, sharper focus on social problems, healthier democracies, higher quality and lower prices. But assertive competition policy would improve our odds, whether through helping consumers to make empowered choices, splitting up large corporations or blocking megamergers. Such structural approaches are more effective than looking over the shoulders of giant corporations and nagging them; they should be a trusted tool of government rather than a last resort.

As human freedoms go, the freedom to take your custom elsewhere is not a grand or noble one – but neither is it one that we should abandon without a fight.

Also published at ft.com.

16th of August, 2014Other WritingComments off
Other Writing

Pity the robot drivers snarled in a human moral maze

Robotic cars do not get tired, drunk or angry but there are bound to be hiccups, says Tim Harford

Last Wednesday Vince Cable, the UK business secretary, invited British cities to express their interest in being used as testing grounds for driverless cars. The hope is that the UK will gain an edge in this promising new industry. (German autonomous cars were being tested on German, French and Danish public roads 20 years ago, so the time is surely ripe for the UK to leap into a position of technological leadership.)

On Tuesday, a very different motoring story was in the news. Mark Slater, a lorry driver, was convicted of murdering Trevor Allen. He had lost his temper and deliberately driven a 17 tonne lorry over Mr Allen’s head. It is a striking juxtaposition.

The idea of cars that drive themselves is unsettling, but with drivers like Slater at large, the age of the driverless car cannot come quickly enough.

But the question of how safe robotic cars are, or might become, is rather different from the question of the risks of a computer-guided car are perceived, and how they might be repackaged by regulators, insurers and the courts.

On the first question, it is highly likely that a computer will one day do a better, safer, more courteous job of driving than you can. It is too early to be certain of that, because serious accidents are rare. An early benchmark for Google’s famous driverless car programme was to complete 100,000 miles driving on public roads – but American drivers in general only kill someone every 100m miles.

Still, the safety record so far seems good, and computers have some obvious advantages. They do not get tired, drunk or angry. They are absurdly patient in the face of wobbly cyclists, learner drivers and road hogs.

But there are bound to be hiccups. While researching this article my Google browser froze up while trying to read a Google blog post hosted on a Google blogging platform. Two seconds later the problem had been solved, but at 60 miles per hour two seconds is more than 50 metres. One hopes that Google-driven cars will be more reliable when it comes to the more literal type of crash.

Yet the exponential progress of cheaper, faster computers with deeper databases of experience will probably guarantee success eventually. In a simpler world, that would be the end of it.

Reality is knottier. When a car knocks over a pedestrian, who is to blame? Our answer depends not only on particular circumstances but on social norms. In the US in the 1920s, the booming car industry found itself under pressure as pedestrian deaths mounted. One response was to popularise the word “jaywalking” as a term of ridicule for bumpkins who had no idea how to cross a street. Social norms changed, laws followed, and soon enough the default assumption was that pedestrians had no business being in the road. If they were killed they had only themselves to blame.

We should prepare ourselves for a similar battle over robot drivers. Assume that driverless cars are provably safer. When a human driver collides with a robo-car, where will our knee-jerk sympathies lie? Will we blame the robot for not spotting the human idiosyncrasies? Or the person for being so arrogant as to think he could drive without an autopilot?

When such questions arrive in the courts, as they surely will, robotic cars have a serious handicap. When they err, the error can be tracked back to a deep-pocketed manufacturer. It is quite conceivable that Google, Mercedes or Volvo might produce a robo-car that could avoid 90 per cent of the accidents that would befall a human driver, and yet be bankrupted by the legal cases arising from the 10 per cent that remained. The sensible benchmark for robo-drivers would be “better than human”, but the courts may punish them for being less than perfect.

There are deep waters here. How much space is enough when overtaking a slow vehicle – and is it legitimate for the answer to change when running late? When a child chases a ball out into the road, is it better to swerve into the path of an oncoming car, or on to the pavement where the child’s parents are standing, or not to swerve at all?

These are hardly thought of as ethical questions because human drivers make them intuitively and in an instant. But a computer’s priorities must be guided by its programmers, who have plenty of time to weigh up the tough ethical choices.

In 1967 Philippa Foot, one of Oxford’s great moral philosophers, posed a thought experiment that she called the “trolley problem”. A runaway railway trolley is about to kill five people, but by flipping the points, you can redirect it down a line where it will instead kill one. Which is the right course of action? It is a rich seam for ethical discourse, with many interesting variants. But surely Foot did not imagine that the trolley problem would have to be answered one way or another and wired into the priorities of computer chauffeurs – or that lawyers would second-guess those priorities in court in the wake of an accident.

Then there is the question of who opts for a driverless car. Sir David Spiegelhalter, a risk expert at Cambridge university, points out that most drivers are extremely safe. Most accidents are caused by a few idiots, and it is precisely those idiots, Sir David speculates, who are least likely to cede control to a computer.

Perhaps driverless cars will be held back by a tangle of social, legal and regulatory stubbornness. Or perhaps human drivers will one day be banned, or prohibitively expensive to insure. It is anyone’s guess, because while driving is no longer the sole preserve of meatsacks such as you and me, the question of what we fear and why we fear it remains profoundly, quirkily human.

Also published at ft.com.

7th of August, 2014Other WritingComments off
Undercover Economist

When crime stops paying

To an economist, tougher sentencing in the wake of the 2011 riots offers a fascinating natural experiment

The third anniversary of the 2011 London riots is this week. They erupted so suddenly and spread so quickly across the capital and to other English cities that at the time the disintegration of British society seemed, if unlikely, at least conceivable. In the rear-view mirror, though, the riots are eclipsed by the London Olympics and much diminished by the passage of time.

For parochial reasons, the riots remain vivid to me. My son was born in Hackney just a few days before they started. As violence flared a couple of streets away to the south and to the north of us, my wife and son slept while I stood on the doorstep of our home and watched as a pair of helicopters droned directly overhead.

A year after the riots I wrote a column pointing out that they were essentially random events. They had a cause, of course. The spark was the shooting of Mark Duggan by the Metropolitan Police, and one source of fuel was the perception that police stop-and-search powers were being used crassly and with a racial bias. Yet similar grievances have emerged at other times and in other places without provoking mass civil unrest. Chance plays a major element in such stories.

The criminal justice system responded sharply to the riots. More than 1,000 suspected rioters were charged by the Metropolitan Police during the first week of trouble, and over the same time period more than 800 of them made a first appearance in court. By September 2012, 4,600 people had been arrested, out of about 13,000-15,000 people who are believed to have participated in the trouble in some way. Given the initial sense of impunity, that is a high rate of unwelcome police attention.

More striking was the way in which judges handed out sentences as though they were on steroids. Two people were sentenced to four years in prison each for Facebook postings inviting others to run amok in Cheshire, an unlikely location for a revolutionary uprising. Nobody showed up to “smash dwn in Northwich town” or “riot in Latchford”, so the sentences raised eyebrows. So did the 10-month sentence handed out to a teenager who carried two left-footed trainers out of a shop in Wolverhampton. She thought better of it and immediately dropped them – surely one of the most short-lived thefts in history. Sentences were, in general, more severe than normal. The thinking behind all this was that the true crime that needed to be punished was not theft or incitement but participation in a moment of grave civil peril.

Were these sentences an essential crisis response or a draconian overreaction? To an economist, they are something else: a fascinating natural experiment. With the news full of crushing punishments, it must have seemed plausible that the risks of committing a crime had soared. So did the threat of harsh punishments deter crime?

The usual statistical problem is that sentencing policy might influence crime rates but crime rates might equally influence sentencing policy. Cause and effect are hard to disentangle. In the case of the riots, however, the surge in crime that provoked the crackdown was sudden, unexpected, highly localised and brief. The sentencing response was drawn-out and stories of harsh sentences appeared in the national and London press for months.

. . .

As a result, a mugger or burglar in an area of London entirely unaffected by the riots might still feel conscious that the mood of the judiciary had changed. Three economists, Brian Bell, Laura Jaitman and Stephen Machin, used this sudden change in the judicial wind to measure the impact of tough sentences on crime. Across London, they found a significant drop in “riot crimes” – burglary, criminal damage and violence against the person – over the six months following the riots. Meanwhile, other crimes showed a tendency to increase, as though criminals were substituting away from the “expensive” crimes and towards the “cheaper” ones.

This shouldn’t be too much of a surprise. (I wrote an entire book, The Logic of Life, arguing that the most unlikely people in the most unlikely circumstances turn out to be greatly influenced by simple incentives.) But it’s a useful result because rigorous evidence on such matters is hard to find.

One of my favourite exceptions is an article by two economists, Jonathan Klick and Alex Tabarrok, who examined the impact of periodic terrorism alerts in Washington DC in the couple of years following the attacks of September 11 2001. Whenever alert levels were raised, police officers flooded sensitive locations, most of which (such as the White House and the Capitol) are on or near the National Mall.

Over the 16 months studied, the Mall and surrounding district experienced about 8,500 crimes, often theft from or of cars, not really al-Qaeda territory. Klick and Tabarrok argued that the occasional surges in police numbers were not caused by car thefts but did successfully deter them.

There may well be cheaper, more effective and more humane ways to reduce the crime rate.

But such studies have helped to build confidence that the world isn’t an entirely irrational place. Raise the costs of crime and criminals will respond.

Also published at ft.com.

Undercover Economist

Why tax systems are trickier than Martian algebra

Only radical restructuring has a chance of creating fair taxation, writes Tim Harford

Tax is a divisive subject but everyone seems to agree on one point: taxes are too complicated and should be simpler. Unfortunately, tax systems did not receive the memo.

In the UK only a few years ago, almost everyone in work used to be taxed at a marginal rate of either 31 per cent or 41 per cent, depending on how much they earned. (If Brits do not recognise those numbers, it is because the UK has two cumulative systems of income tax, one of which goes by the code name of “national insurance”.)

The system is trickier today than Martian algebra. Paul Johnson of the Institute for Fiscal studies points out that, over different levels of income, a non-working spouse with two children will be taxed at marginal rates of 12 per cent, 32 per cent, infinity, 42 per cent, 60 per cent, 42 per cent, 60 per cent, 42 per cent and 47 per cent. You might ask what kind of muppet designed a tax schedule like that, and one answer would be George Osborne, chancellor of the exchequer, and Alistair Darling, his predecessor – the last two men to be in charge of the UK tax system.

Another answer would be that this is just the sort of thing that happens without diligent maintenance. Window frames rot. Iron structures rust. Tax systems become complex.

Having nine different marginal tax rates is an ugly sign that things are not well. There are others. Cereal bars attract value added tax at 20 per cent but flapjacks enjoy a zero rate; vegetable chips are tax-free if the vegetable in question is not a potato; dried fruit is subject to VAT unless destined for a cake. On a gingerbread man, chocolate icing attracts a substantial VAT liability unless the icing constitutes the eyes. There are more things in tax accounting, Horatio, than are dreamt of in your philosophy.

If a tax break for unfrosted gingerbread seems uniquely British in its eccentricity, it is not. Officials in New York state have been obliged to rule on the tax status of burritos. (Legally they are sandwiches and attract sales tax of 8 per cent.) Or consider Pillow Pets, a stuffed toy/ pillow whose slogan – “It’s a pillow, it’s a pet, it’s a Pillow Pet” – poses a dilemma for US Customs. For the purposes of levying a tariff, is it a pillow? Or is it a tariff-free toy pet?

Then there are tax subsidies for agricultural land in places such as Florida. Agricultural land is no easier to define than a flapjack or a sandwich. Rent a cow, let it graze on your garden or vacant lot; if that is not agriculture, what is?

All this matters not just because the rules are hard to understand and expensive to obey but also because taxes shape our behaviour. The “camelback” houses of late 19th century New Orleans, with a hump of two storeys at the rear and a long single-storey snout stretching to the street, were tax-efficient because property taxes were levied based on the number of storeys at the front of the house. Abba’s outlandish outfits are reported to have been inspired by tax rules: they were tax-deductible only if they were too outré to be worn anywhere other than on stage.

These are trivial examples of tax-efficient charm but the same principle can be harnessed for a far greater good: a carbon tax to shift our energy system towards low-carbon fuels. Well-designed taxes can raise revenue while rewarding green behaviour.

Meanwhile complex, illogical taxes raise less revenue while rewarding clever accountants. There has been outrage over celebrity tax-dodging in the UK but the tax avoidance schemes usually involve a government attempt to provide a tax incentive for the British film industry or some other hobbyhorse.

What is behind such insanities? Partly, absurd loopholes exist because special interest groups demand them; hence the subsidies for land with cows on it. Partly, voters are given the tax systems they deserve because we sympathise with highly vocal losers whenever a loophole is closed and we fall for simple tricks that hide taxes behind a veil of complication.

The UK’s two-tier income tax system is a good example. Basic income tax rates have tended to fall over time, while national insurance rates have tended to rise. True income tax rates for the typical worker are similar to those of 35 years ago but they seem much lower. The sleight of hand is politically convenient but increases complexity, creates unfairness and opens opportunities for tax avoidance.

It is tempting, then, to call for a radical simplification, for taxes simple enough to write on the back of a postcard. But this ignores the third reason that taxes are complex, which is that fair taxation is a genuinely complex business. This year’s piecemeal reform efforts become next year’s loopholes.

Only radical, systemic reform has much chance of success – and it may be less elegant than some reformers hope. A per-person “poll tax” was introduced in the UK 25 years ago, and promptly ended the premiership of Margaret Thatcher. It was undoubtedly simple – but in taxation, as in life, simplicity is not the only virtue.

Also published at ft.com.

Previous Next

Elsewhere

  • 1 Twitter
  • 2 Flickr
  • 3 RSS
  • 4 YouTube
  • 5 Podcasts
  • 6 Facebook

Books

  • The Undercover Economist Strikes Back
  • Adapt
  • Dear Undercover Economist
  • The Logic of Life
  • The Undercover Economist

About Tim

Tim Harford is an author, columnist for the Financial Times and presenter of Radio 4's "More or Less".
Learn More ›

Enter your email address to receive notifications of new articles by email.

Tim’s Tweets

Do NOT follow this link or you will be banned from the site!