Tim Harford The Undercover Economist
  • THE UNDERCOVER ECONOMIST STRIKES BACK
    “Every Tim Harford book is cause for celebration. He makes the ‘dismal science’ seem like an awful lot of fun.”
    – Malcolm Gladwell, author of “The Tipping Point”
  • Adapt – Why Success Always Starts with Failure


    “A highly readable argument... Very impressive”
    – Nassim N Taleb, author of “The Black Swan”
  • Dear Undercover Economist
    “Harford has a knack for explaining economic principles and problems in plain language and, even better, for making them fun”
    – The New York Times
  • The Logic of Life
    “Highly readable, funny and daringly contentious... a whopping good time.”
    – The San Francisco Chronicle
  • The Undercover Economist
    “Required reading”
    – Stephen J. Dubner, co-author of “Freakonomics”
Undercover Economist

When regulators are all out to déjeuner

Just because a problem exists does not mean that a new regulation will solve it

“Each time I visit the city the food gets worse and worse.” Tyler Cowen, economics professor, foodie and author of An Economist Gets Lunch, despairs of Paris. Cowen isn’t the only person to lament the state of French cuisine. This may be why – in a quintessentially French move – the nation’s government has introduced a new law in an attempt to improve standards.

The quixotic law in question is public decree No. 2014-797, more popularly known as the “fait maison” rule, in which restaurants may use a new saucepan-with-a-roof-and-chimney logo on the menu beside any dish that is made on the premises. More accurately, the restaurants must use the saucepan-with-a-roof symbol to denote house-made dishes, but the definition of house-made is rather whimsical, thanks to French legislators.

The entire affair seems unlikely to improve French cuisine but it does provide a nice lesson in practical economics: regulation is a superficially appealing answer to life’s problems but often fails to provide real solutions.

The first difficulty is that regulations are developed by politicians, and politicians pay close attention to lobbyists. In the case of the fait maison rules, it is perfectly legitimate to buy industrially prepared ingredients, provided the dish itself is assembled on the premises. Frozen fish is fine. Skinned and boned chicken is fine. Onion powder seems to be fine. Certain types of factory-made pastry are fine, although others are not. Exceptions are baffling: frozen pommes frites are not allowed unless, of course, the fries are to be oven-baked. Diced, vacuum-packed vegetables are fine but be sure to add a home-made sauce. Eliminating prepared sauces was a priority – but still, it is hard to understand these rules as anything other than the outcome of a prodigious lobbying effort by industrial food companies.

Even if the rules were more logically laid out, the French government would still be committing a classic managerial blunder. To borrow the title of a 1975 article by management professor Steven Kerr, they are engaged in “the folly of rewarding A while hoping for B”.

What France demands, naturellement, is good French food. But insisting on home-made food ensures neither quality nor Frenchness. Freshly prepared food can be terrible, while some food prepared elsewhere is superb. (My brother-in-law is a master baker operating out of an industrial estate by Oxenholme station in northwest England. I’d back his frozen sourdough loaves against fresh bread baked by a more generalist kitchen any day.) The French parliament presumably hopes that by rewarding house-made food it will indirectly improve quality. This is optimistic.

A third problem is that the regulation may produce unintended consequences. Consider a chef who offers a fresh fruit crumble alongside a selection of factory-made cakes and puddings. By law, he or she must display the fait maison logo beside the crumble, implicitly damning all his or her other dishes. Such chefs might decide to offer no house-made dishes at all, rather than bring unwelcome questions to the forefront of their customers’ minds.

Policymaking is flawed and crude while the world is subtle and unpredictable. That is why regulations are often rigged from the start, are only peripherally related to the real matter of concern and have a tendency to backfire.

“There is no substitute for consumers who demand the right kind of food and who otherwise won’t buy it,” says Cowen. This is true. The British surely get the food we deserve, and because we have become less clueless about food, our food has become less appalling. (Perhaps I am wrong. Perhaps Tony Blair passed a law back in the late 1990s outlawing prawn cocktails and tinned vegetables, and I missed it. But I suspect not.)

Yet if informed and demanding consumers are essential for food, they are essential for other markets too. In banking, there is no substitute for consumers who refuse to be sucked in by teaser rates and fines in the small print. In investment, there is no substitute for consumers who avoid high charges and are unmoved by selective claims about past performance. In medicine, there is no substitute for consumers who can tell the difference between an expert doctor, a defensive pusher of scans and blood tests, and an outright quack. But such customers are rare. In fairness to the customers who struggle, it is far harder to identify a good pension than a good pizza.

Regulators, then, must muddle through. Sometimes they outsource the job to professional bodies who will punish egregious offenders. Sometimes they try to outlaw particularly troublesome practices. Sometimes (too rarely) they decide that anything they did would make things worse.

There are few easy answers. Regulations are sometimes essential; they are also sometimes both burdensome and useless. The UK’s planning laws should ensure an adequate supply of elegant, well-built homes. They do not. International rules on financial stability did not give us financial stability. Just because a problem exists does not mean that a new regulation will solve it.

This summer I went to Italy for my summer break. I have always found the food there far better than in France or Britain. I doubt that the credit for that should go to the Italian parliament.

Also published at ft.com.

Undercover Economist

Crushing the competition – at any price

It was Selten’s chain store paradox that first attracted me to economics, with a heady mixture of logic, psychology and military strategy

Forty years ago a German economist, Reinhard Selten, published a working paper with the title “The chain store paradox”. It was simple and profound and showed a discomfiting disconnect between the fashionable mathematical tools known as “game theory” and the recommendations of common sense.

This is the set-up. Imagine a chain store with 20 branches, one in each of 20 small towns. Lacking any competition, these branches charge high prices and are lucrative. In each town, an entrepreneur is considering opening a rival shop. These 20 local entrepreneurs will, one by one, decide whether to compete against the chain store or to sink their capital into something else.

Much depends on the chain store’s response to a competitor. The chain store could be aggressive, ruthlessly slashing prices. That would make life painful for both retailers. The entrepreneur, finding herself committed to a low-margin business, would wish she’d invested her money in something else. Or the chain store could be accommodating, letting prices stay high and sharing a profitable market. So what will happen?

Selten offered two lines of reasoning. One of them is intuitive: the chain store will launch a brutal price war against the first entrepreneur with the temerity to set up as a rival. It will lose money in that market but other entrepreneurs will take note of the bloodbath and will steer clear. The reward for giving up one or two cosy local duopolies is that in the other 18 or 19 towns the chain store will remain a monopolist. The chain store will use price wars as a deterrent, and the deterrent will work.

The alternative line of reasoning is a matter of inductive logic. Consider the 20th and final town. Deterrence is pointless there, since there are no further entrepreneurs to deter. The chain store may as well eschew a price war and accommodate the competitor. Yet if it is obvious that there will be no price war in the 20th town, what about the 19th? If everyone knows there will be no price war in the 20th, what would a price war in the 19th be designed to achieve? Again, deterrence is pointless. But if neither the 20th nor the 19th town will see fierce competition, what is the point of deterrence in the 18th town? The logic rolls back to the beginning of the game; it shows that deterrence is futile and will not be used at any stage.

Selten, an expert in game theory, knew that the second scenario was logically watertight. But as a practical matter he found the first scenario “much more convincing”. If he were the chain store, Selten wrote, he would launch price wars in the hope of deterring later competitors. “I would be very surprised if it failed to work.”

If deterrence works in practice, how to make it work in theory? Selten devoted himself to sharpening up game theory to deal with such apparent paradoxes, and in 1994 he was one of two men to share a Nobel memorial prize with John Nash, he of A Beautiful Mind.

One step forward is to add some vagueness or uncertainty. Can we be sure that the decision will be repeated only 20 times? Can we be sure that the competitors really understand each other? What if the chain store manager is a thug who loves to crush the competition and cares not at all for his shareholders’ dividends? Deterrence becomes a logical solution as well as an intuitive one.

It was Selten’s chain store paradox that first attracted me to economics, with a heady mixture of logic, psychology and something approaching military strategy. And it is a hypothetical game with some close parallels today.

Brad Stone’s excellent book, The Everything Store: Jeff Bezos and the Age of Amazon, paints Amazon’s founder to be a visionary entrepreneur, dedicated to serving his customers. But it also reports that Bezos was willing to take big losses in the hope of weakening competitors. Zappos, the much-loved online shoe retailer, faced competition from an Amazon subsidiary that first offered free shipping and then started paying customers $5 for every pair of shoes they ordered. Quidsi, which ran Diapers.com, was met with a price war from “Amazon Mom”. Industry insiders told Stone that Amazon was losing $1m a day just selling nappies. Both Zappos and Quidsi ended up being bought out by Amazon.

When the weapons of war are low prices, consumers benefit at first. But the long term looks worrying: a future in which nobody dares to compete with Amazon. Apple is a striking contrast: the company’s refusal to compete aggressively on price makes it hugely profitable but has also attracted a swarm of competitors.

Consider a grimmer parallel. Vladimir Putin’s Russia is the chain store. Georgia, Ukraine and many other former Soviet states or satellites must consider whether to seek ties with the west. In each case Putin must decide whether to accommodate or open costly hostilities. The conflict in Ukraine has been disastrous for Russian interests in the short run but it may have bolstered Putin’s personal position. And if his strategy convinces the world that Putin will never share prosperity, his belligerence may yet pay off.

I feel a little guilty comparing Bezos and Putin. My only regret about Bezos’s Amazon is that there aren’t three other companies just like it. I do not feel the same about Putin’s Russia.

Also published at ft.com.

Highlights

How to see into the future

Billions of dollars are spent on experts who claim they can forecast what’s around the corner, in business, finance and economics. Most of them get it wrong. Now a groundbreaking study has unlocked the secret: it IS possible to predict the future – and a new breed of ‘superforecasters’ knows how to do it

Irving Fisher was once the most famous economist in the world. Some would say he was the greatest economist who ever lived. “Anywhere from a decade to two generations ahead of his time,” opined the first Nobel laureate economist Ragnar Frisch, in the late 1940s, more than half a century after Fisher’s genius first lit up his subject. But while Fisher’s approach to economics is firmly embedded in the modern discipline, many of those who remember him now know just one thing about him: that two weeks before the great Wall Street crash of 1929, Fisher announced, “Stocks have reached what looks like a permanently high plateau.”

In the 1920s, Fisher had two great rivals. One was a British academic: John Maynard Keynes, a rising star and Fisher’s equal as an economic theorist and policy adviser. The other was a commercial competitor, an American like Fisher. Roger Babson was a serial entrepreneur with no serious academic credentials, inspired to sell economic forecasts by the banking crisis of 1907. As Babson and Fisher locked horns over the following quarter-century, they laid the foundations of the modern economic forecasting industry.

Fisher’s rivals fared better than he did. Babson foretold the crash and made a fortune, enough to endow the well-respected Babson College. Keynes was caught out by the crisis but recovered and became rich anyway. Fisher died in poverty, ruined by the failure of his forecasts.

If Fisher and Babson could see the modern forecasting industry, it would have astonished them in its scale, range and hyperactivity. In his acerbic book The Fortune Sellers, former consultant William Sherden reckoned in 1998 that forecasting was a $200bn industry – $300bn in today’s terms – and the bulk of the money was being made in business, economic and financial forecasting.

It is true that forecasting now seems ubiquitous. Data analysts forecast demand for new products, or the impact of a discount or special offer; scenario planners (I used to be one) produce broad-based narratives with the aim of provoking fresh thinking; nowcasters look at Twitter or Google to track epidemics, actual or metaphorical, in real time; intelligence agencies look for clues about where the next geopolitical crisis will emerge; and banks, finance ministries, consultants and international agencies release regular prophecies covering dozens, even hundreds, of macroeconomic variables.

Real breakthroughs have been achieved in certain areas, especially where rich datasets have become available – for example, weather forecasting, online retailing and supply-chain management. Yet when it comes to the headline-grabbing business of geopolitical or macroeconomic forecasting, it is not clear that we are any better at the fundamental task that the industry claims to fulfil – seeing into the future.

So why is forecasting so difficult – and is there hope for improvement? And why did Babson and Keynes prosper while Fisher suffered? What did they understand that Fisher, for all his prodigious talents, did not?

In 1987, a young Canadian-born psychologist, Philip Tetlock, planted a time bomb under the forecasting industry that would not explode for 18 years. Tetlock had been trying to figure out what, if anything, the social sciences could contribute to the fundamental problem of the day, which was preventing a nuclear apocalypse. He soon found himself frustrated: frustrated by the fact that the leading political scientists, Sovietologists, historians and policy wonks took such contradictory positions about the state of the cold war; frustrated by their refusal to change their minds in the face of contradictory evidence; and frustrated by the many ways in which even failed forecasts could be justified. “I was nearly right but fortunately it was Gorbachev rather than some neo-Stalinist who took over the reins.” “I made the right mistake: far more dangerous to underestimate the Soviet threat than overestimate it.” Or, of course, the get-out for all failed stock market forecasts, “Only my timing was wrong.”

Tetlock’s response was patient, painstaking and quietly brilliant. He began to collect forecasts from almost 300 experts, eventually accumulating 27,500. The main focus was on politics and geopolitics, with a selection of questions from other areas such as economics thrown in. Tetlock sought clearly defined questions, enabling him with the benefit of hindsight to pronounce each forecast right or wrong. Then Tetlock simply waited while the results rolled in – for 18 years.

Tetlock published his conclusions in 2005, in a subtle and scholarly book, Expert Political Judgment. He found that his experts were terrible forecasters. This was true in both the simple sense that the forecasts failed to materialise and in the deeper sense that the experts had little idea of how confident they should be in making forecasts in different contexts. It is easier to make forecasts about the territorial integrity of Canada than about the territorial integrity of Syria but, beyond the most obvious cases, the experts Tetlock consulted failed to distinguish the Canadas from the Syrias.

Adding to the appeal of this tale of expert hubris, Tetlock found that the most famous experts fared somewhat worse than those outside the media spotlight. Other than that, the humiliation was evenly distributed. Regardless of political ideology, profession and academic training, experts failed to see into the future.

Most people, hearing about Tetlock’s research, simply conclude that either the world is too complex to forecast, or that experts are too stupid to forecast it, or both. Tetlock himself refused to embrace cynicism so easily. He wanted to leave open the possibility that even for these intractable human questions of macroeconomics and geopolitics, a forecasting approach might exist that would bear fruit.

. . .

In 2013, on the auspicious date of April 1, I received an email from Tetlock inviting me to join what he described as “a major new research programme funded in part by Intelligence Advanced Research Projects Activity, an agency within the US intelligence community.”

The core of the programme, which had been running since 2011, was a collection of quantifiable forecasts much like Tetlock’s long-running study. The forecasts would be of economic and geopolitical events, “real and pressing matters of the sort that concern the intelligence community – whether Greece will default, whether there will be a military strike on Iran, etc”. These forecasts took the form of a tournament with thousands of contestants; it is now at the start of its fourth and final annual season.

“You would simply log on to a website,” Tetlock’s email continued, “give your best judgment about matters you may be following anyway, and update that judgment if and when you feel it should be. When time passes and forecasts are judged, you could compare your results with those of others.”

I elected not to participate but 20,000 others have embraced the idea. Some could reasonably be described as having some professional standing, with experience in intelligence analysis, think-tanks or academia. Others are pure amateurs. Tetlock and two other psychologists, Don Moore and Barbara Mellers, have been running experiments with the co-operation of this army of volunteers. (Mellers and Tetlock are married.) Some were given training in how to turn knowledge about the world into a probabilistic forecast; some were assembled into teams; some were given information about other forecasts while others operated in isolation. The entire exercise was given the name of the Good Judgment Project, and the aim was to find better ways to see into the future.

The early years of the forecasting tournament have, wrote Tetlock, “already yielded exciting results”.

A first insight is that even brief training works: a 20-minute course about how to put a probability on a forecast, correcting for well-known biases, provides lasting improvements to performance. This might seem extraordinary – and the benefits were surprisingly large – but even experienced geopolitical seers tend to have expertise in a subject, such as Europe’s economies or Chinese foreign policy, rather than training in the task of forecasting itself.

“For people with the right talents or the right tactics, it is possible to see into the future after all”

A second insight is that teamwork helps. When the project assembled the most successful forecasters into teams who were able to discuss and argue, they produced better predictions.

But ultimately one might expect the same basic finding as always: that forecasting events is basically impossible. Wrong. To connoisseurs of the frailties of futurology, the results of the Good Judgment Project are quite astonishing. Forecasting is possible, and some people – call them “superforecasters”– can predict geopolitical events with an accuracy far outstripping chance. The superforecasters have been able to sustain and even improve their performance.

The cynics were too hasty: for people with the right talents or the right tactics, it is possible to see into the future after all.

Roger Babson, Irving Fisher’s competitor, would always have claimed as much. A serial entrepreneur, Babson made his fortune selling economic forecasts alongside information about business conditions. In 1920, the Babson Statistical Organization had 12,000 subscribers and revenue of $1.35m – almost $16m in today’s money.

“After Babson, the forecaster was an instantly recognisable figure in American business,” writes Walter Friedman, the author of Fortune Tellers, a history of Babson, Fisher and other early economic forecasters. Babson certainly understood how to sell himself and his services. He advertised heavily and wrote prolifically. He gave a complimentary subscription to Thomas Edison, hoping for a celebrity endorsement. After contracting tuberculosis, Babson turned his management of the disease into an inspirational business story. He even employed stonecutters to carve inspirational slogans into large rocks in Massachusetts (the “Babson Boulders” are still there).

On September 5 1929, Babson made a speech at a business conference in Wellesley, Massachusetts. He predicted trouble: “Sooner or later a crash is coming which will take in the leading stocks and cause a decline of from 60 to 80 points in the Dow-Jones barometer.” This would have been a fall of around 20 per cent.

So famous had Babson become that his warning was briefly a self-fulfilling prophecy. When the news tickers of New York reported Babson’s comments at around 2pm, the markets erupted into what The New York Times described as “a storm of selling”. Shares lurched down by 3 per cent. This became known as the “Babson break”.

The next day, shares bounced back and Babson, for a few weeks, appeared ridiculous. On October 29, the great crash began, and within a fortnight the market had fallen almost 50 per cent. By then, Babson had an advertisement in the New York Times pointing out, reasonably, that “Babson clients were prepared”. Subway cars were decorated with the slogan, “Be Right with Babson”. For Babson, his forecasting triumph was a great opportunity to sell more subscriptions.

But his true skill was marketing, not forecasting. His key product, the “Babson chart”, looked scientific and was inspired by the discoveries of Isaac Newton, his idol. The Babson chart operated on the Newtonian assumption that any economic expansion would be matched by an equal and opposite contraction. But for all its apparent sophistication, the Babson chart offered a simple and usually contrarian message.

“Babson offered an up-arrow or a down-arrow. People loved that,” says Walter Friedman. Whether or not Babson’s forecasts were accurate was not a matter that seemed to concern many people. When he was right, he advertised the fact heavily. When he was wrong, few noticed. And Babson had indeed been wrong for many years during the long boom of the 1920s. People taking his advice would have missed out on lucrative opportunities to invest. That simply didn’t matter: his services were popular, and his most spectacularly successful prophecy was also his most famous.

Babson’s triumph suggests an important lesson: commercial success as a forecaster has little to do with whether you are any good at seeing into the future. No doubt it helped his case when his forecasts were correct but nobody gathered systematic information about how accurate he was. The Babson Statistical Organization compiled business and economic indicators that were, in all probability, of substantial value in their own right. Babson’s prognostications were the peacock’s plumage; their effect was simply to attract attention to the services his company provided.

. . .

When Barbara Mellers, Don Moore and Philip Tetlock established the Good Judgment Project, the basic principle was to collect specific predictions about the future and then check to see if they came true. That is not the world Roger Babson inhabited and neither does it describe the task of modern pundits.

When we talk about the future, we often aren’t talking about the future at all but about the problems of today. A newspaper columnist who offers a view on the future of North Korea, or the European Union, is trying to catch the eye, support an argument, or convey in a couple of sentences a worldview that would otherwise be impossibly unwieldy to explain. A talking head in a TV studio offers predictions by way of making conversation. A government analyst or corporate planner may be trying to justify earlier decisions, engaging in bureaucratic defensiveness. And many election forecasts are simple acts of cheerleading for one side or the other.

“Some people – call them ‘superforecasters’– can predict geopolitical events with an accuracy far outstripping chance”

Unlike the predictions collected by the Good Judgment Project, many forecasts are vague enough in their details to allow the mistaken seer off the hook. Even if it was possible to pronounce that a forecast had come true or not, only in a few hotly disputed cases would anybody bother to check.

All this suggests that among the various strategies employed by the superforecasters of the Good Judgment Project, the most basic explanation of their success is that they have the single uncompromised objective of seeing into the future – and this is rare. They receive continual feedback about the success and failure of every forecast, and there are no points for radicalism, originality, boldness, conventional pieties, contrarianism or wit. The project manager of the Good Judgment Project, Terry Murray, says simply, “The only thing that matters is the right answer.”

I asked Murray for her tips on how to be a good forecaster. Her reply was, “Keep score.”

. . .

An intriguing footnote to Philip Tetlock’s original humbling of the experts was that the forecasters who did best were what Tetlock calls “foxes” rather than “hedgehogs”. He used the term to refer to a particular style of thinking: broad rather than deep, intuitive rather than logical, self-critical rather than assured, and ad hoc rather than systematic. The “foxy” thinking style is now much in vogue. Nate Silver, the data journalist most famous for his successful forecasts of US elections, adopted the fox as the mascot of his website as a symbol of “a pluralistic approach”.

The trouble is that Tetlock’s original foxes weren’t actually very good at forecasting. They were merely less awful than the hedgehogs, who deployed a methodical, logical train of thought that proved useless for predicting world affairs. That world, apparently, is too complex for any single logical framework to encompass.

More recent research by the Good Judgment Project investigators leaves foxes and hedgehogs behind but develops this idea that personality matters. Barbara Mellers told me that the thinking style most associated with making better forecasts was something psychologists call “actively open-minded thinking”. A questionnaire to diagnose this trait invites people to rate their agreement or disagreement with statements such as, “Changing your mind is a sign of weakness.” The project found that successful forecasters aren’t afraid to change their minds, are happy to seek out conflicting views and are comfortable with the notion that fresh evidence might force them to abandon an old view of the world and embrace something new.

Which brings us to the strange, sad story of Irving Fisher and John Maynard Keynes. The two men had much in common: both giants in the field of economics; both best-selling authors; both, alas, enthusiastic and prominent eugenicists. Both had immense charisma as public speakers.

Fisher and Keynes also shared a fascination with financial markets, and a conviction that their expertise in macroeconomics and in economic statistics should lead to success as an investor. Both of them, ultimately, were wrong about this. The stock market crashes of 1929 – in September in the UK and late October in the US – caught each of them by surprise, and both lost heavily.

Yet Keynes is remembered today as a successful investor. This is not unreasonable. A study by David Chambers and Elroy Dimson, two financial economists, concluded that Keynes’s track record over a quarter century running the discretionary portfolio of King’s College Cambridge was excellent, outperforming market benchmarks by an average of six percentage points a year, an impressive margin.

This wasn’t because Keynes was a great economic forecaster. His original approach had been predicated on timing the business cycle, moving into and out of different investment classes depending on which way the economy itself was moving. This investment strategy was not a success, and after several years Keynes’s portfolio was almost 20 per cent behind the market as a whole.

The secret to Keynes’s eventual profits is that he changed his approach. He abandoned macroeconomic forecasting entirely. Instead, he sought out well-managed companies with strong dividend yields, and held on to them for the long term. This approach is now associated with Warren Buffett, who quotes Keynes’s investment maxims with approval. But the key insight is that the strategy does not require macroeconomic predictions. Keynes, the most influential macroeconomist in history, realised not only that such forecasts were beyond his skill but that they were unnecessary.

Irving Fisher’s mistake was not that his forecasts were any worse than Keynes’s but that he depended on them to be right, and they weren’t. Fisher’s investments were leveraged by the use of borrowed money. This magnified his gains during the boom, his confidence, and then his losses in the crash.

But there is more to Fisher’s undoing than leverage. His pre-crash gains were large enough that he could easily have cut his losses and lived comfortably. Instead, he was convinced the market would turn again. He made several comments about how the crash was “largely psychological”, or “panic”, and how recovery was imminent. It was not.

One of Fisher’s major investments was in Remington Rand – he was on the stationery company’s board after selling them his “Index Visible” invention, a type of Rolodex. The share price tells the story: $58 before the crash, $28 by 1930. Fisher topped up his investments – and the price soon dropped to $1.

Fisher became deeper and deeper in debt to the taxman and to his brokers. Towards the end of his life, he was a marginalised figure living alone in modest circumstances, an easy target for scam artists. Sylvia Nasar writes in Grand Pursuit, a history of economic thought, “His optimism, overconfidence and stubbornness betrayed him.”

. . .

So what is the secret of looking into the future? Initial results from the Good Judgment Project suggest the following approaches. First, some basic training in probabilistic reasoning helps to produce better forecasts. Second, teams of good forecasters produce better results than good forecasters working alone. Third, actively open-minded people prosper as forecasters.

But the Good Judgment Project also hints at why so many experts are such terrible forecasters. It’s not so much that they lack training, teamwork and open-mindedness – although some of these qualities are in shorter supply than others. It’s that most forecasters aren’t actually seriously and single-mindedly trying to see into the future. If they were, they’d keep score and try to improve their predictions based on past errors. They don’t.

“Successful forecasters aren’t afraid to change their minds and are comfortable with the notion that fresh evidence might mean abandoning an old view”

This is because our predictions are about the future only in the most superficial way. They are really advertisements, conversation pieces, declarations of tribal loyalty – or, as with Irving Fisher, statements of profound conviction about the logical structure of the world. As Roger Babson explained, not without sympathy, Fisher had failed because “he thinks the world is ruled by figures instead of feelings, or by theories instead of styles”.

Poor Fisher was trapped by his own logic, his unrelenting optimism and his repeated public declarations that stocks would recover. And he was bankrupted by an investment strategy in which he could not afford to be wrong.

Babson was perhaps wrong as often as he was right – nobody was keeping track closely enough to be sure either way – but that did not stop him making a fortune. And Keynes prospered when he moved to an investment strategy in which forecasts simply did not matter much.

Fisher once declared that “the sagacious businessman is constantly forecasting”. But Keynes famously wrote of long-term forecasts, “About these matters there is no scientific basis on which to form any calculable probability whatever. We simply do not know.”

Perhaps even more famous is a remark often attributed to Keynes. “When my information changes, I alter my conclusions. What do you do, sir?”

If only he had taught that lesson to Irving Fisher.

Also published at ft.com.

Undercover Economist

Ice bucket challenge: the cold facts

In a world of limited generosity, who is to say which cause should be at the head of the queue?

Last week I finally succumbed to social pressure and invited some colleagues at the BBC to film me having a bucket of iced water tipped over my head. As surely nobody needs telling by now, the deal is that people film themselves being drenched, donate money to the US-based ALS Association or its British equivalent, the Motor Neurone Disease Association, and then nominate three further people for the same treatment.

The challenge is an infectious plague, humiliation TV and pyramid scheme all rolled into one, and it’s fundraising genius. Lady Gaga’s done it; Mark Zuckerberg has done it; George W Bush has done it. By the time this column is in print, I imagine everyone on the planet will have done it.

Social pressure is a powerful thing, and it’s refreshing to see it being used to spread smiles and encourage a generous spirit. This is not new, of course. Charities have long sought celebrity endorsements, and seeing famous people have liquids poured on them is a venerable tradition. As for seeking sponsorship to run a marathon or climb Kilimanjaro, we all know that shamelessly pressuring friends and colleagues to give money is the very essence of the exercise.

Peer pressure can also produce reluctant givers. Adriaan Soetevent, an economist at the University of Groningen, studied church collections in an open basket versus a closed collection bag. The open basket elicited larger donations. And in another clever field experiment run by three economists, Stefano DellaVigna, John List and Ulrike Malmendier, fundraisers went door to door raising money. Some households, chosen randomly, had received a flyer warning them exactly when the fundraisers would be around: this warning dramatically increased the chance that the door would not be opened. Not all of us welcome the opportunity to give money to randomly selected charities, it seems.

This time, the social element seems to be a source of no small joy: at a family gathering recently, people were gleefully ice-bucketing each other until the garden had become a swamp. Surely the ice bucket challenge is a good thing, raising money for a worthy cause while giving us a good chuckle into the bargain.

But any good economist has to ask – and I do apologise about this – “a good thing compared to what?” Some critics have suggested that charitable donations are a zero-sum game: more money for the ALS and MND associations means less money for other charities. The evidence for that proposition is thin, as it happens, but even if the many tens of millions raised by the ice bucket challenge are brand-new charitable giving, we could still ask where that money would best be spent.

The strength of a viral giving campaign is also its weakness: people join in for a laugh because their friends have put them up to it, rather than because of a logical analysis of the most worthy cause. Motor neurone disease traps people in their own bodies as they lose the ability to move, speak, eat and, eventually, even to breathe. It is a truly dreadful condition – but so is bowel cancer, fatal diarrhoea or simply starving to death. In a world of limited generosity and finite resources, who is to say which cause should be at the head of the queue?

The fact that ice-bucketeers are donating to the ALS Association feels entirely arbitrary. If the Red Cross or the American Cancer Society had happened to be the beneficiaries instead, very little else about the viral campaign would have changed. Would that have been a better situation?

GiveWell is an organisation which seems well placed to answer such questions: it aims to give donors the information they need to make the most effective donations. It sounds like an impossible job. GiveWell’s approach is to find cost-effective, evidence-based approaches such as distributing antimalarial bednets, and then search for transparent, efficient charities pursuing that approach. One of their top recommendations, for example, is the Schistosomiasis Control Initiative – a charity that could use a catchier name. It organises treatment for parasitic worms, a very unsexy cause indeed. But the worms can do a lot of harm and are absurdly inexpensive to treat – hence the finding that the SCI offers value for your donated money.

In the end, I sent a few pounds of my ice bucket donation to the Motor Neurone Disease Association. It would have felt wrong, somehow, to do otherwise. I sent a more substantial donation to SCI, surely one of the least media-friendly charities on the planet. All lives are equally valuable but some lives may be saved far more cheaply than others. It seems strange not to respond to a philanthropic bargain.

No doubt some will find this line of reasoning colder than a bucket full of iced water. But the truth is that whenever we give money to one cause rather than another, we’re making a decision about how deserving that cause is. When a social media campaign gathers momentum, it is human nature to make that decision spontaneously and without a moment’s reflection. It feels good. But feeling good and doing good are not the same thing.

You can donate to the SCI at www3.imperial.ac.uk/schisto

Also published at ft.com.

Undercover Economist

Here today, gone tomorrow

Don’t draw up your task list in the morning – do it the evening before, when you will have a more distant perspective

What does going on a diet have in common with time management? Here’s a musical clue: Little Orphan Annie sings: “Tomorrow, tomorrow, I love you, tomorrow – you’re always a day away.” Sheila Hancock’s song “My Last Cigarette” has a more cynical bent: “I’ll give up the habit, I will even yet, when I’ve had just one more cigarette.”

The songs could hardly be more different but the common thread is the way that the promise of tomorrow is transformed overnight into something altogether different: today. Strange things happen to us when tomorrow turns into today. Tomorrow we’ll eat fruit rather than candy bars. Tomorrow we’ll watch Krzysztof Kieślowski’s Blue rather than Sleepless in Seattle. And yet curiously when tomorrow arrives, we eat chocolate and watch romcoms. Our preferences flip.

This isn’t just my whimsical summary of human nature: the psychologist Daniel Read of Warwick Business School and his colleagues have conducted experiments finding pretty much exactly this behaviour. Experimental participants, given the opportunity to select food or movies in advance, are more likely to choose the highbrow film or the healthy snack. When the moment of truth arrives, they often change their minds if given the option.

Economists give this tendency the charmless name of hyperbolic discounting. Hyperbolic discounting poses some obvious and well-understood problems for those of us going on a diet or saving for a pension. The problem of personal productivity, however, is far thornier.

On any typical day – indeed, from moment to moment – we have to decide how to spend our time. We have a choice of long-term and short-term projects, big and small tasks/jobs, fixed commitments and free time, all within a daily rhythm of productive moments and postprandial slumps. To add to the challenge, unexpected tasks are always arriving in the inbox.

Armed with traditional tools of to-do list and calendar, this already looks like a tough enough optimisation problem. Add hyperbolic discounting and it looks vicious.

“Managing time is almost inhumane in its requirements,” says Dan Ariely, a behavioural scientist at Duke University. He’s right. While trying to figure out the wisest way to spend our time, we are constantly tempted to surf around on YouTube. Or perhaps we engage in busy-work, reorganising the filing cabinet and kidding ourselves that just because it’s work, it’s worth doing. Tomorrow’s priorities – applying for a promotion, starting the next big project, learning a new language – keep evaporating whenever tomorrow turns into today.

What are the solutions?

One possibility is to schedule tasks ahead of time in the calendar. The big presentation, the Japanese revision, the washing-up, all of it gets a diary slot. There’s promise in this approach. It still requires willpower but putting long-term priorities firmly in the calendar helps deal with the hyperbolic discounting problem. But an overstuffed diary is inflexible and one missed target means an entire calendar must be reworked. The system is unlikely to work for all but the most predictable lists of tasks.

Perhaps technology can save us. Ariely is part of a team producing a new smartphone app, Timeful, which aims to deliver the diary-stuffing approach more intelligently. The idealised form of the software would know everything you wanted to get done – from writing a novel to having a drink with old friends to doing the laundry. It would know how long each task would take, by when it had to be done, how important it was and when might be a productive time to do it. The software would also have access to your calendar, and it would tentatively schedule your tasks wherever it found free space. Over time it would learn about your productivity.

This seems enormously useful, although much depends on how close the algorithm comes to this idealised vision – and how much fuss it is to interact with it. (Users of iPhones can give it a try right now.)

. . .

For those who prefer a pen-and-paper approach to productivity, what to do about the hyperbolic discounting problem? I have two suggestions. The first helps bring a long-term perspective to the daily to-do list. Don’t draw up your list of tasks first thing in the morning – do it the previous evening, when you will have a slightly more distant perspective. When you do so, think about the two or three tasks you would feel most satisfied to have ticked off. Put those at the top of the list and make them your priority.

The second suggestion flips the telescope around and brings today’s perspective to tomorrow’s commitments. When being invited to do things months in advance, the diary usually looks pretty clear and it’s tempting to say “yes”. But whenever a new invitation arrives, ask yourself not, “should I accept the invitation in March?” but, “would I accept the invitation if it was for this week?”

The fundamental insight of hyperbolic discounting is that while tomorrow always looks different, eventually tomorrow will be today. If the flattering invitation would be impossible to accept for this week, what on earth makes you think the first week of March will look any different once it arrives? Tomorrow is always a day away – but your rash commitments are not.

Also published at ft.com.

Video

The secrets of superforecasting

Newsnight videos aren’t embeddable, it seems, but you may enjoy this four minutes of fun, in which I discuss the latest research on how to see into the future.

I discuss the research in much more detail in my FT article, “How to See Into the Future” – no paywall, so enjoy.

8th of September, 2014VideoComments off
Other Writing

Why inflation remains best way to avoid stagnation

The prospect is that central banks will find themselves helpless, writes Tim Harford

People who were not born when the financial crisis began are now old enough to read about it. We have been able to distract ourselves with two Olympics, two World Cups and two US presidential elections. Yet no matter how stale our economic troubles feel, they manage to linger.

Given the severity of the crisis and the inadequacy of the policy response, it should be no surprise that recovery has been slow and anaemic: that is what economic history always suggested. Yet some economists are growing disheartened. The talk is of “secular stagnation” – a phrase which could mean two things, neither of them good.

One fear has been well-aired: that future growth possibilities will be limited by an ageing population or perhaps even technological stagnation.

The second meaning of secular stagnation is altogether stranger: it is that regardless of their potential for growth, modern economies may suffer from a persistent tendency to slip below that potential, sliding into stubborn recessions. The west’s lost decade of economic growth may be a taste of things to come.

This view was put forward most forcefully by Lawrence Summers, who was Treasury secretary under Bill Clinton and a senior adviser to President Barack Obama. It has been discussed at length in a collection of essays published last week by the Centre for Economic Policy Research. But what could it mean?

Normally, when an economy slips into recession, the standard response is to cut interest rates. This encourages us to spend, rather than save, giving the economy an immediate boost.

Things become more difficult if nominal interest rates are already low. Central banks have to employ radical tactics of uncertain effectiveness, such as quantitative easing. Governments could and should borrow and spend to support the economy. In practice they have proved politically gridlocked (in the US), institutionally hamstrung (in the EU) or ideologically blinkered (in the UK). There is not much reason to think the politics of fiscal stimulus would be very different in the future, so the zero-interest rate boundary is a problem.

The awful prospect of secular stagnation is that this is the new normal. Interest rates will be very low as a matter of course, and central banks will routinely find themselves nearly helpless.
“A cut in interest rates encourages us to spend, rather than save, giving the economy an immediate boost”

Before we startle ourselves at shadows, let us ask why Prof Summers might be right. Real interest rates – the rates paid after adjusting for inflation – have been falling. In the US, real rates averaged about 5 per cent in the 1980s, 2 per cent in the 1990s and 1 per cent in the Noughties. (Since Lehman Brothers failed they have been negative, but the long-term trend speaks more eloquently.) Real interest rates have also been declining in the EU for 20 years. The International Monetary Fund’s estimate of global real interest rates has been declining for 30 years.

This does not look good, so why is it happening? The background level of real interest rates is set not by central banks but by supply and demand. Low real rates suggest lots of people are trying to save, and particularly in safe assets, while few people are trying to borrow and invest. Only with rates at a very low level can enough borrowers be found to mop up all the savings.

If secular stagnation is a real risk, we need policies to address it. One approach is to try to change the forces of supply and demand to boost the demand for cash to invest, while stemming the supply of savings, and reducing the bias towards super-safe assets.

This looks tricky. Much policy has pushed in the opposite direction. Consider the austerity drive and long-term goals to reduce government debt burdens; this reduces the supply of safe assets and pushes down real rates. Or the tendency in the UK to push pension risk away from companies and the government, and towards individuals; this encourages extra saving, just in case. Or the way in which (understandably) regulators insist that banks and pension funds hold more safe assets; again, this increases the demand for safe assets and pushes down real interest rates. To reverse all these policies, sacrificing microeconomic particulars for a rather abstract macroeconomic hunch, looks like a hard sell.

There is a simple alternative, albeit one that carries risks. Central bank targets for inflation should be raised to 4 per cent. A credible higher inflation target would provide immediate stimulus (who wants to squirrel away money that is eroding at 4 per cent a year?) and would give central banks more leeway to cut real rates in future. If equilibrium real interest rates are zero, that might not matter when central banks can produce real rates of minus 4 per cent.

If all that makes you feel queasy, it should. As Prof Summers argues, unpleasant things have a tendency to happen when real interest rates are very low. Bubbles inflate, Ponzi schemes prosper and investors are reckless in their scrabble for yield.

One thing that need not worry anyone, though, is the prospect of an inflation target of 4 per cent. It will not happen. That is particularly true in the place where the world economy most needs more inflation: in the eurozone. The German folk memory of hyperinflation in 1923 is just too strong. That economic catastrophe, which helped lay the foundations for Nazism and ruin much of the 20th century, continues to resonate today.

What practical policy options remain? That is easy to see. We must cross our fingers and hope that Prof Summers is mistaken.

Also published at ft.com.

5th of September, 2014Other WritingComments off
Video

In which I take the ice-bucket challenge the nerdy way…

On “More or Less” this week we discuss the ice bucket challenge: are viral memes a good way to get people giving to charity, or does it make us careless about what the charities are doing with the money? Is it possible to use data to identify the very best charity in the world? Listen on bbc.co.uk/moreorless.

You can donate to the Schistosomiasis Control Initiative here: http://www3.imperial.ac.uk/schisto

 

 

I’d just like to clarify that I an not sitting on the loo – it’s a shower with a seat. Oh, why am I even bothering to try to maintain my dignity?

5th of September, 2014VideoComments off
Other Writing

Monopoly is a bureaucrat’s friend but a democrat’s foe

The challenges from smaller competitors spur the innovations that matter

“It takes a heap of Harberger triangles to fill an Okun gap,” wrote James Tobin in 1977, four years before winning the Nobel Prize in economics. He meant that the big issue in economics was not battling against monopolists but preventing recessions and promoting recovery.

After the misery of recent years, nobody can doubt that preventing recessions and promoting recovery would have been a very good idea. But economists should be able to think about more than one thing at once. What if monopoly matters, too?

The Harberger triangle is the loss to society as monopolists raise their prices, and it is named after Arnold Harberger, who 60 years ago discovered that the costs of monopoly were about 0.1 per cent of US gross domestic product – a few billion dollars these days, much less than expected and much less than a recession.

Professor Harberger’s discovery helped build a consensus that competition authorities could relax about the power of big business. But have we relaxed too much?

Large companies are all around us. We buy our mid-morning coffee from global brands such as Starbucks, use petrol from Exxon or Shell, listen to music purchased from a conglomerate such as Sony (via Apple’s iTunes), boot up a computer that runs Microsoft on an Intel processor. Crucial utilities – water, power, heating, internet and telephone – are supplied by a few dominant groups, with baffling contracts damping any competition.

Of course, not all large businesses have monopoly power. Tesco, the monarch of British food retailing, has found discount competitors chopping up its throne to use as kindling. Apple and Google are supplanting Microsoft. And even where market power is real, Prof Harberger’s point was that it may matter less than we think. But his influential analysis focused on monopoly pricing. We now know there are many other ways in which dominant businesses can harm us.

In 1989 the Beer Orders shook up a British pub industry controlled by six brewers. The hope was that more competition would lead to more and cheaper beer. It did not. The price of beer rose. Yet so did the quality of pubs. Where once every pub had offered rubbery sandwiches and stinking urinals, suddenly there were sports bars, candlelit gastropubs and other options. There is more to competition than lower prices.

Monopolists can sometimes use their scale and cash flow to produce real innovations – the glory years of Bell Labs come to mind. But the ferocious cut and thrust of smaller competitors seems a more reliable way to produce many of the everyday innovations that matter.

That cut and thrust is no longer so cutting or thrusting as once it was. “The business sector of the US economy is ageing,” says a Brookings research paper. It is a trend found across regions and industries, as incumbent players enjoy entrenched advantages. “The rate of business start-ups and the pace of employment dynamism in the US economy has fallen over recent decades . . . This downward trend accelerated after 2000,” adds a survey in the Journal of Economic Perspectives.

That means higher prices and less innovation, but perhaps the game is broader still. The continuing debate in the US over “net neutrality” is really an argument about the least damaging way to regulate the conduct of cable companies that hold local monopolies. If customers had real choice over their internet service provider, net neutrality rules would be needed only as a backstop.

As the debate reminds us, large companies enjoy power as lobbyists. When they are monopolists, the incentive to lobby increases because the gains from convenient new rules and laws accrue solely to them. Monopolies are no friend of a healthy democracy.

They are, alas, often the friend of government bureaucracies. This is not just a case of corruption but also about what is convenient and comprehensible to a politician or civil servant. If they want something done about climate change, they have a chat with the oil companies. Obesity is a problem to be discussed with the likes of McDonald’s. If anything on the internet makes a politician feel sad, from alleged copyright infringement to “the right to be forgotten”, there is now a one-stop shop to sort it all out: Google.

Politicians feel this is a sensible, almost convivial, way to do business – but neither the problems in question nor the goal of vigorous competition are resolved as a result.

One has only to consider the way the financial crisis has played out. The emergency response involved propping up big institutions and ramming through mergers; hardly a long-term solution to the problem of “too big to fail”. Even if smaller banks do not guarantee a more stable financial system, entrepreneurs and consumers would profit from more pluralistic competition for their business.

No policy can guarantee innovation, financial stability, sharper focus on social problems, healthier democracies, higher quality and lower prices. But assertive competition policy would improve our odds, whether through helping consumers to make empowered choices, splitting up large corporations or blocking megamergers. Such structural approaches are more effective than looking over the shoulders of giant corporations and nagging them; they should be a trusted tool of government rather than a last resort.

As human freedoms go, the freedom to take your custom elsewhere is not a grand or noble one – but neither is it one that we should abandon without a fight.

Also published at ft.com.

16th of August, 2014Other WritingComments off
Other Writing

Pity the robot drivers snarled in a human moral maze

Robotic cars do not get tired, drunk or angry but there are bound to be hiccups, says Tim Harford

Last Wednesday Vince Cable, the UK business secretary, invited British cities to express their interest in being used as testing grounds for driverless cars. The hope is that the UK will gain an edge in this promising new industry. (German autonomous cars were being tested on German, French and Danish public roads 20 years ago, so the time is surely ripe for the UK to leap into a position of technological leadership.)

On Tuesday, a very different motoring story was in the news. Mark Slater, a lorry driver, was convicted of murdering Trevor Allen. He had lost his temper and deliberately driven a 17 tonne lorry over Mr Allen’s head. It is a striking juxtaposition.

The idea of cars that drive themselves is unsettling, but with drivers like Slater at large, the age of the driverless car cannot come quickly enough.

But the question of how safe robotic cars are, or might become, is rather different from the question of the risks of a computer-guided car are perceived, and how they might be repackaged by regulators, insurers and the courts.

On the first question, it is highly likely that a computer will one day do a better, safer, more courteous job of driving than you can. It is too early to be certain of that, because serious accidents are rare. An early benchmark for Google’s famous driverless car programme was to complete 100,000 miles driving on public roads – but American drivers in general only kill someone every 100m miles.

Still, the safety record so far seems good, and computers have some obvious advantages. They do not get tired, drunk or angry. They are absurdly patient in the face of wobbly cyclists, learner drivers and road hogs.

But there are bound to be hiccups. While researching this article my Google browser froze up while trying to read a Google blog post hosted on a Google blogging platform. Two seconds later the problem had been solved, but at 60 miles per hour two seconds is more than 50 metres. One hopes that Google-driven cars will be more reliable when it comes to the more literal type of crash.

Yet the exponential progress of cheaper, faster computers with deeper databases of experience will probably guarantee success eventually. In a simpler world, that would be the end of it.

Reality is knottier. When a car knocks over a pedestrian, who is to blame? Our answer depends not only on particular circumstances but on social norms. In the US in the 1920s, the booming car industry found itself under pressure as pedestrian deaths mounted. One response was to popularise the word “jaywalking” as a term of ridicule for bumpkins who had no idea how to cross a street. Social norms changed, laws followed, and soon enough the default assumption was that pedestrians had no business being in the road. If they were killed they had only themselves to blame.

We should prepare ourselves for a similar battle over robot drivers. Assume that driverless cars are provably safer. When a human driver collides with a robo-car, where will our knee-jerk sympathies lie? Will we blame the robot for not spotting the human idiosyncrasies? Or the person for being so arrogant as to think he could drive without an autopilot?

When such questions arrive in the courts, as they surely will, robotic cars have a serious handicap. When they err, the error can be tracked back to a deep-pocketed manufacturer. It is quite conceivable that Google, Mercedes or Volvo might produce a robo-car that could avoid 90 per cent of the accidents that would befall a human driver, and yet be bankrupted by the legal cases arising from the 10 per cent that remained. The sensible benchmark for robo-drivers would be “better than human”, but the courts may punish them for being less than perfect.

There are deep waters here. How much space is enough when overtaking a slow vehicle – and is it legitimate for the answer to change when running late? When a child chases a ball out into the road, is it better to swerve into the path of an oncoming car, or on to the pavement where the child’s parents are standing, or not to swerve at all?

These are hardly thought of as ethical questions because human drivers make them intuitively and in an instant. But a computer’s priorities must be guided by its programmers, who have plenty of time to weigh up the tough ethical choices.

In 1967 Philippa Foot, one of Oxford’s great moral philosophers, posed a thought experiment that she called the “trolley problem”. A runaway railway trolley is about to kill five people, but by flipping the points, you can redirect it down a line where it will instead kill one. Which is the right course of action? It is a rich seam for ethical discourse, with many interesting variants. But surely Foot did not imagine that the trolley problem would have to be answered one way or another and wired into the priorities of computer chauffeurs – or that lawyers would second-guess those priorities in court in the wake of an accident.

Then there is the question of who opts for a driverless car. Sir David Spiegelhalter, a risk expert at Cambridge university, points out that most drivers are extremely safe. Most accidents are caused by a few idiots, and it is precisely those idiots, Sir David speculates, who are least likely to cede control to a computer.

Perhaps driverless cars will be held back by a tangle of social, legal and regulatory stubbornness. Or perhaps human drivers will one day be banned, or prohibitively expensive to insure. It is anyone’s guess, because while driving is no longer the sole preserve of meatsacks such as you and me, the question of what we fear and why we fear it remains profoundly, quirkily human.

Also published at ft.com.

7th of August, 2014Other WritingComments off
Previous

Elsewhere

  • 1 Twitter
  • 2 Flickr
  • 3 RSS
  • 4 YouTube
  • 5 Podcasts
  • 6 Facebook

Books

  • The Undercover Economist Strikes Back
  • Adapt
  • Dear Undercover Economist
  • The Logic of Life
  • The Undercover Economist

About Tim

Tim Harford is an author, columnist for the Financial Times and presenter of Radio 4's "More or Less".
Learn More ›

Enter your email address to receive notifications of new articles by email.

Tim’s Tweets

Do NOT follow this link or you will be banned from the site!