Undercover Economist

Predict the future? We can’t even say what’s happening now

On New Year’s Eve 2007, the Financial Times, in its customary look at the year ahead, declared that “the US will skate along the brink of recession in early 2008, but should avoid tipping over the brink”. In retrospect, we can ruefully enjoy that forecast not because it proved to be wrong – although it was – but because it was wrong even as it was published. The recession actually began in December 2007, just as the FT was announcing that it wouldn’t begin at all. To modify the old quip, “prediction is very difficult, even when it’s not about the future”.

This is one more reason to pity economists. Weather forecasters can glance out of the window if they want to find out what the weather is doing right now. Economists must wait: it took the National Bureau of Economic Research almost a year to declare that the recession had begun at the end of 2007. Economists on the Fed’s Open Market Committee or the Bank of England’s Monetary Policy Committee have the unenviable job of setting interest rates for an economy at whose current state they can only guess.

There’s no mystery as to why “nowcasting” is difficult. The US economy is, to resort to jargon, very big and very complicated. The ideal nowcast would slurp up data on payroll, invoices and payments received in real time from every business in the country. But that asks too much of businesses which are focused on the more pressing matter of giving their customers what they want. Many small businesses simply have no idea how they are doing until several weeks after a quarter has finished, when they figure out how much they owe the taxman. If the business owner herself doesn’t know whether February was a good month until the end of May, it is quite a lot to ask the economists at the Treasury or the Federal Reserve to be more prompt.

Economists continue to wrestle with the problem. A few days before the credit crunch began – more rueful irony – Domenico Giannone and Lucrezia Reichlin of the European Central Bank, with David Small of the Federal Reserve, published a research paper explaining how “nowcasts” of economic growth could be continuously updated as economic data on unemployment, inflation, credit availability and much else trickled out of the offices of economic statistics. David Hendry of Nuffield College, Oxford has, with several colleagues, been working on a particular aspect of the nowcasting problem: spotting a “structural break” – or sharp change in the way the economic variables interact – very shortly after it has taken place. That sounds simple enough, but it is common for forecasting models to miss these structural breaks and then misfire for months or years afterwards.

But if combining official data in new ways is one approach, another is to look for unofficial data sources – such as trending topics on Twitter or the terms most searched for on Google – which are updated much more quickly.

Researchers from Google, including its chief economist Hal Varian and the then head of Google.org, Larry Brilliant, have published research using Google Trends to “nowcast” the spread of influenza, tourist visits to Hong Kong, sales of cars and houses, and unemployment benefit claims. Two economists affiliated with the Bank of Italy, Francesco D’Amuri and Juri Marcucci, have also been tapping into Google data and conclude that it dramatically improves the quality of unemployment “nowcasts” and forecasts – particularly in Italy, where official unemployment data takes two months to arrive.

Real-time data from activity on the web is not perfect, but it’s available almost instantly. I confidently forecast that economists will use more of it in the future. Perhaps they already are.

Also published at ft.com.