Tim Harford The Undercover Economist

Articles published in July, 2013

Have prisoners learned not to snitch?

“They finally tested the ‘prisoner’s dilemma’ on actual prisoners – and the results were not what you would expect”
BusinessInsider.com, July 21

It is amazing that it has taken six decades to get around to testing out the prisoner’s dilemma on real prisoners.

Is it? The prisoner’s dilemma is a little fable economics teachers use to make a formal example more memorable. It was never actually a prediction of how prisoners behaved. Presumably you also think it’s remarkable that two and a half thousand years after Aesop, nobody has got around to re-enacting “the Tortoise and the Hare” with a real hare?

Grumpy today, aren’t we?

Well, I do feel that the prisoner’s dilemma has unjustly monopolised the popular imagination of what game theory is all about. Game theory attempts to analyse situations where a few people interact in such
a way that they need to take each others’ behaviour into account. The prisoner’s dilemma is just one such example.

Why has it garnered so much interest?

Partly because it seems to produce a paradox, and partly because there’s a good story to go with it. Two suspects are charged with a serious crime, carrying a sentence of 20 years. There’s only enough evidence to convict them of something more trivial, with a sentence of two years. The prosecutors tell each prisoner that if he confesses to their joint guilt, his sentence will be halved. Furthermore, if he confesses and the other prisoner is silent, he won’t even be charged with the more serious offence.

Hang on, let me get a pencil.

The prisoner’s dilemma, then, is that you get one year if you confess and the other guy stays silent, two years if you both stay silent, 10 years if you both confess and 20 years if you stay silent and the other guy confesses. The apparent paradox is that no matter what the other prisoner does, you will improve your own situation by confessing – yet as a pair the prisoners would be much better off if they could somehow both commit to remain silent. So the dilemma is seen to symbolise all sorts of problems of social co-operation.

You seem sceptical.

Well, the prisoner’s dilemma has a dark beauty to it but as a model of real life it’s not great – when the game is changed so that the prisoners are put in the same situation repeatedly, for instance, the results change. A less famous game, known as the “stag hunt”, is often a better model for the difficulty of co-operating. A run on a bank is not much like a prisoner’s dilemma but a lot like a stag hunt.

Are you not curious to know what the prisoners did when faced with the prisoner’s dilemma?

Oh, I certainly am curious – curious enough to look up the research. The researchers, Menusch Khadjavi and Andreas Lange, conducted low-stakes prisoner’s dilemma experiments both with female prisoners and female students. The headline-grabbing result: prisoners co-operate with each other more than students do.

What does that tell us – that prisoners take care of each other? Or that they fear reprisals?

Probably not reprisals: they were promised anonymity. It’s really not clear what this result tells us. We knew already that people often co-operate, contradicting the theoretical prediction. We also know, for instance, that economics students co-operate more rarely than non-economists – perhaps because they’ve been socialised to be selfish people, or perhaps because they just understand the dilemma better.

You think the prisoners just didn’t understand the nature of the dilemma?

That’s possible. The students were much better educated and most students had played laboratory games before. Maybe the prisoners co-operated because they were too confused to betray each other.

That seems speculative.

It is speculative, but consider this: the researchers also looked at a variant game in which one player has to decide whether to stay silent or confess, and then the other player decides how to respond. If you play first in this game you would be well-advised to stay silent, because people typically reward you for that. In this sequential game, it was the students, not the prisoners, who were more likely to co-operate with each other by staying silent. So the students were just as co-operative as prisoners but their choice of when to co-operate with each other made more logical sense.

First published on FT.com.

Undercover Economist: The laws of Rockonomics

Three lessons on money and market forces derived from the music business

Economics and rock ’n’ roll don’t necessarily go hand in hand, but there’s something in the air this summer: my attention has been called to not one but three separate economics lessons from the music business.

Lesson one comes from the front line: according to NPR’s economics podcast Planet Money, the rap/country/rock crossover artist Kid Rock has been going to war against ticket touts.

The existence of touts has long been a puzzle for microeconomists: the entire tout business model of buying underpriced tickets and reselling them for a fat mark-up would appear to depend on irrationally low ticket pricing. Why don’t rock stars simply charge more?

There are a number of possible explanations but for Kid Rock, it’s simple: he produces blue-collar music and it doesn’t feel right to be charging a premium price. Fans might be willing to pay hundreds of dollars for a concert ticket, but would resent it if Kid Rock himself tried to charge that much.

Kid Rock’s four-part solution: flood the market with lots of shows to make sure that even at the low price of $20, there are still tickets available; have a small number of super-expensive tickets near the front; allocate the very best seats by lottery to people who bought the cheap tickets; and enforce it all with paperless tickets and an ID check to prevent resale. According to Planet Money, it’s all working: concerts are pretty full but you can still buy $20 tickets from official sources.

Lesson two is about globalisation. A new article in The Economic Journal from Fernando Ferreira and Joel Waldfogel asks whether in a world of MTV and YouTube, national musical cultures are being crushed by American imports. Ferreira and Waldfogel have assembled more than a million data points covering chart hits in 22 countries, in some cases going back to 1960. In practice this covers pretty much the entire global music market, and the data are used to estimate the value of music sales.

At first glance, worries about the cultural dominance of the US seem justified: US artists are responsible for 60 per cent of world music sales. But US artists were responsible for 80 per cent of world music sales in the early 1960s before dramatically losing market share to the British. (We are now, alas, in sharp decline.)

In the early 1980s, less than 50 per cent of music sales were by domestic artists – that is, French artists selling in France, or Brazilian artists selling in Brazil. By 2007 that figure was around two-thirds. Domestically produced music is having a renaissance – proof that globalisation has more complex effects than we tend to assume.

The final rockonomics lesson of the summer comes from Alan Krueger, the chairman of Barack Obama’s Council of Economic Advisers. In a recent speech Professor Krueger sought to draw a parallel between the music industry and the threat to the American middle class. The core of his argument: in music as in life, the rich are getting richer while the rest are being left behind.

Krueger argued that this is partly the result of technological change that allows a winner-takes-all distribution of rewards. He also argued, quite convincingly, that the difference between being one of these ever-wealthier winners and an also-ran is often a matter of luck.

But what to do? Krueger was keen to applaud President Obama’s priorities as being an antidote: reforming healthcare, encouraging pre-school education, revitalising distressed communities, increasing the minimum wage and raising a stink about high pay for bankers and chief executives. I agree with Krueger about the problems but I wonder whether such approaches will do much to solve them.

It isn’t easy to harness market forces, reward the middle class and make a profit into the bargain. Perhaps Professor Krueger should ask Kid Rock for advice.

Also published at ft.com.

Welfare and the Tinker Bell policy

The UK benefits cap is a stunt policy designed to win attention, writes Tim Harford

‘The benefit cap starts being introduced across the country today, restoring fairness to the welfare state . . . with the amount of benefits working-age households can claim limited to the average working wage – £500 a week.’ Press release from the Department for Work and Pensions, July 15

“Restoring fairness to the welfare state.” I like that.

Indeed. The government spends about £2,500 per citizen a year on benefits – £160bn in total – and this policy is projected to save about £3 per citizen a year in the long run. That’s an awful lot of “restoring fairness” resting on a very small sum of money.

Surely it’s the symbolism of the thing.

Less “restoring fairness” than “creating the superficial appearance of fairness”, then? This is a stunt policy, designed to win attention, and I am depressed that you insist on talking about it. I think we should treat Iain Duncan Smith, the work and pensions secretary, the way we treat a three-year-old boy who blows raspberries in public: don’t point and make a fuss, because you’ll only encourage him to do something similar again.

All very well for you to suggest that the policy’s impact is tiny and we should ignore it – households affected by the benefits cap can’t just pretend it isn’t happening.

That is true. The DWP believes that 40,000 households will see their benefits reduced – and 20,000 households will see benefits reduced by £62 a week or more. So with some misgivings, let’s talk about the merits of the policy. We can think about it morally, practically, politically or not think about it at all. Mr Duncan Smith is firmly in the “don’t think about it at all” school of thought. He has made the statistically unsupportable claim that the threat of the benefits cap has pushed people back into work. He has been censured for this by Sir Andrew Dilnot, who chairs the UK Statistics Authority, and when challenged about it he responded that “I have a belief that I’m right”. It’s the Tinker Bell school of public policy: if you believe in fairies, the fairies will live.

If we vetoed any policy just because a foolish or duplicitous minister was attached to it, nothing would ever happen. Ignore Mr Duncan Smith: what about the policy itself?

There’s obvious moral appeal to the idea that people should get more money if they work than if they don’t. But this policy appeals to that moral intuition only in a crude way. What, for instance, is the moral significance of average wages? By definition – since the DWP tells me this is the median average – half of working households earn less than that. Then there’s the regional issue: the cap is national, but whether you look at the cost of living or the average wage, £26,000 looks a lot less generous in London than it does on Tyneside. In any case, working households earning £26,000 after tax enjoy plenty of top-up benefits. The figure of £26,000 sounds plausible but it could scarcely be more arbitrary.

But even if the ethical case is a bit vague, perhaps there is a practical case. Mr Duncan Smith believes that the cap will encourage people to find work.

That is plausible, although if Mr Duncan Smith had convincing evidence to support his belief he wouldn’t need to abuse the data he does have. But “will the benefit cap encourage some people to find work?” is not a very good question to ask. A well-designed benefits system looks at incentives in more detail. The issue isn’t whether a family on benefits is making more money than some entirely irrelevant benchmark. It’s what the effective marginal tax rate is as benefits are withdrawn when a household member gets a full or part-time job at a low wage. Mr Duncan Smith also seems uninterested in the cost of withdrawing benefits – both for the innocent children whose parents are claimants, and for the administrators who must figure out the details. Thanks to a leak, we know that Eric Pickles, the communities secretary, thinks the policy may leave the taxpayer worse off because of the administrative fiddliness and the cost of temporary housing for the suddenly homeless.

But it may still work.

Whether it works or not seems to be irrelevant. If the DWP cared about that, it would have run a randomised controlled trial of the policy. Instead it ran a pilot with the aim of ironing out the kinks. Why bother finding out whether the policy will work? As a DWP spokesperson informed me, “the policy is already decided”.

Also published at ft.com.

Pop Up Ideas with Gillian Tett and Malcolm Gladwell

Gillian Tett’s excellent talk for “Pop Up Ideas” is now available to download (or you can subscribe to the podcast). You can also download Malcolm Gladwell’s superb talk from last week.

16th of July, 2013MarginaliaRadioComments off

Popular perceptions exposed by numbers

What the public believes can be very far from reality, writes Tim Harford

‘A new survey by Ipsos Mori for the Royal Statistical Society and King’s College London highlights how wrong the British public can be . . . ’ Press release, July 9

That seems a bit harsh.

In fairness to the British public, the claim isn’t that we are always wrong. It is that when we are wrong we are very wrong indeed.

How wrong?

By a factor of 10, 15 – or more. Ipsos Mori was asked by the RSS to find statistical howlers and the market research company duly produced a list of its 10 favourite examples. These emerged from surveying more than 1,000 people, and focusing on areas Ipsos Mori had reason to suspect give us trouble.

So the pollsters set the Great British Public up for a fall?

No, they highlighted known weaknesses. For instance, everyone who has been paying attention knows two things about crime: it’s falling, and most people don’t know that it’s falling. Crime is less than half what it was in 1995, and a fair bit lower than in 2007. Violent crime is falling, too. But when surveyed, 58 per cent of people didn’t believe crime was down. This is an extraordinary mass hallucination.

Perhaps the statistics are wrong. You know what they say – damned lies and all that.

All statistics have their limits but I would be truly flabbergasted if some vast statistical conspiracy (or cock-up) had hidden the fact that crime has been rising for years after all. The statistics on immigration are probably more questionable – after all, some immigrants are illegal and so likely to resist being counted. The official view is that 13 per cent of the British population was born overseas; attempts to allow for uncounted illegal immigrants put that figure as high as 15 per cent. But take a survey and people reckon the true figure is 31 per cent.

Do they not know about the official figures, or do they not believe them?

Both. People don’t know about the official figures but when Ipsos Mori enlightened them, many were sceptical. And this matters. Ipsos Mori also found that three-quarters of people felt that too many immigrants were coming to live in the UK. Joe Public has spoken, and he says: “In my fevered imagination, there are too many immigrants around.” Sort that out, politicians.

Ah, well. Do politicians encourage these errors or pander to them?

A chicken and egg question, I fear. Some misperceptions are very convenient for the government. For instance, presented with a list of major spending categories – the National Health Service, defence, pensions and so on – people will tend to identify “interest payments on the national debt” as the biggest item. In fact it’s well down the list, less than half the cost of the NHS. And a substantial minority of people think Jobseeker’s Allowance costs the exchequer more than state pensions do. In reality, pensions cost 15 times more than JSA. People think that capping benefits at £26,000 a household will save a lot of money; it won’t. These misperceptions do play into George Osborne’s political strategy.

Come on, then – what’s the biggest error of all?

It’s probably not the most important error, but in sheer orders of magnitude, our view of underage pregnancy is pretty wrong. We think 15 per cent of girls under the age of 16 get pregnant each year, a figure that strains the limits of biology. The official statistic is that 0.7 per cent of girls aged 13-15 have either a live birth or a legal abortion. We have a grim view of some corners of British life, and fortunately that grim view is utterly wrong. One bright spot: people generally believe that their own area is closer to the way they like it with lower crime, lower unemployment, better policing, fewer immigrants. It’s the rest of the country they worry about.

That’s all very nice – but it doesn’t alter the fact that the British public are about as wrong as anybody can be about some basic facts.

You think so? You obviously haven’t heard of quantitative finance. The chief financial officer of Goldman Sachs commented, at the beginning of the financial crisis, that the investment bank was seeing “25 standard deviation moves, several days in a row”. It’s not totally clear what he had in mind but given some reasonable assumptions, it simply meant that Goldman’s mathematical models were wrong by, oh, 215 or 216 orders of magnitude – that is wrong by a factor of a trillion trillion trillion etc . . . with a total of 18 “trillions”. “Perceptions are not reality,” says the RSS. Well, quite.

Also published at ft.com.

How to give money away

Helping the poor in the most obvious way of all, through direct cash transfer, is starting to look attractive

What should the comfortably off of the world do to help the poor? We could build dams, railways and roads. We could fund education or public health. We could lower our tariffs and let them sell things to us. We could advise them on appropriate economic policies. (I know, I know.) Or we could leave them alone, on the grounds that our help may be doing more harm than good.

But here’s an alternative: why don’t we just give them money? For such a forehead-slappingly obvious plan, it’s received relatively little attention until the past few years. Until recently it has been hard to reach the very poor directly. Making sure that welfare payments go to all and only the right people is hard enough in the UK – it stands to reason that it will be a lot more difficult in Afghanistan.

So far we’ve settled for giving the money to the governments of poor countries instead, often with strings attached aimed at making sure the money is spent on good works. But those strings can be snipped. You may think you’re paying for a hospital, but if the health minister simply uses your aid money to build a hospital he would have xanax built anyway, and steals the money from his own budget, you’re really just paying for his luxury apartment in Monaco.

Yet now we can simply stuff cash into the virtual pockets of the poor. Many very poor people have mobile phones, or access to phones. The government of India has embarked on a vast project to give everyone an ID number. GiveDirectly is a charity that passes donations straight to poor families in Kenya, using the well-established phone-banking system, M-Pesa.

But are direct cash transfers desirable? They may well be. The very deprivation that seems to make direct cash transfers challenging may actually simplify matters. To take the example of, say, Malawi: more than eight in 10 people there earn less than two dollars a day, even after adjusting for the lower cost of living in Malawi. If you give $50 to everyone in the country, and a few rich people are among those who benefit, what harm? And while reasonable people can argue about why poor people in Bristol or Barnsley are poor, there’s no doubt about why poor people in Malawi are poor: it’s because they live in Malawi. Given an extra dollar, the chance that they find some good use for the money seems high.

A number of studies have found that when poor entrepreneurs are given cash grants, they manage to achieve a high rate of return – a typical figure being 40 to 80 per cent a year. A new randomised trial has now been carried out in Uganda, which goes even further, giving cash to young rural people with no particular trade at all.

The study, now a working paper by Chris Blattman, Nathan Fiala and Sebastian Martinez, examined what happened when the Ugandan government handed out $10,000 to groups of young people, selected randomly. Per person, these grants were around twice the annual income of the young people in question. Blattman and his colleagues then looked at what happened over the subsequent four years, compared with the control group. Did they just spend the cash, or did they find a way to invest it?

The results were encouraging: these young people often set themselves up in a new, skilled trade such as carpentry or hairdressing. Earnings rose sharply. The returns were particularly high for young women relative to their cash-constrained comparison group. And as Blattman points out on his blog, this isn’t just about helping a few poor people: it’s about funding a process of industrial transformation, shifting from rural labouring to skilled crafts, one grant recipient at a time. Helping the poor in the most obvious way of all is starting to look attractive.

Also published at ft.com.

Cory Doctorow has Lunch with the FT

Illustration by James Ferguson of Cory Doctorow

Portrait by James Ferguson

Cory Doctorow should be too busy for lunch. He’s co-editor of, and a prolific contributor to, one of the most influential blogs in the world, Boing Boing. Over the past decade the Canadian-born writer has published 16 books, mostly science fiction novels. He campaigns vigorously on the politics of the digital age. His speaking schedule for the two months following our lunch requires three return trips from his home in east London to North America. He has almost 300,000 followers on Twitter. He is an impeccably prompt email correspondent.
More remarkable to me than any of this is that he claims to prepare for himself, his wife and five-year-old daughter “a three-to-four-course, hot/cold tailor-made breakfast every morning, in 20 minutes flat, with handmade coffees”. And although I arrive at Hawksmoor 10 minutes early, he’s there already, sipping sparkling water at the bar and reading a book. He’s wearing thick-rimmed spectacles worthy of Eric Morecambe, a Disney “Haunted Mansion” T-shirt, and a jacket; he’s 41 but looks younger. Did I mention that I have a tiny crush on Cory Doctorow?

As we’re shown to our table at the window, I feel compelled to ask about the breakfasts. How does he do it? After reading about this quotidian feat of fatherhood, I had tried to make my wife a fancy breakfast in bed, with eggs and honey-drizzled yoghurt and other trimmings. It took long enough that well before I was finished she had surfaced to investigate what was going on. I ask Doctorow for tips, confessing that I can scarcely produce a gourmet breakfast for my family once a fortnight.

“That’s your problem,” he says, in a bright, brisk Toronto accent. “You don’t do it often enough. If you did it every day, you’d get very good at it. It would become a habit, and habits are free.” His own breakfasts are prepared the night before – porridge measured out, yoghurt and berries mixed, served and in the fridge, eggs in the saucepan ready to be boiled. It’s obsessive, precise, carefully optimised – and, it seems, highly effective.
The waiter arrives to discuss steak with us. Hawksmoor is a hipsterish steakhouse near Spitalfields market, all dark wood and brick. The menu is unconventional, with steaks priced by the gramme and particular weights of pre-cut steak chalked up on the board. Doctorow – who had sent me a list of places he’d be happy to eat – seems to be a regular, and quizzes the waiter about when the meat is delivered and why, when he comes in the evenings, certain cuts have already been crossed off. The waiter assures us that the meat is delivered fresh every day and never frozen, even though it’s harder to carve the steak in an unfrozen state.
“Unless you have a laser,” offers Doctorow, at which point the waiter, rather curiously, begins to discuss whether certain cuts cooked rare present a risk of food poisoning.
“We’re not really supposed to talk about food poisoning,” admits the waiter. “You’ve got to come up with another name for food poisoning,” suggests Doctorow. “Like, er, ‘exotic gut flora experience’.”
I explain to Doctorow that he can choose whatever he likes and the FT will pay, but the world gets to see the bill. “That’s a very funny little bit of behavioural economics,” he replies.
Some of the steaks are sized for two, and I indicate that I’m willing to share one. Doctorow selects a large porterhouse for us, and we’re persuaded for reasons of flavour rather than safety to go for medium rare rather than rare. Doctorow chooses bone marrow and onion to start, with creamed spinach to accompany the steak. I start with a Caesar salad, and order triple-cooked chips. I press him to order some wine and he reluctantly agrees to drink half a glass, but refuses to choose.
I express surprise that he claims to know nothing about wine although he is obsessive about, for example, coffee. “I specialise,” he explains, adding that he rarely drinks much. I choose two glasses of the cheapest red. It’s not bad, and he downs his swiftly.
. . .
Doctorow’s fiction champions technology, while warning of how easily it can be used by repressive states or corporations. His own life provides an example of how to live with freedom in a technological age – he’s a man with no particular title, no hierarchical authority, no corner office and no secretary, who somehow manages to keep the plates spinning. Is it the same relentless, nerdy optimisation that gets those breakfasts on the table? He quotes from Brian Eno’s collection of not-quite-aphorisms Oblique Strategies , “Be the first to not do what nobody has ever thought of not doing before.”
Boing Boing, a marvellously eclectic blog, is a case in point – it’s a stripped-down vision of a 21st-century media outlet. Founded in 1988 as a print magazine, it went digital in the mid-1990s, and became one of the first blogs to attract a mass audience. Doctorow started writing for it as a favour for the blog’s founder Mark Frauenfelder, who was going on holiday, and never stopped. It’s incorporated as a business and is funded by sponsors, advertising and merchandise. It has a wide reach and yet, by the standards of a newspaper, is produced by a tiny team, with four main writers, three of whom live in California.
“We are spectacularly lean,” says Doctorow. “We have one phone call a year if we need it. We have one meeting a year.” He’s saving on air fare by tacking this year’s editorial meeting – in Los Angeles – on to a pre-existing trip. It’s usually at the Magic Castle, a private club for magicians. This is a typical touch of whimsy; Doctorow is also seriously smitten by Disney theme parks (his first novel, Down and Out in the Magic Kingdom (2003), imagines what Walt Disney World might be in the 22nd century).
Is this “spectacularly lean” operation the future of newspaper publishing, I ask? “It’s a future of publishing. One of the things that newspapers obscured was that they weren’t a medium, they were a collection of media bodged together.” Newspapers are like books, he says, a format that encompasses “anything from actuarial tables to Mein Kampf”.
But I haven’t quite tired of the topic of getting things done. Doctorow says he’s published 16 books in the past decade. How?
“I figure out how much time I have to write a book. I figure out how many words I need to write. I convert that into a daily rate and I write that many words every day come hell or high water.” Before I can raise the question of quality, he goes on to explain that there’s very little correlation with what he thinks is good writing while he’s at the keyboard, and what later turns out to be good writing – and so he might as well just get the words down and sort it all out later. Lest that process sound like pure hackwork, Doctorow novels have won or been nominated for most of the science-fiction awards that count.
The “write it now and fix it later” approach sounds perfectly reasonable to me, but then Doctorow pushes it to an extreme. “For instance, I wrote Homeland [2013] while I was touring Germany to publicise Little Brother [2008]. I had a translator, we’d visit lots of schools, and so I’d be speaking English half the time and he’d be speaking German half the time, and I’d write the book while he was speaking German.”
I point out that he is describing a superpower. Didn’t people wonder what he was doing as he sat in front of an audience tapping away on his computer while his translator spoke?
“Yes, but that was fine. At the end of the talk someone would say,” – and Doctorow assumes a gentle German accent – “‘Herr Doctorow, what are you doing with your computer on the stage?’ and I’d explain that I was writing my next book. They’d love that.”
Science fiction is often a way of exploring issues of contemporary relevance, and Doctorow’s work is no exception. In For the Win (2010), a novel aimed at the “young adult” market, he describes a battle between internationally mobile capital and the attempts of the trade union movement to mobilise “virtual sweatshop” workers across international boundaries. The action moves between India, where anything goes in a deregulated environment, and China, where the state is powerful but allied with the corporations in suppressing workers’ rights. The book manages to explore some complex economics in the context of a well-paced thriller.
Doctorow is clearly fascinated by economic issues, and points out that most science fiction and fantasy economies make no logical sense. The exception, he declares, is when Marxists write science fiction or fantasy. Take the recent Hobbit movie, for example. “How can the goblins have a mine that’s so inefficient?” he laughs, as he pauses from ripping the soft flesh from the marrowbones on his plate with his bare hands.
The porterhouse steak arrives, pre-sliced. It’s very good, charred on the outside but soft and pink beneath the surface. Doctorow has asked for horseradish while I am dipping my steak and chips into béarnaise sauce. The conversation is animated enough to slow our progress, and neither of us raises an eyebrow when a waiter noisily drops something fragile on the other side of the dining room.
So, I ask, if only Marxists get economics right in their novels, does that make Doctorow a Marxist? There’s a tension there, somehow – he’s a successful player in the market economy and fluently speaks the language of business; of profit, marketing reach, margins, and price discrimination. But his political activism seems squarely on the left – pro-labour, pro-equality, pro-rights.
“Marxists and capitalists agree on one thing: they agree that the economy is important. Once we’ve agreed on that we’re arguing over the details,” he says. But no, he’s not a Marxist. “I always missed the explanation of how the state is supposed to wither away.” In his novels and his blogging, the ruthless abuse of state power is just as much of a theme as the grasping amorality of large corporations.
Before long we’re talking about automation, and whether the rise of robots and algorithms is a threat to middle-class jobs. Doctorow’s next book will explore that territory in a suitably dystopian form, and he is keen to pick my brains about how things might play out. We discuss possible scenarios and I recommend an essay by John Maynard Keynes, “Economic Possibilities for our Grandchildren”. (Within hours he’s found it, read it and tweeted a recommendation.)
. . .
We’ve been making sufficiently slow progress through the meal that both of us have room for dessert. In fact, Doctorow effectively orders two – a crumble with cornflake ice-cream on the side. I order peanut butter caramel shortbread. After we both ask for double espresso, he pulls out a small plastic bottle that once contained mineral water. It’s half-full of a pale brown liquid. “I nearly forgot. I brought you some cold brew coffee.” I sniff at the concoction, the product of Doctorow’s latest coffee experiments. It’s made by steeping coffee in cold water overnight, and it smells sweet. When I try it later, the taste is mild but the caffeine jolt is fierce.
As we wait for dessert, I ask him about his recent speeches at technology conferences discussing the “war on general purpose computing”. He runs through the argument with practised fluency. Computers are by nature general-purpose machines. It’s impossible to make a computer that does all the kinds of things we want computers to do yet is somehow disabled from making copies of copyrighted material, or viewing child pornography, or sending instructions to a 3D printer to produce a gun.
“Oh my God, that’s good,” says Doctorow after his first mouthful of crumble. My peanut butter shortbread is fantastic too, if absurdly calorific. We are interrupted only by another waiter dropping a tray of glasses.
He continues with the argument. The impossibility of making limited-purpose computers won’t stop governments or corporations trying to put on the locks, or changing laws to try to make those locks effective. But the only way these limits can possibly work is subterfuge: computers therefore tend to contain concealed software that spies on what their users are trying to do. Such software is inevitably open to abuse and has often been abused in the past.
Digital rights management systems intended to prevent copying have been hijacked by virus-writers. In one notorious case, the Federal Trade Commission acted against seven computer rental companies and the software company that supplied them, alleging that the rental companies could activate hidden software to grab passwords, bank account details and even switch on the webcam to take photos of what the FTC coyly calls “intimate activities at home”. As computers surround us – in our cars, our homes, our pacemakers – Doctorow is determined to make people realise what’s at stake.
We polish off our coffee and desserts, and the conversation rolls on, covering digital media strategy, the future of book publishing, and Rupert Murdoch’s chances of keeping control of News Corp. I ask him about the FT’s business model. He approves of the use of the standard web language HTML5 in the FT app, which makes it less dependent on Apple or any single tablet format. “That’s a good idea,” he says. Then again, he adds, “selling a product that is well-liked by people who are price-insensitive is never a bad thing.”
We’ve been in the restaurant for three hours and are the only customers left. The staff wipe the tables around us and patiently bring flasks of tap water without being prompted. Yet another glass breaks, somewhere on the edge of my vision. “It’s not a good day for gravity,” says Doctorow.
I feel embarrassed that I’m the one who has to call things to a halt, but I’m going to be late for my next appointment. We admire the size of the bill, shake hands, and Doctorow heads off to the pool for a long swim.
That evening, I send him an email. His response is immediate.

157 Commercial Street
London E1 6BJ
Bone marrow & onions £7.00
Doddington Caesar £7.50
Porterhouse 900g £76.50
Triple-cooked chips £4.00
Creamed spinach £4.50
Béarnaise £3.00
Apple crumble £6.75
Peanut butter shortbread £7.00
Cornflake ice-cream £3.00
Double espresso x2 £6.00
Sparkling water x2 £7.00
Moulin de Gassac x2 glasses £12.00
Total (incl service) £162.28

First published in the Financial Times, 13 July 2013

Here is a talk I gave on data visualisation

I gave this talk at the University of Cambridge on 22 May, as part of the International Year of Statistics, and I am introduced by the wonderful Professor David Spiegelhalter.

Enjoy! http://view6.workcast.net/?pak=8615439470431294

Pop-Up Ideas with Malcolm Gladwell, Gillian Tett, and David Kilcullen

We’re following “Pop-Up Economics” by inviting guest speakers to tell us about ideas from anthropology and political science – all wrapped up in stories to remember. We recorded these short (13 minute) talks in front of a live audience in the foyer of the Royal Festival Hall, and I think they sounded terrific. I can’t wait to hear the program itself.

The show is being broadcast on Radio 4 at 9.30am each Tuesday morning, from today. Our guest speakers are Malcolm Gladwell, author of “The Tipping Point”, anthropologist and financial journalist Gillian Tett, and counterinsurgency expert David Kilcullen. Plus me!

The podcast episodes should appear on the Pop-Up Economics feed.

The need for less speed

High-frequency trading may be about to slow down. Regular investors, looking for fundamental value, would be pleased

Most people think of Wall Street as the home of financial trading in the US, if not the world. But many important contracts – in particular futures and options – are traded in Chicago. Chicago is 1,000 miles away if you follow the wiggles of the railway lines, which is what communications cables have tended to do. And why not? It’s much easier to negotiate permission to lay wires along the tracks, and a message can still get from New York to Chicago and back in 14.5 milliseconds – that’s 70 round-trips a second.

But a 14.5 millisecond trip is just too slow for some. In the summer of 2010, a company called Spread Networks completed a fibre-optic cable connection, having secured rights of way along a more direct route: just 825 miles. According to Wired magazine last year, the result was a 13.1 millisecond return trip. Now, as my colleague Clive Cookson recently reported, over a dozen new networks using microwave transmitters promise to reduce the time for a return connection to nine or perhaps even 8.5 milliseconds. Usain Bolt’s reaction time in the 2012 Olympic 100m final? 165 milliseconds.

“High-frequency trading” is a rich environment of algorithms, of predators and prey, all trying to make money by trading financial products at tremendous speed. But the basic proposition is simple to state. When the price of a share rises in New York, the price of related contracts will rise in Chicago just as soon as the news arrives. But if everyone else gets the news on the regular cable, and you’re renting space on the faster cable, you can see into everyone else’s future by (say) 0.7 milliseconds, plenty of time to buy soon-to-rise assets and then, less than a thousandth of a second later, to sell them again.

You don’t have to be a socialist to find this kind of thing discomfiting. There are three concerns. The first is that scarce resources are being spent on high-speed connections that have no social value in what is at best a zero-sum game. The second is that high-frequency traders may be making money at the expense of fundamental investors. The third problem is that such trading appears to introduce systemic risks. The “flash crash” of May 2010 is still poorly understood, which should ring alarm bells – especially since the need for speed means most high-frequency algorithms are simple and therefore stupid.

What, then, should be done? Rather than trying to slow down the algorithms, why not slow down the market? Most financial exchange markets run continuously, effectively assuming that traders can react instantaneously, withdrawing out-of-date offers and replacing them with up-to-the-picosecond prices. It’s this flawed premise – that all trades could be instantaneous – that means that no matter how fast the computers get, there will always be an incentive to go faster still.

A simple way for an exchange to improve matters would be to run an auction once a second, batching together all the offers to buy and sell that have been submitted during that second. Unsuccessful bids and asks would be published and would remain on the books for the next auction, unless withdrawn. One auction a second ought to be enough for anyone; it would deliver a stream of well-behaved data to regulators – currently unable to figure out what is going on – and it is plenty of time for a computer to weigh its options.

Could this happen? Perhaps. Two teams of economists – Doyne Farmer and Spyros Skouras in Europe, and Eric Budish, Peter Cramton and John Shim in the US – are exploring the concept. Regular investors, looking for fundamental value, would be pleased. Even the exchanges themselves may have something to gain from improving the way their businesses work.

High-frequency trading may be about to slow down.

Also published at ft.com.