Tim Harford The Undercover Economist

Articles published in March, 2010

Business Life: Regrets? I’ve had a few

First published in Business Life, August 2009

Would you feel worse if you made a loss because you sold shares and they later surged in value, or if you didn’t sell them and they later fell? In other words, do you feel worse when things have gone wrong after you’ve taken action, or when things have gone wrong after you’ve been passive?
In surveys, most people reckon that they’d feel more regret after taking action that didn’t work out. (This is in stark contrast to the folksy wisdom that you always regret the things you didn’t do – not on the stock market, you don’t.) But hypothetical survey questions don’t always tell the truth about what people really do when money is on the line.
Economists and psychologists would both like to know how our actions – and our failures to act – are shaped by the desire to avoid feeling regret. That would be much easier to do if researchers could observe people making financially significant decisions in an environment that made it clear when mistakes were being made.
Such an environment exists, of course: the blackjack tables in Las Vegas. Two finance professors, Bruce Ian Carlin and David T. Robinson, recently published their analysis of decision-making at the blackjack table, using data from an in-house security system. (The system tracks cards and betting chips in an attempt to identify odd patterns of play that might signify card-counting or outright cheating.)
Blackjack – sometimes called vingt-et-un or pontoon – is the perfect game for the task. The aim of the game is to draw cards with a higher total face value than the dealer does – but up to a maximum of 21, above which the player goes bust.
Unlike poker or chess, the optimal strategy is fairly simple and well-known. Furthermore, the major choice a player faces is to act, by asking for an extra card, or not to act. Overactive play tends to result in the player going bust; overly passive play is easily beaten by the dealer.
The findings are clear-cut: in an effort to avoid regret, players tend to be too passive, preferring to sit pat on a low total and be overhauled by the dealer than to risk busting twenty-one early. Eighty per cent of deviations from the correct strategy are “passive mistakes” – players failing to take a card when they should have done so. The financial implications were large: players who stuck to the strategy won over $60,000 in total; those who strayed lost a total of over $55,000.
Some puzzles remain – notably, the fact that people often seem to trade too often on the stock market, when this research suggests a bias towards passivity. The research effort will continue – and economists will keep on applying for grants to go to Las Vegas.

28th of March, 2010Other WritingComments off

Should I become a vegetarian?

Dear Economist,
A growing trend among my fellow students is converting to vegetarianism for environmental reasons, citing statistics that meat production, in particular beef, is a tremendous cause of greenhouse gas emissions. I was wondering if you could provide some insight into the actual environmental cost of a steak. How does it compare to driving, or flying? Would a simple tax on beef production be much more efficient than vegetarianism?

Dear Max,

Your friends are right to worry about beef; you are right to ask questions. Let’s start with the beef itself. Cows are ruminants, which means they produce large quantities of methane – contrary to what some people believe, much of this is emitted through the mouth. Methane is a powerful but transient greenhouse gas, so it is not straightforward to compare with carbon dioxide emissions, especially those produced by planes at altitude, which are more damaging.

Using a standard rule of thumb – and the work of “Economical Environmentalist“ Prashant Vaze – I can inform you that a 250g steak is responsible for more than 4kg of CO2-equivalent emissions, before cooking the stuff.

Sheep is just as bad; pork and chicken are half as bad; cod or wheat are at least eight times less carbon-intensive; potatoes and herring are far better still. Organic methods reduce carbon dioxide emissions, but not by much.

Yet this all looks modest relative to the costs of transport. Vaze reckons that steak’s emissions will get you about eight miles by plane, or 15 miles if you travel alone in a car.

As for the tax, it should be on all greenhouse gases, not just cow burps. I suspect it would create a few vegetarians, perhaps better diets for cows, and headaches for the taxman. More importantly, it would inspire some loft insulation.

Also published at ft.com.

27th of March, 2010Dear EconomistComments off

At last the con has been taken out of econometrics

In 1983, Edward Leamer published an article with contents that would become almost as celebrated as its title. “Let’s Take the Con Out of Econometrics” began with an analogy that remains useful. Imagine an agricultural researcher who tests the effectiveness of a new fertiliser by dividing land into strips and spreading the new fertiliser only on a randomly chosen selection of those strips. Because of the randomisation, any effect will presumably be thanks to the fertiliser.

Contrast this scrupulous scientist, continued Leamer, with two agricultural econometricians. One notices that crops grow under trees and, after taking careful measurements, announces that bird droppings increase crop yields; the other has noticed the same phenomenon and declares that it can, with confidence, be credited to the benign effects of shade.

This is the “identification problem” – trying to work out whether a statistical pattern is caused by what we think it has been caused by. It muddies any statistical analysis of data that have not been generated by a controlled experiment, and it particularly plagues econometricians, the statistical wing of the economics profession. But, complained Leamer, throughout the 1970s they too rarely cared, and much of their work was dubious at best. Leamer was not alone. David Hendry showed in 1980 that by using the standard methods of the day, he could demonstrate that rainfall caused inflation. Or was it that inflation caused rainfall?

That was then. Now Joshua Angrist of the Massachusetts Institute of Technology and Jörn-Steffen Pischke of the London School of Economics have published a new working paper arguing that econometrics has undergone a “credibility revolution”. Angrist and Pischke argue that the identification problem is now being faced head on and for many questions it is being solved. Modern econometrics works.

Given the recent financial crisis, I pause for sceptical chuckles, but academic econometrics is rarely used for forecasting. Instead, econometricians set themselves the task of figuring out past relationships. Have charter schools improved educational standards? Did abortion liberalisation reduce crime? What has been the impact of immigration on wages?

More data and more powerful computers explain some of the improvement, but the real progress has come through better techniques. One is the use of an “instrumental variable”, some outside force that partly mimics the effect of a proper randomised trial. Angrist, for instance, looks at lotteries that allow children to go to oversubscribed charter schools. The lotteries are imperfect, because some winners do not take up their places, and some losers manage to win places somehow anyway. But allowances can be made for that.

Another technique is to look for sudden jumps in a variable that are hopefully nothing to do with the matter at hand. For instance, struggling children are often put into small classes, making it hard to work out whether small classes help children learn. But Israel has a law that no class can exceed 40 children, so a school year with 39 children will have one large class while a year with 41 children will have two smaller classes. If, as seems likely, the smaller class sizes here have nothing to do with the raw ability of the children, this serves as a good quasi-experiment.

Whether laymen will be persuaded by these sophisticated – often incomprehensibly sophisticated – techniques is not clear. But where experiments are impossible, as they often are, econometricians are at least starting to convince each other. That is probably progress.

Also published at ft.com.

Which is best: chess, romance or A-levels?

Dear Economist,
My son is applying to read medicine at university, and his various options require different combinations of subjects. He would like a place at UCL, with Barts as a fallback.
He feels reasonably confident of attaining an A grade in all four A-levels, but chemistry will be the trickiest. Should he ignore maths, focusing on UCL; or ignore chemistry, which would be fine for Barts?
Also, he does voluntary work in a home for the elderly which, quite honestly, he embarked upon in order to improve his application. In fact he gets a lot out of the voluntary work, including chess and philosophy with retired experts. Should we encourage him to give it up anyway? Should he also give up his musical instrument, sport, an Easter break in Paris and his girlfriend? He doesn’t want to be reductive, but he must meet one of his offers.
Family in Harpenden, Herts

Dear “Family”,

You write collectively but there is a principal agent problem here. Chess, romance and good works count in your son’s eyes, but when you boast to your friends in Harpenden, the hard currency is A grades and university places.

Your risk-management is also amateurish, setting out mechanistic strategies for selecting from an unimaginative set of options: UCL, Barts or oblivion. You talk as if you can shift effort from maths to chemistry with predictable results, but you have no idea what obstacles may lie ahead. Your son might abandon chemistry and then be undone by a tricky maths paper. Or he might find that when he misses a grade, he wins his place anyway because he’s the only applicant who stuck with the voluntary work. Or he may find that he does not care for medicine once he begins to study it. What then?

He may not want to be reductive, but you clearly do. You’re wrong.

Also published at ft.com.

20th of March, 2010Dear EconomistComments off

Tried and tested ways to woo a half-hearted terrorist

In 1973, Yassir Arafat, Chairman of the Palestine Liberation Organisation, had an unusual problem. His elite unit, Black September, had seized international headlines by kidnapping and then murdering Israeli athletes at the 1972 Munich Olympics, along with other high-profile attacks. That had forced the world to take the PLO seriously, but now that Arafat was seeking international recognition, he needed to muzzle his own attack dogs, a hundred ruthless warriors with nothing to live for but the cause.

The solution was simple. The PLO married off most of Black September, offering them Beirut apartments, shapely young brides and a “child bonus” of $5,000 if they had a baby within a year. No longer did these men have nothing to lose.

The story is related in Eli Berman’s new book, Radical, Religious and Violent, an attempt to construct a theory of rational terrorism. Reverse the tale and you get a sense of Berman’s argument: effective terrorist groups are effective only because their members are cut off from the outside world and have little to gain from quitting the group.

For Berman – a Canadian-born economist and professor at the University of California San Diego, who also served in Israel’s Golani Brigade in the early 1980s – thinking about quitters is essential to understanding what might sound like an odd question: why are there so few effective terrorist networks?

Of course, the world is not short of terrorists, but there are many grievances, many disaffected young men and hundreds of thousands of murders or deaths on the battlefield. Given what an impact terrorist violence can have, and how low-tech it can be, Berman is probably right to suggest that the rarity of effective terrorists, however welcome, is a puzzle.

Defectors provide the solution to that puzzle. A single defector can jeopardise a terrorist network, and defections do happen. Sudanese militant Jamal al-Fadl quit al-Qaeda in the mid 1990s and jumped ship to the US, reportedly for huge sums of money. Abu Musab al-Zarqawi, the leader of al-Qaeda in Iraq, died when his safe house was bombed in June 2006 – it has been reported that an associate betrayed him for the $25m bounty on his head.

The higher the stakes, the more tempting it will be for a half-hearted terrorist to defect. Berman argues that radical religious groups are well-equipped to ensure that there is no such thing as a half-hearted terrorist. It is not the theology of such groups – martyrdom, for instance – that makes the difference, but their ability to cut off outside options and create very strong ties between group members.

Seen in this way, the Taliban have something in common with the Amish or ultra-orthodox Jews: a set of rules that makes it unattractive for adherents to leave, and attractive for them to stay. If they stay, they enjoy the membership of a group that provides substantial social services to members. If they leave – having been cut off from a non-religious education and isolated from secular members of society – their options will be limited, even if they do run off with a truck full of smuggled goods or a pay-off from the Americans.

Berman’s book is puzzling in some ways. He devotes very little attention to the fact that the violent religious groups he studies – the Taliban, Hamas, Hizbollah and the Mahdi Army – are all Islamic. But the focus on the way some radical religious groups are able to control defection does seem very fruitful. It points to clear solutions, too: give potential terrorists attractive outside options, offer effective social services and try to cut off their sources of funding. Not at all easy, and not altogether new. But in economics just as in counter-terrorism, what is?

Also published at ft.com.

FT Comment: Political ideas need proper testing

Politician in light bulb by Ingram Pinn

Illustration by Ingram Pinn

I don’t recall it myself, but like most babies born in 1973, I apparently slept face down in my cot. This was the standard advice, made famous by Benjamin Spock in 1948. We now know that for many unlucky families, this well-meaning advice was fatal. According to research published in 2005, putting babies to sleep on their fronts has led to about 60,000 cot deaths.

The story is a favourite of Sir Iain Chalmers, a campaigner for better standards of evidence in medicine and beyond. Because it is possible to do so much unwitting harm in medicine, many medical interventions are now subjected to a randomised controlled trial. Austin Bradford Hill performed the first properly controlled clinical trial in 1948, although he had predecessors, including James Lind, who used a randomised trial to show that citrus fruit prevented scurvy. There’s even a controlled trial in the Bible (Daniel 1:8). Such trials have proved the effectiveness of countless treatments, and the dangers of countless others.
It is a shame, then, that there is so little appetite from politicians for the same standards of evidence outside medicine. In fact it is more than a shame – it’s a scandal. While randomised trials are not going to tell us when to raise interest rates or get out of Afghanistan, there are many policies that could and should be tested with properly controlled trials. Is Jamie Oliver right to emphasise healthy school meals? Run a trial. Should young offenders be sent to boot camp, or to meet victims of crime? Run a trial. What can we do to persuade households to use less electricity? Run a trial.
Yet such trials are not common in the US, and downright rare in the UK. There is no financial, ethical or practical excuse for this. Trials are cheap. (Even if they were expensive, solid practical knowledge is well worth paying for.) This is not a question of carrying out dangerously speculative crank experiments, but simply adding the essential ingredient of randomisation to a standard pilot project that would have happened anyway. Randomising is often what distinguishes proper evidence from statistical mush, by removing biases in the setting of experiments – such as running pilots only in the most needy areas.

When the UK government recently introduced the “synthetic phonics” method of teaching young children to read, they were told by Carole Torgerson, an evaluation expert at the University of York, that they could easily bolster the slim evidence base by randomising which schools joined the programme first. They didn’t. (More encouragingly, Ms Torgerson has been commissioned to evaluate maths teaching.)
Some people feel queasy at talk of “experimentation” in the classroom, prison or benefit office – but politicians experiment on us all the time with their latest policy wheezes. We learn little or nothing because the experiments are badly designed.
What is missing is the political demand for tests of what really works. Too many policies on education, welfare and criminal justice are just so much homeopathy: cute-sounding stories about what works leaning more on faith than on evidence. Politicians and civil servants, faced with some fancy new idea, should get into the habit of asking for a proper randomised trial. And we, as citizens, should be equally demanding.
It’s no coincidence that one of the few fields of social policy to feature more than 100 robust trials is the study of how to get voters to turn out in elections. Politicians seem perfectly happy to turn to scientific method if it will get them elected. They are less interested in using it for the good of the people they govern.
It is embarrassing even to have to make the case for randomised trials in social policy. For medical researchers, such trials are just the start. Realising that inconvenient – or just plain boring – trial results are less likely to appear in print, medical journals now refuse to publish trials that were not logged before they started in a register of trials. Such registers ensure embarrassing results cannot be made to disappear. This is vital in medicine, and just as important in social policy.
Trial registers also feed into systematic review bodies such as the Cochrane Collaboration, which is an international offshoot of a National Health Service initiative. In less than two decades, the Cochrane Collaboration has published 4,000 systematic reviews of medical treatments, digging up data from unpublished trials, and providing the information to save many lives. A parallel body for social policy has far fewer trials to evaluate.
The Cochrane Collaboration has become a byword for a fair and comprehensive review of a treatment – the latest word (never the last word) about what works. We need the same shorthand in policy, a quality kite-mark that tells us politicians have actually done some homework before they roll out their latest brainchild. We’ve had FairTrade coffee – what about FairTest policies? Most voters don’t know much about randomisation or trial protocols, but they’ll know when they see the FairTest logo that a policy has had a proper, scientific test to see if it works.
Many social, educational and economic policies are the modern equivalent of Dr Spock’s advice that babies should sleep face down: well-meaning, authoritative – and wrong. No doubt it would be awkward to see the wisdom of experts punctured and the pet policies of politicians discredited on a regular basis. But if politicians really cared about those they represent, they would insist on more randomised trials and more systematic reviews of what works. Honest policy mistakes, quickly reversed, should embarrass nobody. As voters, we should demand more such mistakes.

First published on ft.com

18th of March, 2010HighlightsOther WritingComments off

Should I try to make school fees fairer?

Dear Economist,
I’m a marketing manager at a British private boarding school. Fees start low and increase during a pupil’s education regardless of the fact that costs remain pretty constant before inflation. I want to reflect this “flat” cost and reward loyalty – but how can I without an ugly hike at reception class?

Dear Fairfeea,

You work in marketing, so I’ll explain this slowly. If your fees are currently designed to rise steadily, and you would like them instead to stay flat, and you would also like not to raise initial fees … well, there’s only one answer: slash final-year fees and lose money.

Let’s take a step back. You say that you’d like to reflect costs. Why? Most businesses set prices to attract customers and increase profits. Costs are not directly relevant. What makes you different? And then there’s this bizarre idea of rewarding loyalty. Most companies say they reward loyalty, but it is more traditional to gouge loyal customers while chasing new ones. Don’t fall for your own propaganda.

Your existing pricing scheme works well. Parents will be nervous about whether the school will suit their children, so a cheap price year in reception is sensible. As they grow more confident and the child settles in, they will be increasingly unwilling to switch. That is why rising fees make sense. If parents are dim enough not to notice them, even better.

The only reason to change your current pricing is if you can figure out a way of concealing some of the later fees. Perhaps you could offer overpriced school trips and exam revision classes in later years – or even put a surcharge on the larger sizes of school uniform.

The only reason to reward loyalty is to create it when it does not exist. A private school will have loyal customers without your fancy pricing schemes. You should take full advantage.

Also published at ft.com.

13th of March, 2010Dear EconomistComments off

The auction site that’s pure temptation

A New York Times columnist called it “devilish”. Jeff Atwood, a blogger, called it “about as close to pure, distilled evil in a business plan as I’ve ever seen”. Another online commenter called it an “evil hack of the human mind”. It’s Swoopo, or as The Register put it, “eBay’s (more) evil twin”.

Swoopo runs “pay-to-bid” auctions, where each bid costs you money. As a result, everyone who participates in the auction ends up paying something, and while the winner may get a bargain, all the losing bidders are out of pocket. The auction rules vary, but might specify that each new bid costs 60 cents and that each of those bids raises the price by just 12 cents.

Consider a $500 game console that sells for $240. The $240 winning bid is the 2,000th bid; Swoopo would have earned $240 for the winning bid, plus $1,200 in bid fees, for a $500 console. Sometimes bid increments are just one cent, meaning that even tiny winning bids can leave bidders collectively out of pocket.

Richard Thaler, a behavioural economist who also has a New York Times column, looked at 26 occasions on which Swoopo had simply auctioned $1,000 cheques. The average revenue was $2,452. No wonder the pitchforks are out.

When I heard of Swoopo, the first thing that sprang to my mind was not distilled evil, but an obscure piece of economics called the revenue equivalence theorem. The theorem, developed by Nobel laureate William Vickrey, shows that – given certain assumptions – all auctions can be expected to raise the same amount of cash for the seller. You can hold an open-outcry auction, or a descending-bid auction, or accept sealed bids in envelopes and charge the winner his bid, or the second-highest bid, or the third-highest bid. You can charge an entrance fee, or even insist that every single bidder forfeits his entire bid. The expected revenue, insists Vickrey, will be the same.

To get a sense of why this might be true, imagine an auction where the “evil” rule is that the winner pays twice his bid. Obviously, everyone in such an auction would simply bid half as much as they really intended to pay. The winning bid would be doubled and the final result would be no different. The revenue equivalence theorem is a more general proof of the same idea: “evil” auction rules simply encourage more cautious bidding – they don’t raise any more money.

Still, Vickrey’s theorem makes assumptions, including that of rational bids. Swoopo may attract foolish bids instead. Thaler looked at only 26 auctions, but two new research papers look at tens of thousands.

Three economists from Brigham Young University, Brennan Platt, Joseph Price and Henry Tappen, find that irrational – or perhaps simply risk-loving – bidders seem to concentrate on game consoles. The bids in other auctions look rational and are not especially lucrative.

Ned Augenblick, a doctoral student from Stanford University, has more data and is more pessimistic: he reckons that the typical Swoopo item raises about 50 per cent more revenue than its price on Amazon. This may not be a grotesque profit margin, but it is certainly handsome.

Both studies conclude that many, perhaps most, Swoopo auctions lose money, but that Swoopo makes up for this because some auctions are very profitable indeed. So perhaps Swoopo is a “hack of the human mind”. Augenblick can identify what the hack is, too: it takes advantage of what economists call the sunk cost fallacy. Bidders get sucked in, refusing to quit once they feel committed by having made a few bids.

Pure, distilled evil? I’d sooner slap that label on slot machines. But I won’t be playing Swoopo in a hurry.

Also published at ft.com.

Help! How do I keep my students in class?

Dear Economist,
I teach special education at an urban public high school. Every period, as many as half my students ask to go to the bathroom. I am pretty confident that most of them are just looking for an excuse to leave the classroom, but going to the bathroom is a right and I don’t want to run my class like a prison.
Beside making my content more engaging, how do I allow students to go to the bathroom in a way that’s equitable and limits the numbers outside the classroom in a given period? Three stipulations: grades are off-limits; I also don’t have much money, so I’m reluctant to offer prizes; and I’d like this system to have as little to do with me as possible. I’m a teacher, not a gatekeeper.
Anon, Washington, DC

Dear Anon,

The great Kenneth Arrow is famous for his “impossibility theorem”, which proves there is no satisfactory way to aggregate individual preferences into a group preference. I am tempted to propose my own “bathroom impossibility conjecture” – which is that the requirements you specify cannot all be simultaneously satisfied. (Let’s summarise them: the bathroom remains a right; making lessons interesting is a side-issue; neither monetary nor academic incentives are to be deployed; no input from you is to be required.)

I do have one suggestion, inspired by Roland Fryer, a Harvard professor who has been running policy pilots that offer cash to good performers at school. Since Fryer’s earlier work suggests that peer pressure explains much slacking at school, some pilots group kids into teams which succeed or fail together.

Perhaps you could group the kids into two or four teams and offer a reward – or just praise – to the team with the strongest bladders. It might work. It might also lead to accidents and mockery. I wish you luck.

Also published at ft.com.

6th of March, 2010Dear EconomistComments off

The hidden histories that shape the way we live now

The largest silver mines in the Spanish empire were the Potosí mines, discovered in 1545 in what is now Bolivia. Exploiting the mines was dangerous, and in the late 16th century, the Spanish introduced the mita system of forced labour. Villages near Potosí were obliged to provide one-seventh of their adult male population to work the mines, and the mita system continued until its abolition in 1812.

That is history. This is not: the former mita districts are 25 per cent poorer than apparently identical districts on the other side of a boundary that ceased to mean anything 198 years ago. A long-abolished colonial system has somehow shaped the modern world.

The discovery, by a young economist at MIT named Melissa Dell, is one of many made recently which show that economic development has a long memory. Here is another example, from Daniel Berger, a PhD student at New York University: the 7° 10’ line of latitude that runs through Nigeria is geographically unremarkable and has had no administrative significance for 96 years. Nevertheless, villages just to the north of this line on a map enjoy more competent government than those just to the south.

Economists became interested in the idea that history means something when three influential research papers were published, the first in 1997. Rafael La Porta and colleagues argued that British common law provided better protection for investors than the Roman civil law tradition, and showed that former British colonies seemed to have more advanced financial markets than former French colonies. Stanley Engerman and Kenneth Sokoloff argued that Latin America had underperformed relative to Canada and the US, because it had a climate better suited to growing plantation crops such as sugar, which in turn led to exploitative institutions. And Daron Acemoglu, Simon Johnson and James Robinson showed that the reason tropical diseases are strongly correlated with underdevelopment is less for the obvious reason – that malaria is bad for the economy – and more because such diseases killed large numbers of settlers, who lacked any resistance to them. This encouraged colonial powers to grab gold, ivory and slaves rather than settling the countries and establishing decent institutions. (The Pilgrim Fathers nearly went to Guyana before thinking better of it when they realised their chances of surviving the local diseases were very low.)

Nathan Nunn, an economist at Harvard, has recently summarised the new wave of economic history. He argues that work such as Dell’s and Berger’s helps us understand not just whether history matters, but, thanks to a better handle on the channels of causation, why it matters.

Dell shows that areas outside the mita system were more likely to have large farms; the owners of such haciendas were politically influential and were able to campaign for public goods such as better roads. Berger argues that the 7° 10’ line of latitude in Nigeria is important because different systems of taxation once prevailed on either side of it. To the south, officials relied on customs duties and other taxes on trade through Nigeria’s ports. North of the line, taxes were levied on people – which meant somebody had to arrange a census and keep proper accounts. The difference in bureaucratic capability has persisted for a century.

All this suggests a fatalistic conclusion about economic development: if today’s economic outcomes are influenced by institutions shaped centuries ago, there is reason to be pessimistic that we can do much to help now. That would be going too far, because history is not the only thing that matters. But matter it does.

Also published at ft.com.


  • 1 Twitter
  • 2 Flickr
  • 3 RSS
  • 4 YouTube
  • 5 Podcasts
  • 6 Facebook


  • Messy
  • The Undercover Economist Strikes Back
  • Adapt
  • Dear Undercover Economist
  • The Logic of Life
  • The Undercover Economist

Tim’s Tweets

Search by Keyword

Free Email Updates

Enter your email address to receive notifications of new articles by email (you can unsubscribe at any time).

Join 3,961 other subscribers

Do NOT follow this link or you will be banned from the site!