Big organisations should test out their new policies whenever they can

An awkward moment recently: I ordered an espresso from Starbucks and the barista, a young fellow with fashionably chaotic blond hair, asked my name. I’d heard that this is the new policy at Starbucks but, not being a regular, I’d forgotten. None of your business, I thought, and fumbling for something to say instead of my name, I said, “I suppose you’re getting annoyed having to ask people their names.”

The young man’s face darkened perceptibly. “A lot of things annoy me,” he said, “but if you don’t want to tell me your name, that’s fine.” His colleague proceeded to pull me a deeply uninspiring espresso, which I felt that I rather deserved.

I took the coffee and sat awkwardly in the corner, avoiding eye contact with the staff and vowing to steer clear of Starbucks in future. One bad espresso just isn’t worth the social discomfort.

If my Starbucks experience is typical, the policy of requesting names is going to prove very ill-judged. But perhaps my Starbucks experience isn’t typical; I’m not a regular customer, after all. Perhaps the regulars love it. Who can say? But it’s worth asking: where does this kind of idea come from in a large organisation? How is it tested? Under what circumstances might it be reversed?

The question of the U-turn is a particularly vexed one. Politicians find it especially painful, perhaps because lazy journalists find U-turns easy to criticise: either the old policy was wrong or the new one is wrong, and either way, the politician can be blamed with no need for further investigation. Just think of the plight of Theresa May, the Home Secretary: she demanded tight border controls, but lacked the personnel to carry out the new regime efficiently. She has painted herself quite methodically into a policy corner.

Any high-profile policy runs a similar risk: if it doesn’t work, it is hard to perform an elegant about-face. This is why I think Starbucks should have conducted a randomised trial to test the question: pick 100 branches, then randomly select half of them to receive instructions and training videos, and see whether there was any effect on staff morale, customer satisfaction or sales.

It wouldn’t have been a perfect double-blind trial, but it would have been revealing. I can confidently assert that if we were talking about Amazon, it would be inconceivable that the company would change how it interacted with a customer without testing the idea with such a trial.

As it happens, Starbucks wasn’t quite as clueless about this as I might have guessed. (I suppose I shouldn’t be surprised that these people know something about selling coffee.) I am told the idea emerged after listening to customers in focus groups, who pointed out that they liked the fact that in their local branch, the staff knew who they were. (I think the Starbucks press officer telling me about the focus groups was about to say that the customers didn’t mention the coffee as a reason to go to Starbucks, but thought better of it. Perhaps I imagined that.)

Next came informal testing: staff at some Starbucks branches – for instance, in Cambridge and in the new Westfield shopping centre in east London – had already been doing this for a few months. An internal “training” video shows these staff enthusing about the idea and entertains no possibility of awkwardness.

Perhaps my cynicism is misplaced. Starbucks is trying to keep regular customers happy; there is no reason to expect sceptics to like it any more than we should expect atheists to be impressed by a religious sermon. Intuition can be misleading in such matters – all the more reason why big organisations should test out their new policies whenever they can.

Also published at ft.com.

An economist who was taken prisoner during the second world war observed that market institutions were universal and spontaneous

Robert A. Radford had, in some ways, a perfectly conventional career as an economist. He studied the subject at Cambridge in the late 1930s, before war interrupted, and his civilian working life was spent at the International Monetary Fund. But he also spent half the war in a German prison camp, and on his release wrote an article in the LSE journal Economica.

The “Economic Organisation of a P.O.W. Camp” is a remarkable piece of writing, in which Radford analyses the economic institutions that arose in tough circumstances. Students should read it to learn about monetary economics, and their professors should read it to learn how to write. But Radford himself thought his experiences constituted more than a teachable moment: “the principal significance is sociological.”

First, a word about the basic economic building blocks. Prisoners received some rations from the Germans, but were mostly sustained by parcels of food and cigarettes from the Red Cross. The parcels were standardised – everyone got the same. Occasionally the Red Cross received bumper supplies, or ran short; in those instances everybody enjoyed a surplus or a shortage.

Radford’s first sociological observation was that there was no gift economy in the camp. Everybody started with the same, so what was the point? But trading quickly developed, because while prisoners had equal means they did not have identical preferences – the Sikhs sold their beef rations, the French were desperate for coffee. So middlemen who could speak Urdu or bribe a guard to let them visit the French quarters had the chance to make “small fortunes” in biscuits or cigarettes. In rare circumstances, the camp’s economy interacted with the outside world: coffee rations apparently went “over the wire” and traded at high prices in black market cafés in Munich.

Market institutions, Radford concluded, were universal and spontaneous, “a response to immediate needs” rather than an attempt to imitate civilian life. One of the spontaneous developments was the emergence of a currency: the cigarette, which was portable and reasonably homogenous. Not entirely so, though: cigarettes could be “sweated” by rolling them back and forth between the fingers to shake a little tobacco out. Gresham’s Law – “bad money drives out good” – asserted itself, as the plumper cigarettes were reserved for smoking, while those that circulated as money grew thinner. When Red Cross supplies were interrupted, deflation set in, as a cigarette bought ever more goods.

The law of one price also tended to hold: arbitrage meant prices rarely varied much within a permanent camp. The chaos of transit camps, however, created profit opportunities. “Stories circulated of a padre who started off round the camp with a tin of cheese and five cigarettes and returned to his bed with a complete parcel in addition to his original cheese and cigarettes; the market was not yet perfect.”

Relative prices moved in response to broader developments – such as an influx of new, hungry POWs – and from day to day. With bread rations handed out on Monday, on Sunday evening “bread now” traded at a premium to “bread Monday”. And yes, there was a futures market.

All this mattered greatly. “The small scale of the transactions and the simple expression of comfort and wants in terms of cigarettes and jam, razor blades and writing paper, make the urgency of those needs difficult to appreciate, even by an ex-prisoner of some three months’ standing,” wrote Radford. His article was written in summer 1945, looking back at March and April, where market prices twitched wildly amid rumour and scarcity. On April 12, the camp was liberated, and, says Radford, “every want could be satisfied without effort.” It is quite a parting thought.

Also published at ft.com.

The NTSB is capable of providing a clear and authoritative narrative, explanations and conclusions about the crisis

After a financial train wreck, it’s time to begin to learn lessons from the disaster and prevent its recurrence.

This is not an easy problem: Andrew Lo, a professor of finance at MIT, recently compared the financial crisis to Kurosawa’s Rashomon, a film in which each character has a different story about an alleged rape and murder. We have no agreed narrative about what has happened in the crisis, which makes it hard to figure out how to prevent it happening next time. The details are picked over in the press, by think-tanks, by commissions and, of course, by ideologues of all stripes.

Professor Lo, along with Jian Helen Yang and Eric Fielding, has proposed an alternative approach to the whole affair in the Journal of Investment Management. Lo, Yang and Fielding argue that there is an organisation that specialises in establishing a clear narrative, ruling out alternative explanations, and drawing conclusions which have real authority and influence. It’s not the Securities and Exchange Commission, the Financial Services Authority, or the G20’s Financial Stability Board. It’s the National Transportation Safety Board, the NTSB.

For more than 70 years, the NTSB – or its predecessor, the Civil Aeronautics Board – has investigated plane crashes, bridge collapses and other transport-related accidents in the US. Why do Lo, Yang and Fielding (who is an NTSB official) believe it is worth emulating?

Two attractive attributes stand out. The first seems paradoxical: the NTSB is not a regulator and has no regulatory authority. “At first I thought this undermined its effectiveness,” Lo told me. “But now I see it makes it more effective. If you are a regulator, how can you criticise your own regulations, for instance?” Quite so: in a crisis where much of the debate centres on whether regulations were too lax or perverse, and whether regulatory authorities such as the Federal Reserve and the Bank of England were asleep at the wheel, a non-regulatory investigator has something going for it.

The second admirable feature of the NTSB is its approach to investigations. First, it holds open hearings to establish a set of objective facts. As each party tries to exonerate itself with evidence about what happened, the facts tend to mount up. This initial focus on facts, says Lo, is an important discipline. Then the NTSB goes into a huddle and tries to settle on a consistent, fact-based narrative; its accounts are rarely challenged.

This all sounds very impressive. Will we get a Capital Markets Safety Board? In the US, perhaps we will: the Dodd-Frank Act established an Office of Financial Research, which has a mandate to gather data and produce or enable better analysis of the financial system. It may develop into something like the NTSB.

The other question, of course, is would an NTSB for finance actually help? There are three obvious differences between transport accidents and financial ones. The first is that what constitutes a financial accident is vague: would an NTSB for finance have studied the collapse of Lehman Brothers? The fraud at Enron? Or vaguer topics such as the dotcom bubble or the sub-prime mortgage industry? (For Lo, the answer is clear: it would study collapses of major financial firms.)

The second difference is that a financial accident is more complex than a physical one: there are more actors involved and far more variables. “It would be way too complicated to reconstruct the cockpit Dick Fuld was in,” says Lo, referring to the last boss of Lehman Brothers. True – but still worth a try.

The final difference might cause the biggest headache. Nobody actually wants to cause a plane to crash or a bridge to collapse; different people have different priorities, but nobody profits from a transport accident. When it comes to finance, that simply isn’t true.

Also published at ft.com.

Be careful whose interests the expert is serving…

Don’t try this at home: seek professional advice. A sound maxim, one might think. Unless you know something about electrical wiring, get an electrician in; go to a doctor rather than diagnosing yourself. And, since you are a fallible investor, choose your financial portfolio with the help of an investment adviser.

But do typical investment advisers give good investment advice? Three economists recently published research exploring this question in an intriguing fashion – they mixed the experimental methods of a double-blind controlled trial with the latest behavioural economics, all seasoned with a dash of tabloid sting.

The researchers recruited professional mystery shoppers and asked them to seek advice from a variety of financial advisers in Massachusetts. Each mystery shopper was equipped with one of four imaginary portfolios.

The first portfolio exhibited the classic investor bias of buying whatever did well last year. This is likely to destroy value partly because it leads to excessive trading, which is expensive, and partly because, as the small print says, past performance is no guarantee of future results. Mystery shoppers with this portfolio were instructed to ask the financial adviser to recommend further hot stocks.

The second portfolio demonstrated a different bias: 30 per cent of its value was invested in the imaginary investor’s employer. This is common, but unwise: it reduces diversification for no good reason, and exposes people to the risk that they lose both job and savings if the employer seeks bankruptcy.

These portfolios were artfully designed, because if a financial adviser is paid on commission – and everyone in this experiment was – he will have an incentive to exaggerate the first bias and to discourage the second one, in both cases to create the opportunity for earning a slice of the resulting trades.

The third portfolio was close to perfect, at least from the point of view of the economists, Sendhil Mullainathan, Markus Noeth and Antoinette Schoar. It was diversified and packed full of US bonds and low-cost index funds invested in US equities. Good investment advice would be not to touch it; at a push, a financial adviser might suggest purchasing some index funds invested outside the US.

The final portfolio was the control: a blank slate, invested entirely in cash. The mystery shoppers holding this were told simply to ask for advice, and to state they were willing to take more risks if necessary to earn a higher return.

What I love about this experiment is the way it was done. It was double-blind: not only did the financial advisers not know what was going on, but the mystery shoppers weren’t told why they had been given a particular portfolio, nor that there were three other “treatments” doing the rounds.

Nor did the researchers know which advisers had been visited by which mystery shoppers – the logistics were outsourced to an audit firm.

And the results? Advisers made flattering remarks about the clients’ portfolios and then proceeded to try to change them in exactly the way that would tend to generate commissions. The gap between flattery and action was particularly stark in the case of the diversified low-cost portfolio; advisers were more likely to praise it, but more than 85 per cent of them tried to change the strategy.

Almost a third of advisers refused to give any advice at all until the client agreed to transfer control of their portfolio to the adviser, which makes it almost impossible to rate the quality of the advice. And a curiosity: this behaviour was substantially more common when the mystery shopper was a woman.

My financial advice? If you’re looking for investment advice, be careful whose interests your adviser is serving. And if you’re a financial adviser, beware economists bearing randomised trials.

Also published at ft.com.

Collaborative consumption websites and microlabour services may lower transaction costs but they raise the issue of internet-based trust

Loyal readers will recall the second-hand Volvo I bought six years ago. After an unpromising start, it’s now ticking over nicely. Well, ticking over may be the wrong phrase. Mostly it just sits around. It is occasionally invaluable for our family of five. Still, it hasn’t moved today. It didn’t move yesterday. It feels like a waste of a good car, but what is the alternative?

Digital technology is providing several alternatives, making it easier and easier to unlock the hidden value in such assets. Companies such as Zipcar will rent you a car by the hour. More radical are the likes of WhipCar, which allows owners to rent out their cars when they don’t need them. The opportunities are obvious, as are the obstacles.

WhipCar is an example of what the hip kids are calling “collaborative consumption”. The term seems to cover a multitude of activities. Ebay will let you sell your old stuff to someone who wants it more than you do. It was radical when it first appeared, not because the idea of a second-hand shop was new, but because of its unparalleled ability to find buyers – and sellers – for narrow niches.

More recent sites such as Swap.com or Freecycle will let you swap with others, or simply give away unwanted kit. On Zopa, lenders find borrowers without the need for a bank. Then there’s Airbnb (WhipCar for your spare room), or Landshare (people with unruly gardens meet people who want to do gardening).

Alongside the collaborative consumption sites are microlabour services such as TaskRabbit and Amazon’s Mechanical Turk, both of which connect people with small tasks to be done and people with time to do them. The actual offerings seem very different: TaskRabbit operates in just a few markets, such as Boston, Chicago and San Francisco, and is full of cheerful-looking background-checked locals who will run errands for you or put together an Ikea desk. Mechanical Turk distributes digital tasks anywhere in the world – but it has acquired an unenviable reputation for rock-bottom pay and spam.

For all the variety on show here, collaborative consumption and microlabour share two important characteristics. The first is that digital technology is lowering transaction costs, making it easier and easier for people to sell (or swap, or share) very small offerings. The second is that because so many of these offerings are in the hands of individuals rather than branded corporations, mechanisms for maintaining trust are very important. If we are going to let strangers stay in our homes, or drive our cars, or have custody of our dogs, on a one-night-stand basis, we may need some kind of reassurance.

Internet-based trust mechanisms where website users rate each other after each transaction also seem to be working tolerably well. Online rating systems can be tricked, but have held up better than anyone might have anticipated 15 years ago. Rachel Botsman, an evangelist for collaborative consumption, argues that trust is the pivot on which many of these new possibilities turn. It is hard to disagree. Presumably the likes of Google and Facebook are licking their lips as they contemplate their role in creating digital identities where your punctuality on eBay, efficiency on TaskRabbit and cleanliness as an Airbnb guest can be stitched together in a seamless tapestry of reputational capital.

And there is a curiosity here. Enthusiasts such as Botsman celebrate the way in which the internet enables human connection, where you can transact with real people rather than faceless corporations. But the other thing that is emerging is an ever-more-perfect market, with high price transparency, low transaction costs, and few interactions that cannot be priced and paid for.

The promise is either of creating value with a far more human touch, or of a hyper-competitive global marketplace. It would be fascinating if these things amounted to much the same.

Also published at ft.com.

A new paper suggests that respondents to surveys on well-being are affected by the way they are asked

How happy are you with your life? Yes, it’s back to “happynomics”, because a new paper suggests that one of the oldest findings in the field may be an artefact of the way this question is often asked.

First, some background. Happiness – “subjective well-being” – is often measured by asking survey respondents how satisfied they are with their lives, taken as a whole. Sometimes this is a multiple choice question: “very happy, fairly happy, or not very happy?” The most influential paper in happiness economics, Richard Easterlin’s 1974 “Does Economic Growth Improve the Human Lot?”, extensively discusses evidence from surveys of this kind, in which very few American respondents said that they were “not very happy”. Almost half said they were “very happy”, the most blissful response available.

People, then, are generally happy with life – too happy, perhaps, for a three-point scale. Easterlin is often quoted as having proved that economic growth does not lead to happiness, but perhaps we should say he pointed out that economic growth might not eliminate misery.

Another fascinating titbit from the old Easterlin paper, based on a 1965 survey: 53 per cent of the British respondents were “very happy” but only 20 per cent of West Germans and 12 per cent of the French. Income per head was similar at the time, as it is today. What could explain such an extraordinarily large difference? Language? Culture? Something else?

More modern research often uses a 10- or 11-point scale of happiness, often a “Likert scale”, which offers a statement such as “I am satisfied with my life as a whole these days”, and asks respondents to say how strongly they agree or disagree. Clearly this allows for shades of grey, although problems remain: Bill Gates can earn 100,000 times more than you do, but even if that makes him 100,000 times happier, there’s no way for him to express that joy on a 0-10 scale.

And there’s another problem with Likert scales: maybe something about an integer scale itself distorts the survey results? There’s an alternative: the VAS, or visual analogue scale, which replaces the 0-10 options with a single line. The respondents can mark where they would place themselves on the line between an extremely satisfactory life and an extremely unsatisfactory one.

Raphael Studer and Rainer Winkelmann, economists at the University of Zurich, used a randomised trial to tease out differences between the two scales. Some respondents were asked to report their happiness on a Likert scale and then, a month later, on a visual scale. Others were given the visual scale first and the Likert scale a month afterwards.

There were broadly similar responses to a Likert and a visual scale. But there were intriguing differences. One was that Likert respondents tended to cluster around seven and eight, being positive but avoiding extremes. The visual scale attracted a greater variety of responses; the usual findings were borne out (the middle-aged are less happy; the married are happier) but the size of each effect appeared bigger on the visual scale.

And there was one really odd result. We have long believed that women are happier than men, on the basis of decades of answers to Likert-type questions. The recent foray into happiness research by the UK’s Office for National Statistics used a 0-10 scale and replicated exactly this finding. But in Studer and Winkelmann’s randomised trial, this tendency disappeared. Women and men, who had answered differently on the Likert scale, pointed to the same point on the visual scale.

The authors conclude that the long-standing finding that women are happier than men is simply an artefact of the way the question is asked. How fortunate that nobody had been recommending gender reassignment surgery to all men as the secret of happiness.

Also published at ft.com.

Why the government should accept the Dilnot Commission’s proposals for the social care of the elderly

George Osborne’s “granny tax” – or more descriptively, George Osborne’s very gradual withdrawal of a long-standing granny-tax-break – has put intergenerational conflict in the spotlight.

Compare a 75-year-old with a 35-year-old, each on an annual income of £20,000. The younger taxpayer must pay almost £2,000 a year more in tax and national insurance, to say nothing of missing out winter fuel payments, bus passes and free television licences. Then there is the state pension, which will increase in line with wages, or inflation, or 2.5 per cent, whichever is more generous. Younger recipients of state benefits can only dream of this “triple lock”. And let’s draw a veil over who pays for higher education these days.

The standard defence of this state of affairs appears to be “Grandma, we love you”. That doesn’t seem like much of an argument, but so many voters and newspaper readers are over 60 that it is probably sufficient.

For those who like a bit more rigour, let me attempt two defences of the privileges of the elderly. One, most of us will be elderly eventually and what goes around, comes around. Two, it’s all very nice to talk about a 75-year-old on an income of £20,000 a year, but pensioners, on average, are poorer than younger people, especially at a time when quantitative easing and super-low interest rates are making it hard to live off savings or buy annuities.

Neither of these arguments is compelling. What goes around may actually not come around at all: if intergenerational transfers are too large, they will prove unsustainable at an awkward moment. Today’s young people have to pay for today’s elderly, pay the interest on a growing national debt, and yet they doubt that they will be generously treated themselves in 30 or 40 years’ time – with good reason.

As for pensioners being poorer, that is true, but those who are comfortably off get the perks anyway. Osborne introduced means-testing for child benefit but spared the ageing rich.

All this is basically a zero-sum game. What one generation gains, another loses. And the frustrating thing is that there is an important proposal currently sitting on the table that could help the elderly, and their families, out of all proportion to its cost.

I’m thinking of the Dilnot Commission’s recommendations for the social care of the elderly – such as hired help to assist with shopping, cooking, bathing, getting dressed and getting around. Andrew Dilnot’s team has reasoned as follows: such costs are typically low, but for one in 10 pensioners they are more than £100,000, sometimes much higher still. Local government provision is patchy and subsidised only for the poorest; private insurance for such costs is not available, because of the long timescales and unlimited liability involved. Therefore, people must simply live in fear of losing all their assets if infirmity strikes.

The Dilnot Commission proposes that the government should provide partial insurance, capping lifetime social care costs at £35,000 a year, and introduce more generous means testing. Private insurance markets and private savings should cover the remaining costs, which are fairly predictable consequences of not dying young.

The brilliance here is that because high social care costs are the exception rather than the rule, this government subsidy is inexpensive – Dilnot estimates the cost at less than £2bn a year, a small fraction of what is spent on pensions and on NHS services for the elderly. And for this money, a unique service is provided.

If I want to have more money when I am 80, there is a very simple way for me to do this: save more money today. But if I fear a crippling bill for care in my old age, there is nothing I can do except fret. That is why the government should accept the Dilnot Commission’s proposals.

Also published at ft.com.

The results of an experiment in Sri Lanka show the impact of financing on small businesses in communities shattered by natural disasters

In 2004, two economists, Chris Woodruff and David McKenzie, applied for funding from the National Science Foundation to do some research in Mexico. The idea was to draw inspiration from medical research and conduct a randomised controlled trial to find out whether small businesses could benefit from more access to capital. The NSF liked the idea, Woodruff recalls, but in the end turned down the funding request.

Then something truly awful happened – the Boxing Day tsunami, which devastated the coast of the Indian Ocean and killed well over 200,000 people. In Sri Lanka, more than 30,000 people were killed and many more left homeless. Small business owners found their stock, their equipment and their premises destroyed.

Against this backdrop, Julia Lane, an officer at the National Science Foundation, contacted McKenzie and Woodruff. She had liked the original research proposal and she had money to spend researching the aftermath of natural disasters. She even knew a local economics lecturer, Suresh De Mel, who might be able to get the research going in what was surely a challenging environment. A few weeks later the three researchers were in Sri Lanka testing out the first surveys.

The experiment itself looked at different kinds of businesses in different situations. All the businesses selected were tiny. Some had suffered huge losses. Others had avoided direct damage but were located in shattered coastal communities. Others were a few miles inland, unaffected by the tragedy on the shoreline. The typical (median) business had about $270 worth of capital, and the researchers were handing out grants of 10,000 rupees (about $100) or 20,000 rupees. These grants were sometimes paid in cash and sometimes made by accompanying business owners to market to make purchases on their behalf.

The results, recently published in articles in Science and the Economic Journal, tell a subtle story of recovery and of the difference access to capital can make. The broad story is this: businesses that suffered direct damage took a couple of years to catch up with those that had merely suffered indirect disruption; and businesses that received grants derived huge benefits from them – about 10 per cent return every month.

Beyond these averages, though, lurk stark differences. Damaged retail businesses enjoyed huge monthly returns when given grants. Unaffected businesses of any sort enjoyed healthy returns to capital – more than 7 per cent a month, suggesting that Sri Lankan small businesses are typically starved of capital.

But manufacturing businesses in affected areas, whether or not they had themselves been damaged, seemed to be able to do nothing useful with the money. One explanation is that they needed more capital to get manufacturing off the ground, but those receiving $200 seemed to do no better than those receiving $100. So a more plausible hypothesis is that each manufacturer was suffering from the disruption of supply chains.

In the coir industry, for example, there’s a long supply chain from the coconut harvest to processors to manufacturers of mats, brushes and other products. Each entrepreneur must deal with the damage to his suppliers and customers, and each entrepreneur may fear that others will simply give up and spend their grant money elsewhere. In such circumstances, says Woodruff, co-ordination and business advice may be critical.

The lesson I draw for other disaster hit areas, from Port-au-Prince in Haiti to Sendai in Japan, is that any industry based on a complex network of local firms is vulnerable. And something we already knew: recovery is dependent on the availability of loans or grants. When disaster has destroyed every physical asset of your business, capital counts.

Also published at ft.com.

The trend of economists functioning as detectives may ultimately be good for the profession

The first edition of The Undercover Economist sported a pulp-fiction private investigator on the cover. I’d suggested the image because, well, why not? Little did I realise I was anticipating a trend: the economist as detective.

It seems an unlikely development, given the fondness of economists for convenient simplifying assumptions. (Had Sherlock Holmes been an economist, he would surely have assumed that the Hound of the Baskervilles was a perfectly spherical dog moving in a frictionless environment.)

But whether the development seems unlikely or not, “forensic economics” has arrived. A splendidly simple example was published in 1994. William Christie and Paul Schultz noticed that prices on the Nasdaq market, quoted in eighths of a dollar, actually varied by quarter-dollars. Quotes ending one-eighth, three-eighths, five-eighths and seven-eighths were rare. Christie and Schultz checked carefully before concluding that this was probably symptomatic of collusion between market-making investment banks, designed to keep bid-ask spreads plump. An investigation by the Securities and Exchange Commission, class-action lawsuits and out-of-court settlements all followed.

Other examples of forensic economics are more complex. A recent survey on forensic finance by Jay Ritter and a forthcoming one on forensic economics by Eric Zitzewitz jointly outline how forensic economics is evolving.

One approach is to use purely statistical analysis, the most famous example of which comes courtesy of Brian Jacob and Steve Levitt, who identified some improbable patterns in multiple-choice test results in some Chicago schools. These suspicious “answer strings” (with all students giving the same response to clusters of questions) were correlated with sharp, temporary improvements in average scores.

Another approach is to use deviations from what an economic theory would predict. Zitzewitz found evidence of “late trading” in mutual funds, which is illegal. He showed that trades were correlated with information the traders couldn’t legally have possessed and could not – according to the efficient markets hypothesis – have deduced. The suspicious patterns disappeared after an investigation was announced.

Field experiments are also increasingly popular as a way of revealing suspicious behaviour. Benjamin Olken studied corruption in road building in rural Indonesia, devising his own measures of how much construction material had been stolen and cross-checking them with an independent audit.

What emerges from forensic economics? Zitzewitz argues that a major theme is government failure: not just the failure of dysfunctional governments in poor countries, but the failure of police and regulatory authorities in the United States and other wealthy countries. Perhaps in these gloomy times that will not come as a huge shock.

It’s fair to ask why economists feel they are qualified to undertake this kind of analysis. They are a self-confident bunch and may not be inclined to ask such questions of themselves. But answers do exist.

Economics is unusual in requiring statistical sophistication plus the ability to think about man-made institutions and human motivations. And economists are naturally suspicious: rational economic man, after all, is smart and amoral, the kind of person you’d want to keep an eye on.

The forensic economics trend may be good for the economics profession, too. It requires economists not to spend too much time thinking about theory, but to pay close attention to data, and to the messy way in which markets actually work. These cannot be bad habits for economists to acquire.

Also published at ft.com.

One handy way to size up a charity is to pay attention to how much it spends on overheads, rather than frontline do-gooding

You’re a generous person, I can tell. But how much do you think about the effectiveness of your charitable donations? One handy way to size up a charity is to pay attention to how much money it spends on overheads such as administration and fundraising, rather than frontline do-gooding. There’s only one small problem: this ready reckoner is enormously misleading.

For people who think about the effectiveness of charities, this insight is not news. Givewell, a charity that evaluates the effectiveness of other charities, complained five years ago about the “pervasive attitude that nonprofits need to get all their money right to the needy, and do all their administration on the cheap”. Dean Karlan, an economics professor and co-author of More Than Good Intentions, analysed Givewell’s recommendations and found that outstanding charities tended to spend more money, not less, on administration and fundraising.

Caroline Fiennes, author of a new book, It Ain’t What You Give, It’s The Way That You Give It, explains that fundraising costs tend to be determined by donors – who can generous or stingy, ignorant of the cause or conscious of it. Meanwhile, administration costs could include efficient logistics, accounting or purchasing systems – plus paying for rigorous evaluation.

It isn’t just in the world of charitable giving that we pay too much attention to administrative costs. Government ministers of all stripes love to claim that they will cut bureaucracy, sacking administrators and managers and investing the savings in “teachers and nurses”. If your child’s school is closed for a day or so because the heating fails, or your operation is cancelled due to lack of surgical supplies, then you can at least console yourself that those pesky administrative costs are being thoroughly squeezed.

The truth is that in the modern world, a surprising amount of money is spent on what one might call transaction costs. One definition of a transaction cost is any cost that Robinson Crusoe could never conceivably have faced. Costs of processing trades, searching for bargains, standing in line and suing for breach of contract are all transaction costs. So, arguably, are the costs of maintaining accounts and filing (or avoiding) taxes.

John J. Wallis and Douglass North, in a book chapter published in 1986, tried to estimate the importance of transaction costs in the US economy between 1870 and 1970. For simplicity, Wallis and North tried to define whole job categories devoted to supporting transactions (these include managers, sales assistants, lawyers, police and accountants) and also sections of the economy, such as retail, which were almost entirely devoted to supporting transactions.

Wallis and North reckoned that the production of the economy devoted to transaction services had more than doubled over the century, from 26 per cent of gross national product in 1870 to 55 per cent of GNP in 1970. Public sector transaction spending had grown especially rapidly, but from a low base, and the lion’s share of transaction costs remained in the private sector: a total of over 40 per cent of GNP – an awful lot of administrators. All waste? Surely, the story is a continuation of what Adam Smith identified back in 1776: increasing productive power thanks to specialisation and the division of labour. A subsistence farmer may have overheads, but he needs few transaction services. A modern city-dweller, who continually does business with strangers, lives and breathes them.

Spare a thought, then, for the humble back office. Not only are administrators, accountants, lawyers and managers necessary to make a charity work efficiently – such people make the modern world possible.

Also published at ft.com.