Tim Harford The Undercover Economist

Articles published in 2016

The Undoing Project – Book Review

Michael Lewis could spin gold out of any topic he chose but his best work has shone a spotlight into corners of the world that weren’t getting enough attention until Lewis came along. Liar’s Poker described bond trader Lewis Ranieri and the way securitisation revolutionised Wall Street in the 1980s. Moneyball covered baseball manager Billy Beane and anticipated the “quants” taking over the world. And The Big Short depicted Steve Eisman and Michael Burry, the men who spotted the financial crisis coming and bet vast sums on it.

The Undoing Project, then, is a departure, because it’s a biography of two well-established figures: Daniel Kahneman and Amos Tversky, the Israeli psychologists whose partnership produced the foundations of what we now call behavioural economics. Despite an introduction by Lewis declaring that he hadn’t heard of them until 2003, neither man remotely counts as an unknown.

When Tversky died young, in 1996, he was on the secret shortlist for a Nobel memorial prize in economics, and received a detailed obituary in The New York Times. Kahneman won the Nobel economics prize in 2002 and published his own bestselling book, Thinking, Fast and Slow, in 2011. Their ideas are everywhere; it’s almost impossible to find a book in the “smart thinking” section of a bookshop that doesn’t cite Kahneman and Tversky: an irony, since their work highlights many of the ways in which our thinking isn’t smart at all.

For example, they identified the “representativeness heuristic” — our tendency to make judgments by comparing an example to some mental model. When we meet a nervous, geeky-looking gentleman we note that he matches our stereotype of a programmer and, therefore, probably is a programmer. We forget that most of the people we meet are not, in fact, programmers, no matter how much they might resemble them.

This matters, because when judging probabilities we often skip over the real question, “Is this likely?”, in favour of a representativeness question: “Does this match my preconceptions?”. “Is the lump likely to be a malignant tumour?” becomes “Does the lump match my idea of what a malignant tumour looks like?”. It’s a reasonable rule of thumb that can lead us seriously astray.

All this is well known to anyone who has read Kahneman himself or popularisations of his work, so what does Lewis add? He’s a far better writer than most, meaning that even the familiar is fresh. And there is a great deal here that feels new. Lewis has done his homework; he has evidently talked to the right people — with the inevitable omission of the much-missed Tversky — and he knows how to tell a story simply, powerfully and with an eye for the telling detail.

Yet The Undoing Project gets off to a shaky start with a chapter discussing the selection of basketball players and the way in which basketball scouts commit various cognitive errors. Perhaps the success of Moneyball encouraged Lewis and his editor to think this was wise but it adds very little to our appreciation of the main characters, and much of the chapter is baffling unless one happens to be a fan of American sports.

All is forgiven in chapter two, when we meet the young Danny Kahneman, a Paris-raised Jew whose family spend the war dodging the Nazis and their sympathisers. No matter how many accounts one reads of such horrors, the reader is filled with sadness and a kind of awe at the survivors. At the age of seven, Danny was caught on the streets after curfew by an SS soldier. The man didn’t notice the yellow star under his sweater; instead, he hugged little Danny and, full of emotion, showed him a photograph of another young boy. Then he gave Danny some money and sent him on his way. “People were endlessly complicated and interesting,” Kahneman recalled.

Tversky is no less deftly portrayed: as a child, he was so bullish that he was willing to leap from a high diving board despite being unable to swim — he simply arranged for a bigger boy to be on hand to drag him to safety. As a soldier, Tversky saw a comrade pull the pin on a grenade-like explosive, then faint. As his commanding officer yelled orders to stay put, Tversky dashed forward, dragged the stricken man a few yards away, then dived to cover him, taking the shrapnel into his own body. Yet he berated his own men for carelessly taking risks. “If a bullet is going to kill me, it has my name on it anyway,” they would say. Tversky, a quick wit, reminded them that many bullets were addressed “To Whom It May Concern”.

Today, Kahneman and Tversky’s view of human psychology is widely accepted, and thanks to his longevity and his Nobel Prize, Kahneman is a more famous figure than Tversky. But Lewis takes us back in time, conjuring up the 1970s, when their ideas were new and controversial, they were operating in the backwater of Israeli academia, and when it was the mesmerising Amos rather than the quiet Danny who won all the attention.

Behavioural economics itself is not a major part of the book. Richard Thaler, the most important intellectual conduit between Kahneman and Tversky and economics, does not appear in the story until the closing chapters. While Tversky loved to have an intellectual foe to slay, it would diminish his work with Kahneman to define it merely as a takedown of textbook economics. By writing less about behavioural economics Lewis gives Kahneman and Tversky’s ideas room to breathe.

Lewis admires his subjects and believes they are right about everything important. He has no time for rational economic man, and brutally dismisses one noted critic of Kahneman, the psychologist Gerd Gigerenzer. But this isn’t a hagiography. Tversky is depicted as intellectually aggressive, contemptuous of many academics and perversely compelled to needle the vulnerable Kahneman. Meanwhile, a new side to Kahneman emerges. In my limited personal experience, Kahneman seems wise, kindly and stoic in the face of his advancing years. But Lewis describes the younger Kahneman as depressed, envious of his celebrated partner and desperately needy.

Nevertheless, it is clear that Lewis is cheering our heroes on, and the reader cannot help but join him. The story he tells of their intellectual love affair, and its painful disintegration, is vivid, original and hard to forget.
Written for and first published in the Financial Times.

My new book “Messy” is now out and available online in the US and UK or in good bookshops everywhere.

Free email updates

(You can unsubscribe at any time)

23rd of December, 2016Other WritingComments off

Why family traditions make for happy holidays

The Japanese have a particularly engaging ritual at this time of year: Kurisumasu ni wa kentakkii, which may sound like great wisdom but in fact refers to “Kentucky for Christmas”, the national habit of eating Kentucky Fried Chicken as their Christmas feast.

It began as an inspired bit of marketing. In the 1970s, KFC noticed that western expatriates in Japan were turning to fried chicken because they couldn’t get hold of turkey. Now it has become a ritual with enough selling power that there are queues around the block, and customers will order their chicken in November or even October. What’s particularly impressive is that Christmas isn’t even a holiday in Japan.

Kurisumasu ni wa kentakkii is an audacious piece of commercialisation, but it’s not the first time Christmas has been boldly hijacked to sell something; indeed, Christmas itself piggybacked on earlier midwinter festivals. Many Christmas traditions are fairly recent — in the UK, Christmas cards, turkey, crackers and trees are all 19th-century innovations. On the other hand, Christmas consumerism, which we tend to think of as a modern vice, is also a 19th-century habit. Joel Waldfogel’s book Scroogenomics shows that the boom in December spending can be traced back many decades. (In 1867 in New York, Macy’s decided it was worth keeping its doors open until midnight on Christmas Eve.)

So I don’t begrudge the Japanese their southern-fried Christmas ritual. In fact, I have been intrigued by Christmas rituals in general; some are good and some less so. As I pointed out last week (and last year), the ritual of giving gifts at Christmas is extremely wasteful, channelling valuable resources into ill-fitting clothes and tacky golf memorabilia that nobody would choose if they were buying for themselves.

Over the past decade or so, we’ve seen a new combination of two older traditions: giving gifts, and donating to charity. With much the same cheek as Kentucky Fried Chicken in Japan, Oxfam has been pushing the idea that you can “give twice” by donating to Oxfam on someone else’s behalf. In effect, you’re “giving” a dozen chicks, or a vegetable garden, or, notoriously, a goat. “A unique, symbolic gift,” says Oxfam, although a moment’s reflection will reveal it not to be unique at all.

And what does it symbolise? For some, it represents wit, anti-consumerism and the true spirit of Christmas. For others, it represents smug self-obsession. After all, if I give you an Oxfam goat, what have I really done? I’ve made a donation to charity, marinated in my own sense of superiority, and then mailed you the receipt. To add injury to this insult, I’ve also not bothered to buy you a gift. We can only hope that Oxfam, at least, is able to find someone who actually needs the goat.

Three researchers, Lisa Cavanaugh, Francesca Gino and Gavan Fitzsimons, recently published research into this sort of “socially responsible” gift. They found evidence that people systematically overestimate how welcome such gifts will be, particularly when they are given to people they don’t know terribly well. As I described last week, Gino has contributed to other research on gift-giving, discovering that recipients are often happier with gifts chosen from a wishlist or otherwise explicitly requested, even though that does seem to lose some of the charm. Combining the two insights suggests that if you want to make a donation to charity on someone’s behalf, it might be wise to ask for their blessing first.

***

What of the Christmas rituals we share as a group? Different families will have their own rituals — perhaps the gifts are to be opened at a particular time or in a particular order. Perhaps the Queen’s Speech must be watched, respectfully. My wife’s family, religiously minded, would delay opening gifts until after a Christmas Day service. My own family were more secular but fond of delayed gratification — many gifts had to wait until after Christmas lunch. For some families, there’s a particular film to be watched; for others, a board game to be played.

Ovul Sezer, Michael Norton, Francesca Gino (again) and Kathleen Vohs have been examining the effect of rituals on the way we experience Christmas and New Year’s Eve. The researchers find a correlation between these rituals — sacred or secular — and various positive experiences of the holiday season, including feeling more curious, paying closer attention, liking the family more, enjoying the seasonal holiday, and life satisfaction in general. Of course, causation may run the other way — it may be that disliking your family causes you to avoid sharing Christmas rituals with them rather than the other way round.

Still, Sezer and her colleagues seem to be on to something. Life is full of social rituals but their details often do not seem to matter much: the wake, the prom night, the baby shower, even Kurisumasu ni wa kentakkii — we form our social habits and we stick to them. And the research suggests we’re absolutely right to do so. The rituals seem to make our lives richer and more enjoyable.

And without going into too much detail, there is one Christmas tradition that’s unlikely to disappear any time soon: the number of children conceived peaks over the Christmas holidays, leading to a bump in birth rates in September. That’s certainly one way to bring a family closer together.

Written for and first published in the Financial Times.

My new book “Messy” is now out and available online in the US and UK or in good bookshops everywhere.

Free email updates

(You can unsubscribe at any time)

The economist’s guide to gift-giving

“There are worlds of money wasted, at this time of year, in getting things that nobody wants, and nobody cares for after they are got.” That was Harriet Beecher Stowe in 1850, reminding us that concerns over Christmas consumerism aren’t new.

 Also not new is Joel Waldfogel’s notorious research paper, The Deadweight Loss of Christmas, published 23 years ago in American Economic Review, a respected journal. Waldfogel, now a professor of economics at the University of Minnesota, expanded on his ideas in 2009 in a brief and witty book, Scroogenomics. He showed that gifts typically destroy value, in the sense that the giver had to pay more to buy the gift than the recipient would ever have been willing to spend on it. The total deadweight loss of Christmas in the US alone was $12bn.

Alas, what sounds like wisdom from Stowe tends to be mocked when published in an academic journal. But the problem that Waldfogel quantifies is quite real. If you give someone a jumper that doesn’t fit, a book they’ve already read or a box of chocolates when they’re on a diet, this is a waste of valuable resources. Fossil fuels have been burnt, tedious hours have been worked, trees have been felled, all to produce products that were unwanted. The same resources could have been devoted, instead, to goods that people actually do value.

Still, one cannot simply spit “Bah! Humbug!” and have done with Christmas gifts; people have their expectations. Nor can one simply dole out cash — at least, not to grown-ups. So, then, what to do? I asked Waldfogel himself, and several other social scientists, how they resolve the tension between the fact that Christmas gifts are a shameful waste and the fact that they are socially obligatory.

It’s not easy. Andrew Haldane, chief economist at the Bank of England, tells me: “I start out with the best of intentions — some small, inexpensive but deeply meaningful gifts that will stir the soul of the recipient — and then, at the last minute, end up panic-buying rather thoughtless, often expensive and largely unwanted stuff.” It is good to know that the Bank is in touch with how the rest of us act.

Dan Ariely, a psychologist at Duke University and author of behavioural economics books including Payoff, might encourage Haldane to forgive himself. Ariely rejects the basic Waldfogel premise. Economists, he told me, just don’t get it. They are seduced by their own training to be selfish and narrowly focused on efficiency. Giving gifts, says Ariely, is “inefficient economically but efficient socially”.

Well, perhaps. Certainly, if I give you a bruise-blue cardigan that you detest, we can all agree that this is no tragedy because it’s the thought that counts. But we can also agree that it would have been better if I had chosen a nicer cardigan.

One approach, independently advocated by Kimberley Scharf of the University of Warwick and Francesca Gino of Harvard Business School, is to buy only what has been explicitly requested.

This idea has some science behind it. Gino, the author of Sidetracked, has published research (with Stanford’s Frank Flynn) into how people feel about wishlists.

“Gift recipients prefer to receive items they’ve asked for, and they think givers who fulfil this ideal are more thoughtful,” says Gino. “Yet when we’re the one who is doing the giving, we fail to realise that people tend to prefer receiving what they told us they want.”

Basically, when we’re the giver, we scorn the wishlist and get creative, imagining that we’re smarter choosers than we really are; when we’re the receiver, we would simply be delighted to receive exactly what we asked for.

Gino herself looks for a wishlist whenever possible, and can recall several occasions when she was on the brink of buying an expensive and entirely inappropriate gift, only to be saved by having an honest conversation with the target of her generosity.

But what of poor Waldfogel himself? When the subtitle of your book is Why You Shouldn’t Buy Presents for the Holidays, you’re setting yourself up for a lifetime of giftless ostracism. But, says Waldfogel, he does still receive gifts, often “coffee, chocolate, or cognac . . . things I am known, by my friends and family, to use.”

He adds: “Honestly, how bad can any of these things turn out to be?”

Well, quite. Although my wife wouldn’t touch any of the three items — a reminder that there is no such thing as the all-purpose gift.

Waldfogel argues that it’s possible to do even better than the wishlist. The ideal, he says, is to find a gift that transcends what a person would be able to buy for themselves. And his answer is the gift of permission. “If I want something that’s a little extravagant, then I run it past my wife, who gives me permission to buy it.”

This does make a strange kind of sense. Last Christmas I bought my wife an expensive piece of camera kit — after carefully quizzing her to make sure I had exactly the right thing. We have a joint bank account, so what was I really giving her? Not money, and not effort. I was giving her my blessing.

Written for and first published in the Financial Times.

My new book “Messy” is now out and available online in the US and UK or in good bookshops everywhere.

Thomas C. Schelling, 1921 – 2016

Thomas C. Schelling, who died on December 13 at the age of 95, was a self-described “errant economist” who worked as a Cold War strategist and won the most prestigious prize of his profession.

Schelling was a popular winner of the Nobel memorial prize for economics. Journalists found his lively prose and counterintuitive ideas easier to describe than the complex equations of his fellow laureate Robert Aumann.

But the Californian-born economist was an unlikely laureate. His ideas were rich and influential — and easily expressed in plain English. He highlighted weaknesses in standard economic approaches, deploying vivid thought experiments more suited to moral philosophy than to economics, and rarely cited other academics.

Instead, Schelling used academia as a vantage point from which to advise the administrations of Presidents Kennedy, Johnson and Nixon. He was at Harvard University for 31 years, and said of one role there that it had given him a decade of “freedom to write and to consult, and I spent much of my time, especially during the summer, doing advisory work for the government.”

That advisory work drew on one discipline in particular.

Game theory had been dreamt up by the mathematician John von Neumann, as an attempt to model in mathematical terms human interactions from poker through to strikes or cartels.

The Hungarian-born von Neumann was a hawk (“If you say why not bomb [the Soviets] tomorrow, I say why not today?”) but Schelling took game theory in a new direction. He emphasised that even the most implacable foes could find areas of common interest — most obviously, during the Cold War, the necessity of avoiding mutual annihilation.

To this end, in the late 1950s and the 1960s, Schelling’s advisory work and his publications focused on issues of effective deterrence, communication, and the strategic limitation of arms. He was a consultant for Stanley Kubrick’s 1964 film Dr Strangelove, a nuclear annihilation comedy which introduces a “doomsday device”. The device is the ultimate deterrent: it will be triggered automatically in the case of war. Alas, it’s a secret, which limits the deterrent effect.

The doomsday device was the perfect illustration of some of Schelling’s favourite themes: strategic commitment, miscommunication, and unintended consequences. It is no coincidence that it was Schelling who insisted that Washington and Moscow establish a secure hotline and work out protocols for ensuring it was tamper-proof. This attention to details that others overlook was a spark for his best academic work. It is also is one reason why nuclear weapons have not been used since 1945.

Schelling ended his advisory work with a letter opposing the 1970 US military campaign in Cambodia. He later worked on the problem of addiction, developing models of self-control that were precursors to what is today known as behavioural economics. This research was partly motivated by his own long and eventually successful struggle to stop smoking. And in 1980, at the request of President Carter, he became one of the first economists to work on the problem of human-induced climate change.

He also anticipated the use of complexity science in economics with a celebrated “chessboard” model of segregation. This showed how two racial groups could completely segregate from each other in a chain reaction despite being quite comfortable in a mixed neighbourhood.

These days such modelling is done on a computer, but Schelling originally explored the idea in a notebook doodle on a long flight. “It was hard to do with pencil and paper,” he told the FT in a 2005 interview. “You had to do a lot of erasing.”

Thomas Crombie Schelling was born in Oakland, California, on 14 April 1921. His father was in the US Navy, but despite Thomas Schelling’s crew cut, square jaw and family history, he did not fight in the war. For medical reasons, the military would not accept him. Instead he studied economics at the University of California, Berkeley and earned his PhD at Harvard. After a spell working on the Marshall Plan, he taught at Yale, Harvard and finally the University of Maryland.

Schelling married Corinne Saposs in 1947. After that four-decade marriage ended in divorce, he married Alice Coleman, who survives him, as do four sons, two stepsons and his younger sister, Nancy.

Written for and first published in the Financial Times.

17th of December, 2016Other WritingComments off

Game changers: the importance of the puzzle

Looking forward to a few family board games at Christmas? Good, good. Steer clear of Monopoly — a dreadful, grinding game. Settlers of Catan is the game that Monopoly wishes it was although, for committed players, Puerto Rico is even better. (Agricola is the best game. But you knew that already.)

 Games are a serious business in the Harford household these days but they, along with puzzles and other fripperies, have long been important elsewhere.

Consider the puzzle of the bridges of Königsberg — 18th-century Königsberg had seven bridges connecting two sides of a river and two large islands. The puzzle was: is there a walking route through the city that crosses each bridge only once? When the great mathematician Leonhard Euler heard about the problem, he found it “banal” but was intrigued by the fact that, despite its apparent simplicity, “neither geometry nor algebra nor even the art of counting” could solve it. And so Euler invented an entirely new branch of mathematics, graph theory.

“That was the beginning of all network analysis,” says Alex Bellos, the author of Can You Solve My Problems?, a new book celebrating the joys and history of puzzles. Euler’s graph theory has been enormously fruitful in chemistry, physics, sociology and, of course, computer science. The internet relies on Euler’s analysis. And it all started with a brain-teaser.

Perhaps it’s no surprise that a puzzle might lead to a mathematical breakthrough: after all, puzzles are designed to be intellectual challenges. But other pastimes have also spurred fresh ideas — for example, gambling. Perhaps the first gambler to draw inspiration from this vice was the Renaissance mathematician Girolamo Cardano, who produced the foundations of probability theory.

Three-and-a-half centuries after Cardano died, the smartest man in the world decided to use mathematics to work out how best to play poker. John von Neumann was one of the driving forces behind the development of both the atomic bomb and the computer, and he wanted to apply mathematics to social sciences — for example, analysing the success or failure of negotiations, or the formation of alliances. His contention was that a mathematical theory that could explain life should start by explaining poker: “Real life consists of bluffing, of little tactics of deception, of asking yourself what is the other man going to think I mean to do, and that is what games are about in my theory.”

The result of von Neumann’s musings — first alone, and then with the economist Oskar Morgenstern — was “game theory”, one of the building blocks of modern economics and an important tool in evolutionary biology.

This is an impressive list of ways in which games have inspired us — and we haven’t even touched on the way that computer scientists have used chess as a testing ground for their machines.

But it would do games a disservice to treat them merely as a source of intellectual inspiration. They have inspired us in other ways too. In his new book, Wonderland, Steven Johnson makes a convincing case for the transformative power of play and delight — for example, the Victorian designer of paleo-computers, Charles Babbage, was inspired by a captivating mechanical toy dancer.

This wasn’t the last time that pure fun changed the world of computers. The first video game that mattered, Spacewar!, was designed in the early 1960s by enthusiastic students at the Massachusetts Institute of Technology who wanted to demonstrate just what the latest computers could do. And what they could do went way beyond the technical: they could hijack our attention, trigger Pavlovian responses, even addict us, by providing a compelling and engaging challenge.

Silicon Valley visionary Stewart Brand wrote about Spacewar! in Rolling Stone magazine in 1972. He saw what the game represented: computers that ordinary people would come to love. “I saw them having some kind of out-of-body experience,” he said recently of Spacewar!’s players. “Their brains and their fingers were fully engaged.” That curious compulsion felt by every PlayStation junkie or Instagram addict was felt first by the players of this early game.

Now, researchers at DeepMind, Google’s artificial intelligence outfit, are turning to computer games to train artificial intelligences. The AI is shown the game screen, given access to the score and a controller, and then — with no further information — figures out how to master the game. At first, DeepMind started with simple games such as Atari’s Breakout but it has recently moved on to Starcraft II, a game that requires tactics, military strategy, surprise and economic planning. Like von Neumann’s poker, learning to play Starcraft II is good training for the rigours of reality.

One of Alex Bellos’s puzzling heroes is Hubert Phillips — the man who coined the word “mezzobrow” to refer to “people of the highest intelligence who enjoy such things as crosswords, chess problems, inferential puzzles and parlour games”. Even as a child I knew about Phillips, the author of a veritable bible of card games. Phillips also wrote 200 detective stories, compiled crosswords and championed a number of classic puzzles — including the “logic grid” problems that are such a staple of the genre.

But that is not how Hubert Phillips began his career. He was a noted economist, adviser to the Liberals in the 1920s and the head of Bristol University’s department of economics. All very respectable, but thank goodness he turned to the far more important practice of having fun.

Written for and first published in the Financial Times.

My new book “Messy” is now out and available online in the US and UK or in good bookshops everywhere.

A Christmas reading list

A few books that I’ve been enjoying recently…

Lyonesse (US) (UK) by Jack Vance – I recently re-read this in preparation for a forthcoming discussion on the Fictoplasm podcast. It really is magnificent – the finest fantasy trilogy out there. Vance is witty, he’s inventive, and joyful. His bad guys are truly wicked, his heroes and heroines are compelling, and his fairies are utterly mysterious. The dialogue is as distinctive and enjoyable as anything in Tarantino.

The Undoing Project (US) (UK) by Michael Lewis. I have a review coming soon of this book in the Financial Times. It’s a biography of the great psychologists Daniel Kahneman and Amos Tversky, and it’s nearly perfect – an odd and unnecessary opening chapter, but just start at chapter two and you’ll love it. Kahneman and Tversky are fascinating characters; Kahneman won a Nobel memorial prize and wrote the excellent Thinking Fast and Slow (US) (UK). I thought I knew a fair bit about their collaboration but Lewis’s story is full of surprises and no small degree of tragedy.

Newly crossed my desk, Jonathan Portes’s 50 Ideas You Really Need To Know About Capitalism (US) (UK) is presented, as you might expect, in bite-sized chunks. No spellbinding narrative here but that was not the aim. Portes is knowledgeable and the chapters are sharp and clear and full of interesting nuggets.

And, I’ve mentioned it before but Steven Johnson’s Wonderland (US) (UK) is on form – a playful and surprising guide to how play, puzzles and delight have shaped innovation.

Einstein’s Greatest Mistake (US) (UK) by David Bodanis is a great short biography with some super storytelling.

And finally, it would be perverse of me to omit my own book, Messy. Happy reading!

Free email updates

(You can unsubscribe at any time)

6th of December, 2016MarginaliaComments off

Why forecasters failed to predict Trump’s victory

British Remainers watched the US presidential campaign with an uneasy sense that they had seen it all before: brazen lies from a populist movement, experts lining up to attest that all sensible people agree on what should be done … Those of us who saw the EU referendum campaign up close have been well prepared for the possibility of a Trump victory. US Democrats had less of a visceral warning and so were more surprised.

The truth is that once Trump had secured the nomination, a Trump presidency was always a strong possibility. The betting markets seemed to recognise this, offering odds of three-to-one a week or so before the poll. Three-to-one shots happen all the time — or at least, about a quarter of the time. A defeat for Hillary Clinton may be far more consequential than a defeat for Manchester City and, therefore, far more shocking but it shouldn’t be any more surprising. Favourites do not always win.

Forecasting is a tough job but we make it harder than it has to be by committing some familiar cognitive errors. So what are the lessons that we should learn?

First, wishful thinking has struck again. After the Brexit vote, I described the research of the economist Guy Mayraz. At Oxford university’s Centre for Experimental Social Science, Mayraz ran experiments in which participants were told that they were either “farmers”, who would be paid more if wheat prices were high, or “bakers”, who would be paid more if wheat prices were low. They were then shown a graph, purportedly tracking the wheat price, and invited to forecast the future price, with a cash reward for accurate forecasts. Despite the fact that they were being paid for accuracy, the farmer-participants systematically forecast higher wheat prices than the bakers. Everyone predicted what they hoped would happen. Does that sound familiar?

The second lesson is that — as a large experiment conducted by the Good Judgment Project has shown — self-critical, open-minded forecasters do a better job than narrow-minded, overconfident ones. Of course that is obvious — except that open-mindedness is a quality in short supply. Dwelling on our own fallibility is like dwelling on our own mortality: most people find it uncomfortable, so we don’t do it.

Whether you’re sinking a beer with friends or prognosticating on cable news, the social pressure is to make an interesting, confident statement rather than hum and haw about all the ways in which you might be wrong. Confident, eye-catching forecasts are the snack food of analysis and commentary: everybody knows they’re doing us no good but we can’t seem to resist.

It was interesting to see the self-critical dynamic in action at Nate Silver’s political forecasting website FiveThirtyEight. I interviewed Silver at a public event when he was riding high after successful forecasts in 2012. The audience questions were fawning but I was impressed at how Silver kept emphasising that he’d been lucky and future forecasts would be harder.

This time round, FiveThirtyEight botched the analysis of the Republican nomination: the polls said Trump was clearly ahead but nobody could quite believe that. Yet the early failure provoked some introspection. FiveThirtyEight learnt a lesson. While other forecasters were writing off Trump in the presidential race, FiveThirtyEight kept looking at the polls, and continued to declare that it was close.

A third lesson is that we have to keep an open mind that more than one outcome is possible. Too many people equated “Clinton is the favourite” with “Clinton will win”. That’s an obvious error, but it’s common. Even expert forecasters often treat a strong possibility as though it is a certainty. This tendency is one reason that dart-throwing chimps give the experts a run for their money. The chimps make lots of forecasting errors too, but at least they don’t systematically overrate their chances.

Perhaps the best way to keep more than one outcome in mind is to develop scenarios. I’ve written before about the scenario-planning method: scenarios are persuasive, coherent stories about the future, and like all good stories they have a tendency to stick with the listener. A scenario-planner will create at least two contradictory stories — this helps people to keep an open mind, and to prepare for more than one outcome.

FiveThirtyEight does not quite embrace scenario-planning but it did produce a clear account of the kind of polling error that would be necessary to deliver a Trump win — along with a reminder that such polling errors are ubiquitous. Clinton was the favourite even for FiveThirtyEight, but the site presented its readers with a very clear description of how the less-likely prospect of a Trump win would unfold, if it were to happen.

Scenario-thinking is not really intended to produce better forecasts — although I think it does no harm. What it should deliver is preparedness. Trump’s victory has caught a lot of people napping; that’s strange. We take precautionary measures against far less likely events than the victory of a presidential underdog.

Amid many depressing features of the politics of 2016 has been our failure to prepare for perfectly foreseeable possibilities. If there is a silver lining, it’s this: the uncertainties are not going away, so it’s not too late to learn.

Written for and first published in the Financial Times.

My new book “Messy” is now out and available online in the US and UK or in good bookshops everywhere.

Free email updates

(You can unsubscribe at any time)

The best ways to combat bias from Airbnb to eBay

Six months ago, tech entrepreneur Rohan Gilkes tried to rent a cabin in Idaho over the July 4 weekend, using the website Airbnb. All seemed well, until the host told him her plans had changed: she needed to use the cabin herself. Then a friend of Rohan’s tried to book the same cabin on the same weekend, and his booking was immediately accepted. Rohan’s friend is white; Rohan is black.

This is not a one-off. Late last year, three researchers from Harvard Business School — Benjamin Edelman, Michael Luca and Dan Svirsky — published a working paper with experimental evidence of discrimination. Using fake profiles to request accommodation, the researchers found that applicants with distinctively African-American names were 16 per cent less likely to have their bookings accepted. Edelman and Luca have also published evidence that black hosts receive lower incomes than whites while letting out very similar properties on Airbnb. The hashtag #AirbnbWhileBlack has started to circulate.

Can anything be done to prevent such discrimination? It’s not a straightforward problem. Airbnb condemns racial discrimination but, by making names and photographs such a prominent feature of its website, it makes discrimination, conscious or unconscious, very easy.

“It’s a cheap way to build trust,” says researcher Michael Luca. But, he adds, it “invites discrimination”.

Of course there’s plenty of discrimination to be found elsewhere. Other studies have used photographs of goods such as iPods and baseball cards being held in a person’s hand. On Craigslist and eBay, such goods sell for less if held in a black hand than a white one. An unpleasant finding — although in such cases it’s easy to use a photograph with no hand visible at all.

The Harvard Business School team have produced a browser plug-in called “Debias Yourself”. People who install the plug-in and then surf Airbnb will find that names and photographs have been hidden. It’s a nice idea, although one suspects that it will not be used by those who need it most. Airbnb could impose the system anyway but that is unlikely to prove tempting.

However, says Luca, there are more subtle ways in which the platform could discourage discrimination. For example, it could make profile portraits less prominent, delaying the appearance of a portrait until further along in the process of making a booking. And it could nudge hosts into using an “instant book” system that accelerates and depersonalises the booking process. (The company recently released a report describing efforts to deal with the problem.)

But if the Airbnb situation has shone a spotlight on unconscious (and conscious) bias, there are even more important manifestations elsewhere in the economy. A classic study by economists Marianne Bertrand and Sendhil Mullainathan used fake CVs to apply for jobs. Some CVs, which used distinctively African-American names, were significantly less likely to lead to an interview than identical applications with names that could be perceived as white.

Perhaps the grimmest feature of the Bertrand/Mullainathan study was the discovery that well-qualified black applicants were treated no better than poorly qualified ones. As a young black student, then, one might ask: why bother studying when nobody will look past your skin colour? And so racism can create a self-reinforcing loop.

What to do?

One approach, as with “Debias Yourself”, is to remove irrelevant information: if a person’s skin colour or gender is irrelevant, then why reveal it to recruiters? The basic idea behind “Debias Yourself” was proven in a study by economists Cecilia Rouse and Claudia Goldin. Using a careful statistical design, Rouse and Goldin showed that when leading professional orchestras began to audition musicians behind a screen, the recruitment of women surged.

Importantly, blind auditions weren’t introduced to fight discrimination against women — orchestras didn’t think such discrimination was a pressing concern. Instead, they were a way of preventing recruiters from favouring the pupils of influential teachers. Yet a process designed to fight nepotism and favouritism ended up fighting sexism too.

 … 

A new start-up, “Applied”, is taking these insights into the broader job market. “Applied” is a spin-off from the UK Cabinet Office, the Behavioural Insights Team and Nesta, a charity that supports innovation; the idea is to use some simple technological fixes to combat a variety of biases.

A straightforward job application form is a breeding ground for discrimination and cognitive error. It starts with a name — giving clues to nationality, ethnicity and gender — and then presents a sequence of answers that are likely to be read as one big stew of facts. A single answer, good or bad, colours our perception of everything else, a tendency called the halo effect.

A recruiter using “Applied” will see “chunked” and “anonymised” details — answers to the application questions from different applicants, presented in a randomised order and without indications of race or gender. Meanwhile, other recruiters will see the same answers, but shuffled differently. As a result, says Kate Glazebrook of “Applied”, various biases simply won’t have a chance to emerge.

When the Behavioural Insights Team ran its last recruitment round, applicants were rated using the new process and a more traditional CV-based approach. The best of the shuffled, anonymised applications were more diverse, and much better predictors of a candidate who impressed on the assessment day. Too early to declare victory — but a promising start.

Written for and first published in the Financial Times.

My new book “Messy” is now out and available online in the US and UK or in good bookshops everywhere.

Free email updates

(You can unsubscribe at any time)

 

Why economics is a discipline in need of diversity

Does economics have a problem with diversity? The example of Janet Yellen, the most powerful economist in the world, suggests not. But she doesn’t have much company at the top: Christine Lagarde at the IMF is a lawyer; Elinor Ostrom, the only woman to win the Nobel Memorial Prize in Economics, was a political scientist. Even when women win the highest elected office in the land, a woman has never run the US Treasury or been finance minister in either unified Germany or West Germany. Economics, it seems, is a particularly male bastion.

A new survey by Amanda Bayer of Swarthmore and Cecilia Elena Rouse of Princeton puts some firm statistics on this general impression. Looking at the US, the academics find that fewer than 30 per cent of bachelor’s degrees in economics are awarded to women — and the percentage is similar for doctorates.

Is this just part of the age-old problem that women do not study mathematical subjects? In the UK, perhaps: girls are less likely than boys to study A-level mathematics. But in the US, the explanation doesn’t stack up. “Stem” subjects — science, technology, engineering and mathematics — now award more degrees to women than to men at both undergraduate and doctoral level. Economics is a clear outlier.

A similar tale can be told for students who identify as belonging to ethnic groups such as Hispanic, Native American or African American. Such minority groups are less likely to receive economics degrees than degrees in other social sciences, humanities, business studies or Stem subjects. The same is true at doctoral level. There’s no getting around it: economics is, almost uniquely, the preserve of white or Asian men.

Why is this happening? One possibility is the self-perpetuating stereotype. As long as economics professors tend to be white men, women and minority students may feel that the subject just isn’t for them. Survey evidence suggests that women in the US are simply not as likely to find the subject of economics appealing.

There’s also the problem of implicit or explicit discrimination. Here, the evidence is mixed: such discrimination exists but it is not confined to economics. For example, one recent study (by Katherine Milkman, Modupe Akinola and Dolly Chugh) sent 6,500 emails to professors from a fictional student, requesting a 10-minute meeting to discuss applying to a doctoral programme. The emails were all identical except for the race or gender of the imaginary applicant. Generally, emails from white men were more likely to receive a response. But this was true for almost all subjects — economics was not particularly blameworthy.

On the other hand, a 2014 study, “Women in Academic Science”(by Stephen Ceci, Donna Ginther, Shulamit Kahn and Wendy Williams) found that academia had in recent years become a much more level playing field for women than was often claimed. But the study singled out economics as an exception, a subject where well-qualified and productive women continue to miss out on promotions.

If women or people from certain ethnic groups are being denied promotions because of discrimination, then clearly that’s unacceptable. But even if the main story is that economics is simply unappealing to women and to various ethnic minorities, that’s a concern. It’s important that everyone is represented in economics. Partly this is because economists tend to be well paid and at least somewhat influential. (You have to be a really prominent sociologist before the news networks want your opinion on anything much.)

Just as importantly, economics itself needs diverse views. Few endeavours ever benefited from an intellectual monoculture. That is an argument for mainstream economics to engage with other disciplines such as psychology, anthropology, evolutionary biology — and heterodox schools of thought such as Marxism and Austrian economics. But it’s also an argument to engage with people who may see the world differently because of their race, nationality, sexuality, disability or gender.

The obvious reason for this is that when a group of very smart people with similar perspectives find themselves stuck, someone who brings a new intellectual tool or a fresh perspective is far more valuable than one more smart guy in the same mould. (This argument has been brilliantly and rigorously explored by the complexity scientist Scott Page in his book The Difference.)

But there are subtle gains from diversity too. For example, when psychologist Samuel Sommers conducted mock jury trials of a black defendant, with some all-white juries and some racially mixed juries, he found that the mixed juries did a much better job of analysing the information placed in front of them. But this wasn’t just because the black jurors brought a fresh perspective. It was because the white jurors felt under pressure to think rigorously and to justify their views.

This is yet another reason why white guys like me should want to see more diversity in the subject that we love. It’s not just about being fair to everyone. And it’s not even just that people with different perspectives can enrich economics. It’s that when we’re forced outside our cosy group of people who think and look just like us, we become better people.

Written for and first published in the Financial Times.

My new book “Messy” is now out and available online in the US and UK or in good bookshops everywhere.

What I’m reading

Steven Johnson’s Wonderland is out this week. It’s vintage Johnson, a fascinating and surprising guide to the history of innovation. Johnson is focusing on the role of delight and play in producing new ideas – from the desire for Indian calico to the tinkering of MIT’s computer gamers. Strongly recommended. (US) (UK)

I loved Maria Konnikova’s The Confidence Game – Konnikova is a great storyteller and completely in touch with the psychological literature. This book explains why we fall in love with con-men. Topical. And it’s an excellent book. (US) (UK)

Dan Ariely’s new short book Payoff offer Dan’s usual combination of wit, wisdom and memorable experiments. I enjoyed it a lot. (US) (UK)

And one from the archives – I spoke on a panel last week with the remarkable Paul Seabright and was reminded just what a superb book Company of Strangers is. Seabright asks the question – how did we evolve from being violent, suspicious apes to a species capable of amazing feats of cooperation and trade? And how close to the surface are the violent, suspicious apes? (US) (UK)

A reminder that my new book Messy would make the perfect Christmas present – and that my new radio series Fifty Things That Made the Modern Economy is in full swing and you can subscribe, free, to the podcast if you wish.

Meanwhile, under the circumstances, you might want to invest in a copy of Leonard Cohen’s The Future (US) (UK) – “Give me absolute control over every living soul, and lie beside me baby, that’s an order…”

15th of November, 2016MarginaliaComments off
Previous

Elsewhere

  • 1 Twitter
  • 2 Flickr
  • 3 RSS
  • 4 YouTube
  • 5 Podcasts
  • 6 Facebook

Books

  • Messy
  • The Undercover Economist Strikes Back
  • Adapt
  • Dear Undercover Economist
  • The Logic of Life
  • The Undercover Economist

Tim’s Tweets

Search by Keyword

Free Email Updates

Enter your email address to receive notifications of new articles by email (you can unsubscribe at any time).

Join 3,476 other subscribers

Do NOT follow this link or you will be banned from the site!