Undercover Economist

The Doris Day effect – when obstacles help us

She had hoped to become a ballet dancer. After her leg was shattered in an accident at the age of 15, she took singing lessons instead. It was a striking detail in the obituaries. If not for that painful setback, the star that was Doris Day would never have risen.

Was the car accident that redirected her career an extraordinary twist in the story of an extraordinary life? Or was it typical of some broader truth about life, that frustrations can actually help us? Perhaps it is true that what does not kill us makes us stronger. It may, in contrast, be that what does not kill us nevertheless slows us down.

The conventional wisdom is that initial advantages tend to snowball into an avalanche of privilege. Sometimes this reflects genuine achievements: a bit of luck with an early teacher sharpens a student’s skills, lifting her into a higher set, which in turn gets her into a better university, then a job with more stimulating peers, and so on.

An egregious example, made famous by Malcolm Gladwell’s book Outliers, is the tendency of elite athletes to be born early in their school year. Being a few months older at the age of five means you are stronger and faster, are more likely to be picked for school teams, get more practice and are still reaping the benefits as an adult athlete. The effect is particularly well-studied among boys playing ice hockey in Canada, and football in a variety of countries.

At other times, well-deserved acclaim is followed by unearned praise. In academia this tendency was named by the sociologist Robert K Merton as “the Matthew Effect” in reference to a biblical verse: “For to every one who has will more be given, and he will have abundance; but from him who has not, even what he has will be taken away.”

If three researchers collaborate on a problem, and one of them already has a Nobel Prize, the laureate tends to earn disproportionate recognition for the joint work. When a teacher and a student work together, the senior researcher is cited because that name is already recognised. The junior is easily forgotten.

In the wider workplace, we have evidence that the luck of graduating in a benign economic climate can lead to a lasting advantage. One researcher, Paul Oyer, found that young PhD and MBA students who started off in favourable job markets were employed in better places with smarter colleagues, and were still doing better a decade later than those who started out in tougher times.

Hannes Schwandt and Till Marco Von Wachter studied the other end of the US labour market to find the story is even worse there: entering the job market during a recession damages anyone’s prospects, but the harm is deeper and lasts longer for less-educated and otherwise disadvantaged groups.

All this suggests that setbacks are setbacks: they drag us down, perhaps disproportionately. Doris Day was an exception, not the rule.

Yet a striking new study suggests that the Doris Day effect is quite real in one particular group of people: young scientists applying for research grants. Yang Wang, Benjamin Jones and Dashun Wang looked at scientists applying for funding from the US National Institutes for Health, with grants averaging $1.3m. In particular, they focused on borderline decisions, comparing those who scraped through to get a grant with those who just missed out. The near-winners and the near-losers were otherwise indistinguishable before the decision point, but afterwards it was the losers who prospered, publishing substantially more highly cited research papers.

We should remember that anyone in a position to nearly secure a million-dollar research grant has presumably enjoyed a few successes along the way at school and university. Failure at this hurdle may be described as an “early career setback”, but it is not comparable to the setback suffered by an undernourished two-year-old with no books in her bedroom.

Still: this is a counterintuitive finding. Yet I was not entirely surprised to encounter it. It may be that many people respond to a setback by bouncing back with renewed determination. It may also be that the failure provokes a rethink and a fresh course of action. Doris Day, after all, did not respond to a shattered leg by trying even harder to become a dancer. She changed her goals and prospered as a result.

We don’t have to be promising young scientists — or aspiring starlets — to benefit from having obstacles placed in our way. Something as mundane as a strike disrupting regular commuting has been shown to push people towards new habits. Three economists who studied data from London’s public transport network found that after a 48-hour strike in London in February 2014, thousands of commuters changed routes and never switched back. They discovered that they’d been doing the commute wrong their entire working lives.

Often failure is simply failure, and a setback is exactly what it seems. But sometimes the obstacle that has been placed in our path might provoke us to look around, and perhaps to discover that a better route was there all along.

Written for and first published in the Financial Times on 17 May 2019.

My book Messy explores some of these ideas in more depth – feel free to order online or through your local bookshop.

Free email updates

(You can unsubscribe at any time)