Keith Jarrett’s 1975 concert in Cologne should have been a musical catastrophe. Owing to a string of mix-ups and bad luck, he was faced with the choice of attempting his widely admired improvisations on a beaten-up old piano with sticky keys and a harsh upper register — or walking out altogether. He was all for walking out, but felt sorry for the concert promoter and agreed to play the unplayable piano against his better judgment. The result was not a catastrophe but a masterpiece, and a bestselling one at that: The Köln Concert album.
I’m fond of that story, of the way that an obstacle can unleash a creative response — partly by concentrating the mind and partly by forcing the artist to explore fresh approaches. It’s not the only such tale. The great jazz guitarist Django Reinhardt developed his distinctive style after suffering severe burns that left him with only two fully functioning fingers on his fretting hand.
A quotidian parallel is the Tube strike of 2014, which partially shut down the London Underground, prodding several thousand commuters into finding better routes into work.
The three stories share another common element: nobody at the time would have predicted that anything good would come of the problem. Keith Jarrett could have sought out a bad piano; young Django could have tied two fingers together; commuters could have tried a new route any day. But they didn’t, until there was no choice. The blessings came well disguised.
David Epstein’s new book, Range, has brought other instances to my attention. Range is mostly about the benefits of being a generalist rather than a specialist, but a recurring theme is of teaching or training techniques that seem to fall flat but pay off in the long run.
For instance, in the US Air Force Academy, students are initially assigned at random to academic instructors, then later randomly reassigned to follow-on courses — a nice natural experiment. Scott Carrell and James West looked at the data and found snatches of unplayable piano in it: students assigned the “best” instructors for introductory courses, as measured by short-term pupil performance, earned the highest student evaluations but went on to produce the worst results in the long run.
In a nutshell, teachers who “teach to the test”, or otherwise provide simple problem-solving procedures, immediately improve grades and their students thank them for it. But in the long-term, the students suffer from not having been forced to think more deeply.
This isn’t just about pedagogical techniques. One study that stuck with me when I was researching my own book, Messy, was by Katherine Phillips and other psychologists. The researchers set small groups working on a problem before introducing a new team member to help, sometimes a stranger and sometimes a familiar face. The groups forced to work with the stranger were much more likely to solve the problem, but also enjoyed the experience less and sharply underrated their success. Groups of friends did much worse, but had fun and were under the illusion that they had done a good job.
Again the striking thing is not just that the obstacle turned out to be helpful, but that nobody thought so at the time. Only with hindsight did people realise.
Three more examples. Sports scientists now suggest that top endurance athletes should keep most of their training at low intensity, resisting the temptation to train too hard. Traffic planners are familiar with Braess’ Paradox: closing a road can and often does improve traffic flow, even if the number of journeys stays the same. The closure prevents drivers from choosing routes that make sense for them but cause congestion for others.
The inverse is even more common: a pleasing intervention that does no good. For instance, preliminary research suggests that using virtual reality to promote empathy — for example, for refugees, homeless people or those with disabilities — can promote a surge of emotional support in the short run without delivering any long-run insight.
We shouldn’t leap to the conclusion that every blessing comes well disguised, and that every problem is an opportunity. Sometimes blessings are perfectly apparent, and we should remember to count them. Sometimes a problem is simply a problem. But the prevalence of these counterintuitive results is a reminder that we know less about the world — and about our reaction to it — than we like to think.
I draw three lessons. The first is the need to gather solid evidence — for example, by running randomised trials of teaching techniques — and to ensure that the evidence looks at a good range of outcomes over a long enough time to be meaningful. Common sense can lead us astray, especially in situations where short-term pain leads to long-term gain.
The second lesson is to experiment with more variety in our own lives — whether choosing a holiday or a commuting route. It shouldn’t take industrial action to prompt us to try an alternative route to work.
The final lesson is the simplest. When life confronts us with an unplayable piano, perhaps we should sit down and try to play it.
Written for and first published in the Financial Times on 7 June 2019.
My book “Fifty Things That Made the Modern Economy” (UK) / “Fifty Inventions That Shaped The Modern Economy” (US) is out now in paperback – feel free to order online or through your local bookshop.