Tim Bradshaw, head of the Russell Group of leading UK universities, has a curious tale to tell about failure. A few years ago he visited the Cambridge office of an admired Japanese company to find them fretting about the success rate of their research and development. At 70 per cent, it was far too high: the research teams had been risk-averse, pursuing easy wins at the expense of more radical and risky long-shots.
The late Marty Sklar, a Disney veteran, once told me a similar tale — if his colleagues weren’t failing at half of their endeavours, they weren’t being brave or creative enough. My boss at the World Bank 15 years ago had the same worry that too many projects were succeeding.
When the same concern arises in such wildly different contexts, we may be worrying about a common problem: a systematic preference for marginal gains over long shots. It’s not hard to see why. It is much more pleasant to experience a steady trickle of small successes than a long drought while waiting for a flood that may never come.
While marginal gains add up, they need to be refreshed by the occasional long-shot breakthrough. Major innovations such as the electric motor, the photovoltaic cell or the mobile phone open up new territories that the marginal-gains innovators can then explore.
With this in mind, it’s hard not to sympathise with the UK Conservative party’s promise to establish “a new agency for high-risk, high-pay-off research, at arm’s length from government” — a British version of the much-admired US Defense Advanced Projects Research Agency.
Originally known as Arpa, now Darpa, it is most famous for creating Arpanet, the precursor to the internet. It also supported early research into satellite navigation and the windows-and-mouse system for operating a computer. And it helped to catalyse interest in self-driving cars. With successes like that, nobody seems to mind that Arpa’s failure rate is often said to be around 85 per cent. High-risk, high-pay-off indeed.
A collection of essays published recently by the think-tank Policy Exchange concurs that we need an Arpa for the UK. I’ve long argued for the importance of long-shots — the subtitle of one of my books is “why success always starts with failure” — so I can’t help but agree. Yet if this was easy, the UK would have an Arpa already.
At the casino it is easy to double the rewards by doubling the risk, but in the world of research, the trade-off is not so straightforward. While a low failure rate may indeed signal a lack of originality and ambition, we cannot simply decide to fail more often in the hope that originality will follow.
Arpa itself has approached this problem by hiring high-quality scientists for short stints — often two or three years — and giving them control over a programme budget to commission research from any source they wish.
Meanwhile, the Howard Hughes Medical Institute, a foundation, deliberately looks for projects with an unusual or untried approach, but a large potential pay-off. One study suggested that HHMI gets what it pays for — more failures, but larger successes, compared with other grant-makers funding researchers of a similar calibre.
Another large obstacle looms: how long will politicians find failure to be a sign of boldness and originality? Eventually, they will simply call it failure. Now that Arpa has a 62-year record, it is easy to forget that the agency was initially written off by some critics.
A new UK agency will face pressure to deliver. That sits uneasily with the desire to support risk-taking. Consider Arpa’s younger sibling, Arpa-E, created in 2009 to fund new energy projects. As of this week, the section of the Wikipedia entry on Arpa-E entitled “Accomplishments” is empty. Ouch.
The problem is more acute for a UK Arpa, because it is likely to have less funding — perhaps £200m a year. Is that enough? When Arpa’s head Charles Herzfeld heard the initial pitch for the proto-internet, in 1965, he responded, “Great idea . . . Get it going. You’ve got a million dollars more in your budget right now. Go.”
That is $10m-$30m in today’s money — depending on how one adjusts for spending power. It is hard to imagine a modern-day Herzfeld blowing a tenth of the UK-Arpa’s budget after a 20-minute meeting. We are on the Triceratops-horns of a trilemma. Be cautious, or fund lots of risky but tiny projects, or fund a few big, risky projects from a modest budget and accept that every single one may flop.
Keeping this new agency “at arm’s-length from government” is essential. Indeed, Safi Bahcall — the author of Loonshots — persuasively argues that such agencies need to be at arm’s length not just from government but from everybody. Yet somehow they must focus on real, practical, front-line problems. Not too close, not too distant. Not too many successes, but not too many failures, either. It’s quite a balancing act. Still, I’d pay for a ticket to this circus. Let’s give it a try.
Written for and first published in the Financial Times on 14 February 2020.
Catch up on the first season of my podcast “Cautionary Tales” [Apple] [Spotify] [Stitcher]