Illustration by Ingram Pinn
I don’t recall it myself, but like most babies born in 1973, I apparently slept face down in my cot. This was the standard advice, made famous by Benjamin Spock in 1948. We now know that for many unlucky families, this well-meaning advice was fatal. According to research published in 2005, putting babies to sleep on their fronts has led to about 60,000 cot deaths.
The story is a favourite of Sir Iain Chalmers, a campaigner for better standards of evidence in medicine and beyond. Because it is possible to do so much unwitting harm in medicine, many medical interventions are now subjected to a randomised controlled trial. Austin Bradford Hill performed the first properly controlled clinical trial in 1948, although he had predecessors, including James Lind, who used a randomised trial to show that citrus fruit prevented scurvy. There’s even a controlled trial in the Bible (Daniel 1:8). Such trials have proved the effectiveness of countless treatments, and the dangers of countless others.
It is a shame, then, that there is so little appetite from politicians for the same standards of evidence outside medicine. In fact it is more than a shame – it’s a scandal. While randomised trials are not going to tell us when to raise interest rates or get out of Afghanistan, there are many policies that could and should be tested with properly controlled trials. Is Jamie Oliver right to emphasise healthy school meals? Run a trial. Should young offenders be sent to boot camp, or to meet victims of crime? Run a trial. What can we do to persuade households to use less electricity? Run a trial.
Yet such trials are not common in the US, and downright rare in the UK. There is no financial, ethical or practical excuse for this. Trials are cheap. (Even if they were expensive, solid practical knowledge is well worth paying for.) This is not a question of carrying out dangerously speculative crank experiments, but simply adding the essential ingredient of randomisation to a standard pilot project that would have happened anyway. Randomising is often what distinguishes proper evidence from statistical mush, by removing biases in the setting of experiments – such as running pilots only in the most needy areas.
When the UK government recently introduced the “synthetic phonics” method of teaching young children to read, they were told by Carole Torgerson, an evaluation expert at the University of York, that they could easily bolster the slim evidence base by randomising which schools joined the programme first. They didn’t. (More encouragingly, Ms Torgerson has been commissioned to evaluate maths teaching.)
Some people feel queasy at talk of “experimentation” in the classroom, prison or benefit office – but politicians experiment on us all the time with their latest policy wheezes. We learn little or nothing because the experiments are badly designed.
What is missing is the political demand for tests of what really works. Too many policies on education, welfare and criminal justice are just so much homeopathy: cute-sounding stories about what works leaning more on faith than on evidence. Politicians and civil servants, faced with some fancy new idea, should get into the habit of asking for a proper randomised trial. And we, as citizens, should be equally demanding.
It’s no coincidence that one of the few fields of social policy to feature more than 100 robust trials is the study of how to get voters to turn out in elections. Politicians seem perfectly happy to turn to scientific method if it will get them elected. They are less interested in using it for the good of the people they govern.
It is embarrassing even to have to make the case for randomised trials in social policy. For medical researchers, such trials are just the start. Realising that inconvenient – or just plain boring – trial results are less likely to appear in print, medical journals now refuse to publish trials that were not logged before they started in a register of trials. Such registers ensure embarrassing results cannot be made to disappear. This is vital in medicine, and just as important in social policy.
Trial registers also feed into systematic review bodies such as the Cochrane Collaboration, which is an international offshoot of a National Health Service initiative. In less than two decades, the Cochrane Collaboration has published 4,000 systematic reviews of medical treatments, digging up data from unpublished trials, and providing the information to save many lives. A parallel body for social policy has far fewer trials to evaluate.
The Cochrane Collaboration has become a byword for a fair and comprehensive review of a treatment – the latest word (never the last word) about what works. We need the same shorthand in policy, a quality kite-mark that tells us politicians have actually done some homework before they roll out their latest brainchild. We’ve had FairTrade coffee – what about FairTest policies? Most voters don’t know much about randomisation or trial protocols, but they’ll know when they see the FairTest logo that a policy has had a proper, scientific test to see if it works.
Many social, educational and economic policies are the modern equivalent of Dr Spock’s advice that babies should sleep face down: well-meaning, authoritative – and wrong. No doubt it would be awkward to see the wisdom of experts punctured and the pet policies of politicians discredited on a regular basis. But if politicians really cared about those they represent, they would insist on more randomised trials and more systematic reviews of what works. Honest policy mistakes, quickly reversed, should embarrass nobody. As voters, we should demand more such mistakes.