One of the world’s largest ever randomised trials… the results are in
The Democratic Republic of Congo is the poorest country on earth. It suffered what was probably the deadliest war since 1945, and according to Margot Wallström, the UN’s representative on sexual violence, it is the “rape capital of the world”.
Nevertheless the worst of the conflict is over and there is hope for the future of the DRC. Development projects are under way, but in a near-landlocked country that is two-thirds the size of western Europe, such projects aren’t easy to run.
One of them is Tuungane (“Let’s Unite”), a programme funded by the UK’s Department for International Development to the tune of £90m. The first £30m, Tuungane I, funded classrooms, clinics and other investments in 1,250 villages with a total population of almost 1.8 million people.
If the project is ambitious, what’s really fascinating is that Dfid has tried to evaluate Tuungane I rigorously, using a randomised controlled trial. Villages were enrolled in Tuungane through a public lottery. With 1.8 million people in the treatment group and a large control group, such an evaluation would be challenging to conduct in a rich country. In the DRC, where enumerators risk death and must wade through water up to their necks to reach the villages they are surveying, it’s mind-blowing.
Tuungane is a community-driven development project, or CDD. CDD is fashionable in development circles these days. The idea is that communities receive cash from donor agencies on condition that they come up with appropriate institutions for deciding how to spend the money. CDD projects will often insist on standards of transparency, democracy, or gender balance. The hope is that not only will the money be spent well, but that it may also catalyse accountable local institutions.
But does CDD deliver on this? The Tuungane I evaluation has now been completed by a team from Columbia University. The evaluation was based on a second project, Rapid, which was separate from Tuungane. Rapid handed out cash with few strings attached to Tuungane and non-Tuungane villages. The evaluators observed what happened: was the money embezzled, or well spent, and how were the decisions reached? This was the acid test of whether Tuungane had helped to promote effective village institutions.
The good news is that the Tuungane project was well executed and the money typically arrived where it was supposed to. In the DRC this is no small achievement.
Yet the results fall short of the CDD hype. The evaluators found that the Rapid cash grants were reasonably well spent whether or not a village had previously been exposed to three years of community building through Tuungane. Local institutions were more accountable than one might expect, but there was no sign that Tuungane could take any credit for this. Neither did Tuungane projects display any economic returns – although arguably there was too little time between spending the cash and evaluating the results for such returns to become apparent.
It’s a shame that the results haven’t matched the expectations of the CDD optimists, but it’s hardly a surprise. Tuungane is large in absolute terms, but it’s just a few pounds per person. It would be odd to expect miracles. The safe arrival of cash in the hands of very poor people is surely worth celebrating.
Still, because Tuungane was rigorously evaluated, even an unmitigated failure would have produced valuable information about what works and what does not. It is far too common for development projects simply to list their inputs: how much money was spent, and on what. To see a discussion of what happened is rarer; to see rigorous causal evidence assembled is rarer still. It should be applauded.
First published in FT Magazine.
EDIT: This should be implicitly obvious but I should make it explicit: randomisation took place on a village by village level, so although there were almost 2 million people in the treatment group there were only 280 distinct areas evaluated in the treatment group, and 320 in the control group.