I am reading The New Harvest by Calestous Juma on how innovation and entrepreneurship need to be rethought in African agriculture. I can recommend the book—lively, provocative and well-evidenced. One of his key points is that agricultural innovation and knowledge generation in Africa needs to be decentralised. He argues that the interactions of climate change and ecosystems are expressed in variable and unpredictable ways and this makes context specificity more vital than ever. It is not that research and technology developed in one area will not be relevant in another, but just that we must continually challenge whether it will be. I agree with this conclusion.Over the weekend I also read a column by Ben Goldacre on his “fantasy” that UK public policy should be driven by the MIT Poverty Lab approach. I have not yet read the latest book from the Poverty Lab which he refers to, but it strikes me that the randomised controlled trial (RCT) approach is at risk of doing exactly the opposite of what Juma advocates.RCTs run the risk of locking in a result, not only within a country but across countries and over time. This is partly because: (a) they are costly to run (although I agree with Goldacre when he says you have to compare the cost with the costs of adopting the wrong policy) and so replication will be seen as costly and not terribly sexy to researchers or funders, (b) they are iconic (e.g. Progresa in Mexico, Orange Flesh Sweet Potato in Mozambique) because they are expensive and donors and researchers want to maximise their investments by publicising results (if they are positive), (c) they are not great at exploring the distribution of effects (sample sizes get too small if there are too many treatment variants) and (d) they are not particularly interested in external validity (understanding the likelihood of an interventions shown to be successful in one area being successful in another) because this requires a different set of skills.
It is interesting that RCTs have not been applied to UK public policy (I am assuming Goldacre is correct in this assertion). One could argue that heterogeneity (e.g. relative differences in behaviours and contexts by region and class) in the UK is lower than in emerging economies and hence the above worries about RCT portability are less valid. I wonder if this low RCT uptake in the UK is because of an anticipated stronger push back on ethical concerns (e.g. the challenge of getting informed consent when randomisation of treatment and control happens at the community or cluster level), or because of the political problems of doing such a pilot in the context of UK’s muscular media, or because UK communities would not put up with being seen as laboratory subjects.
Don’t get me wrong, I am not blind to the virtues of RCTs, I just don’t want to be blinded by them.