A few weeks ago, a paper was posted on the CSAE 2012 Conference website that seemed to fly in the face of much of the current research that is happening in development economics. The advent of RCTs (randomized control trials) brought about a significant change in the way we do policy analysis, but also in the costs of it. This paper suggested that RCTs were capturing placebo effects. Just like when people believe they are taking curative medicines, they feel better, so do those benefiting from RCTs experience placebo effects from knowing they are part of an experiment.
The answer, according to the researchers, is to conduct a double-blind experiment, where neither the researchers nor the participants whether they were part of the treatment or control.
The paper garnered a lot of attention early on. I noticed many colleagues and others had the immediate and short reaction of “wow” and “yikes”, and I wasn’t the only one. Berk Ozler, at the Development Impact Blog, has a good review of the paper up with a great, punny title. Among other problems:
First, it turns out that the modern seeds are treated with a purple powder in the market in Morogoro (to prevent cheating and protect the seed from insect damage during storage), so the experimenters sprayed the traditional seeds with the same purple powder. As you can immediately tell, this is less than ideal. First, as this is a not a new product, farmers in the blind RCT are likely to infer that the seeds they were given are modern seeds. Given that beliefs are a major part of the story the authors seem to want to tell, this is not a minor detail. Second, if the purple powder really does protect the seeds from insect damage, the difference between the MS and TS is now reduced.
Berk’s analysis is well worth a read. Kim Yi Dionne also addresses placebo effects, though a different paper.
Update: the original post said that this paper was forthcoming in Social Science and Medicine. This is not the case. Sorry for the confusion and thanks to Marc Bellemare for catching it.
Update #2: The Economist has a nice review of this paper up as well on the Free Exchange blog. It doesn’t touch most of the analysis issues, but it does explain well why double-blind experiments might not be useful in Economics. h/t @cdsamii
ERIN K. FLETCHER 