One of the primary problems with survey research is that it relies on recall of past events. In as much as humans are subject to forgetting things (and we are actually designed to forget things), asking someone about how often they perform an activity or how often something has happened in the past few weeks or how much they paid for something is problematic. This is before we even factor in the cultural norms and expectations around the behavior. We probably exaggerate the things we’re proud of of or that match social norms and downplay the incidence of events we’re ashamed of. Econometrically, we tend to say this kind of error is only a problem if it is systematic. That is, if some people overestimate and some people underestimate (with mean zero and some constant standard deviation), it won’t affect our estimates. However, if everyone underestimates, this causes our parameter estimates to be biased. In simpler terms, we don’t accurately assess the relationship between two variables because we’re missing a lot of information about at least one.
A paper explains this problem as it relates to diarrhea incidence recall by parents and definitions (which also gets me thinking about language, and education, but that’s another post or two or three).
Several methodological issues may have an impact on the incidence rates of childhood acute diarrhea reported by community-based studies. This study was performed to assess the impact of parental recall ability and definition of diarrhea on the estimate of incidence of acute diarrhea. Eighty-four children younger than 40 months were randomly selected and visited every other day for four weeks and the occurrence of diarrhea was registered. On the last day of the study, another visit was performed and the informants were inquired about the occurrence of diarrhea during the previous four weeks. Data gathered during the four weeks were compared to those obtained on the last visit. Additionally, the informants’ definition of diarrhea was investigated and compared to the one adopted by this study. During the observation period, 33 children suffered diarrhea, but only 10 (30.3%) informants reported the occurrence of diarrhea. Although 42.4% of those informants reported that their children had been ill over that period, they did not report diarrhea. Further, 60.6% children who had diarrhea suffered at least one episode in the two weeks prior to the visitation. The same definition of diarrhea used in this study was adopted by 52.1% of the informants inquired. Parental recall is an unreliable method to estimate the incidence of diarrhea and studies with a short interval between the visits should be necessary to correctly evaluate this important health problem. Moreover, assessing the informants’ own definition of diarrhea is a significant contribution to the interpretation of the results.
The rub is that we’re not very good at recalling past events, even when we’re being constantly reminded of them. As a separate, but related question, I wonder whether our ability to recall changes over time, or more specifically, over the course of an intervention. I wonder if the percentage of recall changes when you’ve been a recipient of an education program or a new latrine or whether that percentage stays constant. Depending on what the answer is, it could have a large impact on how we evaluate the effectiveness of health and sanitation interventions.
ERIN K. FLETCHER 