I am in the midst of revising a paper that uses a very specific question from the Fragile Families Data set about reading to children. When I began writing the paper, I started looking for evidence with time-use surveys, such as the American Time Use Suvey (ATUS) which asks participants to record everything they do and for how many minutes on two given days (a weekday and a weekend, usually). I noticed, particularly at the PAA meetings this Spring, that there was a lot of controversy about these surveys. What, exactly, can they tell us about general effects, when we are looking at such a small sample of time for any given individual? More specifically, if we want to examine the effects of a particular policy, how does looking at one individual’s day give us a causal effect of a policy? Time use surveys are incredibly useful for seeing exactly how individual spends his time on any given day, and the possibilities for understanding the dynamics of child-rearing and marriage are far-reaching. The trade-off is that you have no way of knowing whether this is a typical day or not. On average, for the population, if we have a random sample of individuals and days are sufficiently randomly assigned, we should get an idea of what the population does, on average. But asking if a particular impetus leads to a specific behavioral change (for instance, does an increase in income mean you invest more in child’s education) is a little more problematic. The alternative is to ask questions in a survey setting about time-use behaviors without specifying the time. That’s what the Fragile Families does, and the question about how many days per week you read with your child has its own problems. I have long argued that when individuals answer the question, they must do some averaging over time. The question is not “how many days did you read with your child last week” as might be preferred or indicated by the literature on work (did you work last week?), but rather a sort of what do you usually do? I’ve been surprised at how much pushback I’ve received on this matter from discussants and reviewers. Most say the natural model to use is a count model, like negative binomial or Poisson, but I think it makes more sense to use an ordered probit, which allows for 4 to be more than 2, but not necessarily twice as much as 2. I don’t think the reading days answer is as firmly countable and identifiable as something like parking tickets, where a count model is the readily apparent model. I imagine the question is a lot like exercise. Over the weekend, I helped a friend with her match.com profile and one of the questions is how many days a week do you exercise? For some, the answer is absolutely 7, every single day. For others, zero, not lifting a finger. For most, though, I’d guess it varies from week to week. One week, you go every day, the next week is busy at work, so you go less often. Perhaps you go on a whole-day hike and tell me two days instead of one because you don’t want to seem lazy. Thus, when I ask you the question of how many days a week you exercise, you’re not really giving me a straight answer, through no fault of your own. You’re averaging over the last couple of weeks, you’re perhaps adjusting your answer to reflect what you think the surveyor is looking for, and you’re partially giving an impression of how much you value exercise. I’m having a hard time making this same argument regarding time spent with children to discussants and reviewers, and I’m not sure what I’m missing in my explanation to make it more convincing.
One thought on “Time use and hindsight”
In my world, we would partially concede and tell the reviewers, “We did it our way and then we did it your way, and the results were not substantially different.” That seems to appease everyone. Of course this is not so simple if the results are indeed different, but such a difference might help you argue on behalf of your preferred method. Good luck 🙂