Measuring Difficult Things

I’m super excited to be giving a talk on Wednesday at the One Earth Future Foundation, a local, homegrown, interdisciplinary think tank here in Colorado working on all sorts of issues related to sustainability and making sure that we can keep living on this planet.

Because I don’t get to give talks very often, I may have gotten a bit excited and planned to talk about, well, basically everything I have done the past few years (and frankly a lot that I’m hoping to do in the next few!). The abstract is below. Really looking forward to a fun discussion!

Measuring Difficult Things

Social norms, unobserved characteristics, and hidden statuses can exert significant effects on outcomes ranging from domestic violence to armed group participation to social capital and network formation. In quantitative analysis, not being able to account for these factors can lead to biases in our estimation and thus incorrect or incomplete policy prescriptions. Even when we are able to see that these factors exist, finding a way to include them in our estimation is often limited by issues of measurement. In this talk, I give an overview of some my research on how to measure difficult things, beginning with social norms and intergenerational effects in a study of refugees, list experiments, and how to generate representative samples of hidden populations. I explore challenges in measurement, offer lessons from the field, and describe prospective work that aims to continue deepening our understanding of how to approach measuring difficult things.

Just how much do adults influence their kids?: Nyarugusu edition

In my first paper with Seth Gitter and Savannah Wilhelm on refugees in Tanzania’s Nyarugusu Camp, we showed that, among child respondents, there appears to be a strong social norm around reporting violence at school. Children believe their peers will report victimization, kind of regardless of the things we might think would matter. Boys and girls think their peers would report at roughly the same rate, and the type of violence doesn’t seem to change their perceptions much either.

A natural question to come out of this is whether these attitudes and perceptions are coming from their parents. Luckily, I designed this survey exactly to get at that kind of question. A novel and important part of this study was to survey linked child-parent pairs, so we can compare children’s responses to their parents. We have a second paper, now a Towson University working paper, out to deal with just that question. Here we’re still using the vignettes (one-line short stories of hypothetical situations), so there is some measure of uncertainty given the randomization of the vignette characteristics, and we’re still concerned with reporting norms, not direct victimization, but we have some pretty interesting findings.

First, parents’ and kids’ answers are positively correlated, so while we cannot necessarily identify the mechanism, there does appear to be some intergenerational transfer of attitudes or social norms.

We use the error terms from the parent regressions (basically the variation in the observations that cannot be explained by the vignette characteristics) to represent this sort of nebulous “parent” effect. When we put these error terms into the kid regressions, there is a large and statistically significant effect on those terms. In other words, the unobservable stuff that determines a parent’s answers is clearly and significantly related to his or her kid’s answers, controlling for both the vignettes that children hear and their demographics.

But they’re not perfectly correlated. We wouldn’t expect them to be, given the vignette randomization, so we rely on grouped vignette characteristics to tell us a little more. Parents saw some violence type distinctions where kids did not; sexual violence (teasing or touching) was perceived as more likely to be reported than other types of violence.

Finally, the specific way the surveys were set up allow us to dig a little deeper into parent thinking. Parents were screened at the beginning of interviews and asked to identify and give consent for one of their children to be interviewed immediately after their own survey. Largely, this was the child in the age range who happened to be at home and we believe this to be relatively random (school is half day, most adults had several children in the age range of both genders, parents were chosen via random walk, etc.). Gender and age are balanced for the child sample.

So, given that the child was selected before any data collection took place, we examine the data to see if parents answer vignettes differently depending on gender of the victim, conditional on the gender of the child who was interviewed after them. Spoiler alert. They do. Parents who selected a boy child to be interviewed and heard vignettes about boys were more likely to think the hypothetical victim would report violence. This isn’t true about girls. We think this effect might be something like priming, that the child to be interviewed might be at the top of the parent’s mind while he/she is taking the survey.

Seth and I will be presenting this paper at the MIEDC conference at the University of Minnesota at the end of April. We’re super excited to share it with everyone!

Pop the Champagne: An ode to great coauthors

Yesterday, I took a quick career mentoring phone call with a post-doc at CUNY Futures Initiative. After I had given her the rundown of what I presume to be my extremely fascinating job and life, the post-doc asked me how I managed to keep one foot in academia with everything else I had going on.

I have great coauthors, I told her.

And it’s so true.

Yesterday, I received notification that the first paper using the data I collected with the IRC in Tanzania’s Nyarugusu refugees camp was accepted for publication at Migration Letters. Time to break out the champagne, but first, lemme go update my CV real quick…

K, I’m back.

As background, the IRC hired me in 2015 to conduct a two-country, two-stage investigation of social norms of violence in schools in refugee camps. We started with focus groups and key informant interviews with Burundian and Congolese refugees in Nyarugusu camp in Tanzania, displaced Iraqis in Arbat IDP camp in Sulaymaniyah, Iraq, and out-of-camp Syrian refugees in urban Sulaymaniyah. These results were used to inform a survey of 300 randomly selected child-parent pairs in Nyarugusu. We were unable to complete the second phase of research in Iraq.

The survey covers lots of things, but specifically in this newly accepted paper, with Seth Gitter at Towson University and his senior thesis student, Savannah Wilhelm (now all graduated and grown-up, *sniff*), we examine norms around reporting violence to authorities and characteristics of children that may predict reporting in Nyarugusu. I love these data for many reasons, not just because they’re my baby, but because I think they’re a unique and novel contribution to an understudied field.

We survey young kids in an environment where very few large-scale surveys have been done (we have ~300 child-parent pairs, though this paper only looks at the children’s responses); we talk about violence in ways that are designed to mitigate retraumatzation and reduce under-reporting; and the findings are really interesting. Gender, either of the respondent or the hypothetical victim, has no significant bearing on the willingness of kids to report violence. In an extremely unstable, vulnerable environment where kids have fled violence, or grown up with stories of their parents fleeing violence, there appears to be a normative expectation that kids report victimization, and they feel comfortable doing that at school.

I think that’s huge.

This is also the first paper for which I’ve gone through the whole process with a student co-author, which was extremely rewarding. I honestly cannot gush enough about how great it’s been to work with Seth and Savannah (and now Seth and Lauren, and hopefully more to come!). Abstract is below. A second paper using the parent responses as well is in the works and hopefully we’ll get to share that soon, too.

Reporting Violence Against Children: Social Norms in Nyarugusu Refugees Camp

There is substantial evidence to show that the two million children living in refugee camps are vulnerable to violence, although little is known about under what circumstances children will report that violence, complicating efforts to reduce vulnerabilities. We presented 300 children in a Tanzanian refugee camp (Nyarugusu) with hypothetical vignettes regarding a victim’s response to violence. Vignette characteristics were randomized (e.g. victim, perpetrator, and location) to test what factors influenced reporting. Respondents believe the victim was more likely to report violence at school or adolescent perpetrators. Surprisingly, we find no substantial difference based on victim or respondent’s gender.

On Giving Tuesday, Please Remember Refugees


Friends and colleagues often ask me questions like, “should I give money to X organization that you have worked with?” and “if I want to support refugees and displaced persons, where should I give?” Since Thanksgiving is behind us, but Giving Tuesday is upon us, I thought I would take the opportunity to share the answers that I often give to these questions, starting with my quintessential “do your research” mantra.

More personally, last week, one of my refugee staff who had been relocated to the US was faced with eviction. My sister and I got a Friday-midnight text message from my friend obliquely explaining the situation. My sister sprung to action on Saturday morning and I have cried at least three times reading the donations list as high school friends and colleagues donated, and then people who follow me on twitter but I’ve never met in real life, and then people who I’ve never even interacted with on twitter opened up their wallets. Y’all are wow, just wow. I do not know how to begin to thank you and hope that my heart hugs reach you from way over here in the Mile High City. I know Eloco’s heart is also soaring.

Wherever and whether you decide to give, or not, on Giving Tuesday or any other day, I hope your Thanksgiving was filled with light and love and that it follows you into a bright, shiny New Year. This is my small way of paying it forward. Here we go.

  1. First and foremost, do some research. Before donating, read up on organizations through charity raters like Charity Navigator or the Catalogue for Philanthropy (DC-area only) or GiveWell. Newer, smaller, and local organizations (both in the US and in camps or other countries) may not be on lists like these, so take their recommendations with a grain of salt, but it’s always good to know what the consensus is on how organizations are using their money from available information and neutral (or as neutral as possible) evaluations. Keeping up on what the charities you want to support are doing takes precedent over all of what I have to say here.
  2. Decide what’s important to you and find an organization that does that. Do you want to support refugees in America? Or in camps? Or out-of-camp refugees? Specific camps? Specific populations? Do you want to support GBV or anti-violence programming or education or safe spaces for children or access to clean water? Do you want your money to go to immediate, need-action crises like the Rohingya or Mediterranean crossings? Or long-lasting, ongoing crises like the displaced Congolese or Somalis?
  3. Or just go with a big organization that does everything. There is plenty of need, so if you don’t have a specific cause and just want to help refugees, consider giving to a big organization that will put your gift towards where they need the most help. Big organizations, like the IRC or Oxfam or other groups that help resettle refugees, will be able to move funds around and devote some of their budget to higher-level advocacy and advertising, which can multiply the impact of your gift.
  4. Balance local and big-organization giving. If you live in New York or Boston or DC or Florida (or anywhere else with a large, existing PR community right now), you are likely seeing an influx of Puerto Ricans into your communities. With ongoing power outages and food and medicine shortages, many Puerto Ricans are moving to the mainland and they need all kinds of support. Local organizations that are on the ground are well positioned to assess their needs and provide those resources. Similarly, local organizations will have a good idea of which populations are already in your community and know what their needs are. There are almost certainly refugees from other countries being relocated in your community (and have been for some time); find out who is supporting them and feel free to contact them, be it New American Pathways, the IRC, or a small local church . For instance, Colorado has long had large Vietnamese and Russian refugee populations, but more recently seen influxes from Iraq, Burma, and Bhutan. I just learned about Street Fraternity Brotherhood in Denver, which works with young male immigrants and refugees, and while I cannot personally vouch for their work, I’m totally behind the idea. If you’re looking for a small organization on the ground in a camp or in a non-camp situation where there are lots of refugees, these exist, too, but go through contacts in-country to find them and make sure they money is going to where you think it’s going.
  5. Resist bandwagoning. Remember when the ACLU was inundated with donations earlier this year with the announcement of the refugee ban? It created awesome momentum and great press (and most accounts suggest that the ACLU has put that money to good use), but small organizations often cannot use all of those funds efficiently. If you’re inclined to donate to a particular cause that is getting a lot of attention, consider finding a smaller organization that works toward the same or similar cause but doesn’t have the big name or fundraising apparatus.
  6. Yeah, yeah, Erin, how about specifics? Okay, fine. (COI note: I do not benefit personally from any of these organizations, but I mention them because I have witnessed their work first-hand or know someone integral to their programming.)
    1. Upwardly Global works with highly educated immigrants (not just refugees) who are searching for opportunities that better match their skills in the US. Economists should love this one (reducing search and job mismatch frictions!)
    2. IRAP works across the country with volunteer lawyers to provide legal support for refugees.
    3. The IRC supports primary and secondary schools, marginalized populations, gender-based violence awareness and reporting, as well as high-level advocacy and resettlement in the US.
    4. MSF is always there, in the worst, most dangerous of situations.
    5. Oxfam provides clean water and sanitation education in many camps.
    6. UNFPA The family planning arm of the UN doesn’t solicit donations on the scale that UNHCR or UNICEF do (both of which do lots of work with refugees generally and education specifically and would also happily take your money), but I have been consistently impressed working with UNFPA at the caliber of their staff, their commitment to evidence-based programming, and their drive to reach the most vulnerable girls, wherever they happen to be working.
    7. Andi Leadership for Young Women supports peacebuilding and conflict resolution across cultures.
    8. “Every refugee in Atlanta comes through Clarkston at some point,” or so says my ATL-bound sister. Clarkston Community Center outside of Atlanta, GA provides bicycles, and computing and language classes in a community where many refugees first land. Clarkston Community Health Clinic offers free health services to the refugee population there, too.
    9. If you’d like to help more organizations that are on the ground and doing pathbreaking work, consider supporting an organization that identifies and supports social innovation start-ups like The Resolution Project or Echoing Green Fellowship. You may not be able to direct your money towards refugees in particular, but you will support social innovation around the world and beneficiaries are often working with refugees due to the increased publicity around displaced populations.
    10. Did you know that the CDC works on immigrant and refugee health? If you’re looking for a low-cost option to help, call your congresspeople and encourage them to support funding for the CDC and population-based surveys like the Census to make sure we know how best to serve these populations.
    11. Obviously, there are tons more organizations that work in camps and as resettlement organizations and to support refugees here and abroad. This list should not be considered exhaustive in any way nor should it be used to dissuade you if you were going to go with someone else. Also, if I failed to include an organization here, it probably just means I am not personally familiar with their work.

Last but not least, Eloco’s family is still in need. If that story in particular resonates with you and you’d like to help an individual or a family, let me know. I would be happy to introduce you to a number of refugees all over East Africa and in America who are constantly asking me to “keep them in mind” and to “search for a benefactor” for them. One trying to set up a milk business in Burundi, another setting up rotating savings and credit associations for war widows in Nyarugusu camp, another trying to go to school in Uganda, another trying to get into a Ph.D. program in Cameroon, etc.

All my love and gratitude.

Of geniuses, violence, and girls

Today is a pretty special day. I woke up this morning to emails and tweets alerting me that one of my favorite people, a senior scholar whom I feel so lucky to have met and been mentored by and to call a friend, won a MacArthur genius grant. I am of the firm belief that there is no one more deserving. So much of my own work has bits of Betsy in it and I know we’re all so excited to see what she does next.

I also got word this morning that one of my staff from Nyarugusu gave birth to a beautiful baby girl. After several miscarriages and almost losing her life in a previous pregnancy, all are healthy and happy, and I’m bursting with joy for them.

For all these reasons, plus it’s International Day of the Girl, it seems like a particularly appropriate time to share a paper that is forthcoming in JDS, written with my colleague and dear friend, Shatanjaya Dasgupta: Paying for Violence? Spousal Abuse and Son Preference in India.

The abstract is below, but the paper rests on a few key ideas:

  1. Son preference in India is well-documented and leads to disadvantages for girls, including in nutritional status.
  2. In the data, girls tend to be worse off than boys in India on average. Kids of any gender in families with parents reporting domestic violence or spousal abuse tend to be worse off than kids in families where it’s not reported.
  3. But, and here’s the kicker, girls in families where DV is reported are better off than the boys in those families, conditioning on parents’ stated preferences for boy and girl children.
  4. Models of domestic violence in economics have sometimes tried to show how interpersonal violence can be used instrumentally, by the abuser as a form of punishment, or by the abused who accepts abuse in exchange for enacting their own preferences.

Ultimately, we ask whether a model of spousal abuse can explain how mothers who want to give their children more equal treatment or even favor girls accept abuse in exchange for enacting those preferences. This “paying for violence” shows up in children’s nutritional status. We even wrote a (toy) model.

Guys, *I* wrote a model! (I’m pretty proud of myself).

The data are a bit sparse and we lack strong causal identification, but the puzzle itself is interesting and we think our answer is plausible, that domestic violence can have differential effects on children when mediated by parental preferences. It’s important to note that we’re not advocating domestic violence as a tool to improve girls’ lots, but it is worth thinking about, and studying further, how marginalized and abused invidviduals cope and carve out space for themselves to act in the face of adversity.

We find a puzzling correlation in the data on domestic violence and children’s outcomes in India. Using the 2005–2006 National Family and Health Survey, we see that girls in families experiencing spousal violence are less worse off than boys when only fathers report a son preference while the gender bias reverses when only mothers report having a son preference. To shed light on the puzzle in the data, we present a non- cooperative theoretical framework based in economic theories of domestic violence, whereby differing parental son preference and bargaining over investments in girl and boy children potentially explains the observed relationship.

A Cesspool of Misogyny

I spent my first Economics job market in my hometown of Denver, running from interview to interview, trying to meet the Becker student who wrote a similar job paper to mine, networking with colleagues, and looking around corners, highly suspicious that every man I met had been one of those who hurled tens, if not hundreds (I permanently logged off before I could see whether there could be hundreds), of misogynistic and violent comments at me via an online forum for economists the week before.

It’s not an ideal way to look for a job, to be sure.

Alice Wu, an undergrad at Berkeley and future Harvard grad student, has now confirmed what all the women in my field already know, that Econ Job Rumors (I’m not even going to link to it, you can feel free to google it if you are so inclined) is a “cesspool of misogyny” (thank you, David Romer, for the concise description).

I made the mistake of going online before the job market thinking I would get information. I didn’t yet understand the intricacies of the timing of the job market, of who contacts whom and when, and I’d had luck getting telemark skiing advice online, why not some job-seeking advice? It was a mistake to start, it was a bigger mistake to try to engage the commenters in the “who is the hottest job market candidate?” thread.

Yes, that’s a real thing.

The worst men seem to get from that site is “regression monkey,” but women are “sluts,” “prostitutes,” “hot,” “lesbians,” “feminazis,” and I don’t want to go on. NYT has a fuller list.

A few weeks ago, I was on a date with a management professor. We easily fell into stimulating conversation about about research and teaching and the bureaucracy of universities, but, incredibly, he kept coming back to the fact that I, as a woman, was a victim of economics’ toxic environment for women.

I get it; I was there. I left, remember? It was not the whole reason for my leaving academia, but it certainly played a part.

I left despite the fact that I have benefited enormously from the groundbreaking of women who went before me, from the mentorship and support of many successful women in my field, including advisors Terra McKinnish and Ann Carlos; CU professors Francisca Antman, Tania Barham, and Carol Shiue; senior colleagues Jean Fletcher, Susan Averett, Laura Argys, and Eileen Stillwaggon; post-doc supervisor Rohini Pande and others on her team like Rema Hanna, Lena Edlund, and Erica Field; junior colleagues Lena Ogrokhina, Mandy Pallais, Simone Schaner, and Amber Peterman; and women from my PhD cohort, Yiqing Xie, Debbie Baker, and Shatanjaya Dasgupta; and program, Lauren Calimeris, Mariya Burdina, and Christina Peters. Outside my field but still within academia, I found Betsy Levy Paluck and Katina Rogers and Jeannie Annan and Stephanie Schwartz and Zoe Marks and Ruth Carlitz and damn, am I lucky.

I’ve also been the recipient of advice and mentorship from wonderfully supportive men in my field including, Ed Tower, Alan Kelley, Tom Nechyba, and Charlie Becker at Duke; Randy Walsh, Murat Iyigun, Edward Morey, and Jeff Zax at CU; Ed Gamber at Lafayette; Ryan Dodd and Rim Baltaduonis at Gettysburg; and Michael Callen at Harvard. I could not have gotten to where I am without my once-twitter now-real-life friends, Marc Bellemare and Seth Gitter, or my economics writing mentors, Francisco Toro and Toby Bottome.

But for each of them, there was a man who told me I wasn’t cut out for graduate school. A man who thought I disrespected him. A man who demeaned me in front of colleagues. A man who thought it was okay to follow me back to my conference hotel room uninvited. A man who yelled at me in front of students. A man whose “question” following my conference presentation included a sexist joke. A man who decided my hesitancy to take on independent study students was disqualifying. A man who told me that coauthoring with women was a crutch and would get me nowhere. A man who thought I was too uppity. A man who thought I was too emotional. A man who told me that what I study is not economics. Actually, there are lots of men who tell me that I what I study is not economics. And there are plenty of men who go online to say things they wouldn’t dare say to my face.

This is not an easy time to play the victim. I’m still white and I still benefit from that enormous privilege. There aren’t men chanting in the streets with tiki torches that I should go back to where I belong or that they want to cleanse my kind so they can have a homeland or calling me names.

But they are doing it online. And ultimately, it sucks for the field of economics. It makes our ranks weaker when we weed out intelligent, thoughtful women and people of color with fresh ideas and perspectives and the ability to ensure that economics stays relevant, inclusive, and useful. Women continue to fall off the academic ladder at much greater rates than men in economics, and many departments are known as particularly unfriendly to women. Online harassment is perhaps not the cause, but it is a symptom, and it has very real consequences.

After my first year of teaching at Gettysburg, I was congratulated and told that more women had signed up for the major than in years past. I can’t claim strong causal identification of course, but hearing that made me feel proud, that my presence and my teaching could open up the field to more women. In leaving academia, I fear that I have failed to deliver on that continued promise of presenting an approachable female economist role model, that young women could see themselves in me and want to pursue more study in the field.

I commend my colleagues still leading that charge. What you feel is real–we now have the evidence in case you were doubting–and I’m here if you want to rage about it.

Ending Child Marriage

I remember an interview, probably not too long ago, when someone asked what my professional goals were.

“To end gender-based violence,” I said, without hesitation.

Thankfully, I’m often involved in work that actively makes headway on that goal, and every once in awhile a concrete piece comes out of it.

Last Fall, I spent a week in Bangkok, Thailand with UNICEF and UNFPA staff from all over South Asia, as well as invited guests from academia, other multilateral instituions, and INGOs working on child marriage. It’s always fascinating for me to join these conversations, an academic inside a practitioner’s reality. For me, the most striking thing to come out of this meeting was the appetite for evaluation. Practitioners all over this space want to know if what they’re doing is effective, is efficient. Subsequently what is striking is the lack of organizational incentives, technical expertise, and resources to support better and more evaluation. One enduring thought I have from working with various UN agencies over the years is that there is so much potential for learning through fostering greater cooperation with academics and those with more time and resources to invest in evaluation.

I think the best parts of this report are the group picture (see if you can find me!) and the background paper, which begins on page 46 as an annex. The background paper is what I presented at the expert group meeting and it made for some lively discussion. I hope we can continue these conversations about how to better and more creatively use data, analysis, and yes, even econometrics, to understand how programming works, how to tweak it, how best to allocate scarce funding, and to ensure that programs benefiting women and girls around the world are contextual, effective, and rigorous.