Addressing recall bias in (post-)conflict data collection and analysis: lessons from a large-scale health survey in Colombia.

BACKGROUND: Much applied research on the consequences of conflicts for health suffers from data limitations, particularly the absence of longitudinal data spanning pre-, during- and post-conflict periods for affected individuals. Such limitations often hinder reliable measurement of the causal effects of conflict and their pathways, hampering also the design of effective post-conflict health policies. Researchers have sought to overcome these data limitations by conducting ex-post surveys, asking participants to recall their health and living standards before (or during) conflict. These questions may introduce important analytical biases due to recall error and misreporting. METHODS: We investigate how to implement ex-post health surveys that collect recall data, for conflict-affected populations, which is reliable for empirical analysis via standard quantitative methods. We propose two complementary strategies based on methods developed in the psychology and psychometric literatures-the Flashbulb and test-retest approaches-to identify and address recall bias in ex-post health survey data. We apply these strategies to the case study of a large-scale health survey which we implemented in Colombia in the post-peace agreement period, but that included recall questions referring to the conflict period. RESULTS: We demonstrate how adapted versions of the Flashbulb and test-retest strategies can be used to test for recall bias in (post-)conflict survey responses. We also show how these test strategies can be incorporated into post-conflict health surveys in their design phase, accompanied by further ex-ante mitigation strategies for recall bias, to increase the reliability of survey data analysis-including by identifying the survey modules, and sub-populations, for which empirical analysis is likely to yield more reliable causal inference about the health consequences of conflict. CONCLUSIONS: Our study makes a novel contribution to the field of applied health research in humanitarian settings, by providing practical methodological guidance for the implementation of data collection efforts in humanitarian contexts where recall information, collected from primary surveys, is required to allow assessments of changes in health and wellbeing. Key lessons include the importance of embedding appropriate strategies to test and address recall bias into the design of any relevant data collection tools in post-conflict or humanitarian contexts.

Determining the effect of environmental accidents on responses to a Gulf of Mexico recreational for-hire fishing industry survey.

A survey designed to collect economic, attitudinal and policy data from the recreational for-hire (RFH) fishing industry in the U.S. Gulf of Mexico was conducted before and during the largest marine oil spill in U.S. history (the April 2010 Deepwater Horizon blowout). Respondents were grouped into two time periods based on when the survey was completed, where the break in groups was determined through the examination of the Pew Research Center's media coverage index and the per cent of fishing area closures due to the oil spill. A logistic regression was used to test variables that might predict the time period of a response. Results indicated that recall bias was not present in the financial variables examined, but that firm operating and demographic characteristics (i.e. vessel size, annual number of trips, number of vessels operating in the firm, tenure and household income) were significant in explaining the time period in which surveys were completed.

Evaluating recall bias in a case-crossover design estimating risk of injury related to alcohol: data from six countries.

INTRODUCTION AND AIMS: Prior work suggests that recall bias may be a threat to the validity of relative risk estimation of injury due to alcohol consumption, when the case-crossover method is used based on drinking during the same six hours period the week prior to injury as the control period. This work explores the issue of alcohol recall bias used in the case-crossover design. DESIGN AND METHODS: Data were collected on injury patients from emergency room studies across six countries (Dominican Republic, Guatemala, Guyana, Nicaragua, Panama and Canada), conducted in 2009-2011, each with n ≈ 500 except Canada (n = 249). Recall bias was evaluated comparing drinking during two control periods: the same six hours period the day before versus the week before injury. RESULTS: A greater likelihood of drinking yesterday compared with last week was seen using data from the Dominican Republic, while lower likelihood of drinking yesterday was found in Guatemala and Nicaragua. When the data from all six countries were combined, no differential drinking between the two control periods was observed. DISCUSSION AND CONCLUSIONS: These findings are in contrast to earlier studies showing a downward recall bias of drinking, and suggest that it may be premature to dismiss the last week case-crossover method as a valid approach to estimating risk of injury related to drinking. However, the heterogeneity across countries suggests that there may be some unexplained measurement error beyond random sampling error.