ABSTRACT
OBJECTIVE: To understand the effects of interviewers on the responses they collect for measures of food security, income and selected survey quality measures (i.e. discrepancy between reported Supplemental Nutrition Assistance Program (SNAP) status and administrative data, length of time between initial and final interview, and missing income data) in the US Department of Agriculture's National Household Food Acquisition and Purchase Survey (FoodAPS). DESIGN: Using data from FoodAPS, multilevel models with random interviewer effects were fitted to estimate the variance in each outcome measure arising from effects of the interviewers. Covariates describing each household's socio-economic status, demographics and experience in taking the survey, and interviewer-level experience were included as fixed effects. The variance components in the outcomes due to interviewers were estimated. Outlier interviewers were profiled. SETTING: Non-institutionalized households in the continental USA (April 2012-January 2013). SUBJECTS: Individuals (n 14 317) in 4826 households who responded to FoodAPS. RESULTS: There was a substantial amount of variability in the distributions of the outcomes examined (i.e. time between initial and final interview, reported values for food security, individual income, missing income) among the FoodAPS interviewers, even after accounting for the fixed effects of the household- and interviewer-level covariates and removing extreme outlier interviewers. CONCLUSIONS: Interviewers may introduce error in food acquisition survey data when they are asked to interact with the respondents. Managers of future surveys with similarly complex data collection procedures could consider using multilevel models to adaptively identify and retrain interviewers who have extreme effects on data collection outcomes.
ABSTRACT
Background: Food acquisition diary surveys are important for studying food expenditures, factors affecting food acquisition decisions, and relations between these decisions with selected measures of health (e.g., body mass index, self-reported health). However, to our knowledge, no studies have evaluated the errors associated with these diary surveys, which can bias survey estimates and research findings. The use of paradata, which has been largely ignored in previous literature on diary surveys, could be useful for studying errors in these surveys.Objective: We used paradata to assess survey errors in the National Household Food Acquisition and Purchase Survey (FoodAPS).Methods: To evaluate the patterns of nonresponse over the diary period, we fit a multinomial logistic regression model to data from this 1-wk diary survey. We also assessed factors influencing respondents' probability of reporting food acquisition events during the diary process by using logistic regression models. Finally, with the use of an ordinal regression model, we studied factors influencing respondents' perceived ease of participation in the survey.Results: As the diary period progressed, nonresponse increased, especially for those starting the survey on Friday (where the odds of a refusal increased by 12% with each fielding day). The odds of reporting food acquisition events also decreased by 6% with each additional fielding day. Similarly, the odds of reporting ≥1 food-away-from-home event (i.e., meals, snacks, and drinks obtained outside the home) decreased significantly over the fielding period. Male respondents, larger households, households that eat together less often, and households with frequent guests reported a significantly more difficult time getting household members to participate, as did non-English-speaking households and households currently experiencing difficult financial conditions.Conclusions: Nonresponse and underreporting of food acquisition events tended to increase in the FoodAPS as data collection proceeded. This analysis of paradata available in the FoodAPS revealed these errors and suggests methodologic improvements for future food acquisition surveys.
