More is not always better: An experimental individual-level validation of the randomized response technique and the crosswise model.

Social desirability and the fear of sanctions can deter survey respondents from responding truthfully to sensitive questions. Self-reports on norm breaking behavior such as shoplifting, non-voting, or tax evasion may thus be subject to considerable misreporting. To mitigate such response bias, various indirect question techniques, such as the randomized response technique (RRT), have been proposed. We evaluate the viability of several popular variants of the RRT, including the recently proposed crosswise-model RRT, by comparing respondents' self-reports on cheating in dice games to actual cheating behavior, thereby distinguishing between false negatives (underreporting) and false positives (overreporting). The study has been implemented as an online survey on Amazon Mechanical Turk (N = 6, 505). Our results from two validation designs indicate that the forced-response RRT and the unrelated-question RRT, as implemented in our survey, fail to reduce the level of misreporting compared to conventional direct questioning. For the crosswise-model RRT we do observe a reduction of false negatives. At the same time, however, there is a non-ignorable increase in false positives; a flaw that previous evaluation studies relying on comparative or aggregate-level validation could not detect. Overall, none of the evaluated indirect techniques outperformed conventional direct questioning. Furthermore, our study demonstrates the importance of identifying false negatives as well as false positives to avoid false conclusions about the validity of indirect sensitive question techniques.

Estimating the prevalence of illicit drug use among students using the crosswise model.

OBJECTIVE: The aim of our study is to compare the prevalence of illicit drug use estimated through a technique referred to as the "crosswise model" (CM) with the results from conventional direct questioning (DQ). METHOD: About 1,500 students from Tehran University of Medical Sciences 2009-2010 were first interviewed by DQ and, then three months later, by the CM. RESULT: The CM yielded significantly higher estimates than DQ for lifetime prevalence of use of any illicit drug (CM = 20.2%,DQ = 3.0%, p < .001) and for lifetime prevalence of use of opium or its residue (CM = 13.6%, DQ = 1.0%, p < .001). Also, for use of any illicit drug in the last month and use of opium or its residue in the last month, the CM yielded higher point estimates than DQ, although these differences were not significant (any drug: CM = 1.5%, DQ = 0.2%, p = .66; opium: CM = 3.8%, DQ = 0.0%, p = .21). CONCLUSION: Our findings suggest that the CM is a fruitful data collection method for sensitive topics such as substance abuse.

Social determinants of sex differences in disability among older adults: a multi-country decomposition analysis using the World Health Survey.

INTRODUCTION: Women represent a growing proportion of older people and experience increasing disability in their longer lives. Using a universally agreed definition of disability based on the International Classification of Functioning, Disability and Health, this paper examines how, apart from age, social and economic factors contribute to disability differences between older men and women. METHODS: World Health Survey data were analyzed from 57 countries drawn from all income groups defined by the World Bank. The final sample comprises 63638 respondents aged 50 and older (28568 males and 35070 females). Item Response Theory was applied to derive a measure of disability which ensured cross country comparability. Individuals with scores at or above a threshold score were those who experienced significant difficulty in their everyday lives, irrespective of the underlying etiology. The population was then divided into "disabled" vs. "not disabled". We firstly computed disability prevalence for males and females by socio-demographic factors, secondly used multiple logistic regression to estimate the adjusted effects of each social determinant on disability for males and females, and thirdly used a variant of the Blinder-Oaxaca decomposition technique to partition the measured inequality in disability between males and females into the "explained" part that arises because of differences between males and females in terms of age and social and economic characteristics, and an "unexplained" part attributed to the differential effects of these characteristics. RESULTS: Prevalence of disability among women compared with men aged 50+ years was 40.1% vs. 23.8%. Lower levels of education and economic status are associated with disability in women and men. Approximately 45% of the sex inequality in disability can be attributed to differences in the distribution of socio-demographic factors. Approximately 55% of the inequality results from differences in the effects of the determinants. CONCLUSIONS: There is an urgent need for data and methodologies that can identify how social, biological and other factors separately contribute to the health decrements facing men and women as they age. This study highlights the need for action to address social structures and institutional practices that impact unfairly on the health of older men and women.

Estimating Heterogeneous Treatment Effects with Observational Data.

Individuals differ not only in their background characteristics, but also in how they respond to a particular treatment, intervention, or stimulation. In particular, treatment effects may vary systematically by the propensity for treatment. In this paper, we discuss a practical approach to studying heterogeneous treatment effects as a function of the treatment propensity, under the same assumption commonly underlying regression analysis: ignorability. We describe one parametric method and two non-parametric methods for estimating interactions between treatment and the propensity for treatment. For the first method, we begin by estimating propensity scores for the probability of treatment given a set of observed covariates for each unit and construct balanced propensity score strata; we then estimate propensity score stratum-specific average treatment effects and evaluate a trend across them. For the second method, we match control units to treated units based on the propensity score and transform the data into treatment-control comparisons at the most elementary level at which such comparisons can be constructed; we then estimate treatment effects as a function of the propensity score by fitting a non-parametric model as a smoothing device. For the third method, we first estimate non-parametric regressions of the outcome variable as a function of the propensity score separately for treated units and for control units and then take the difference between the two non-parametric regressions. We illustrate the application of these methods with an empirical example of the effects of college attendance on womens fertility.