Estimating the extent of selective reporting: An application to economics.

Using a sample of 70,399 published p-values from 192 meta-analyses, we empirically estimate the counterfactual distribution of p-values in the absence of any biases. Comparing observed p-values with counterfactually expected p-values allows us to estimate how many p-values are published as being statistically significant when they should have been published as non-significant. We estimate the extent of selectively reported p-values to range between 57.7% and 71.9% of the significant p-values. The counterfactual p-value distribution also allows us to assess shifts of p-values along the entire distribution of published p-values, revealing that particularly very small p-values (p < 0.001) are unexpectedly abundant in the published literature. Subsample analysis suggests that the extent of selective reporting is reduced in research fields that use experimental designs, analyze microeconomics research questions, and have at least some adequately powered studies.

Are estimates of the value of a statistical life exaggerated?

The magnitude of the value of a statistical life (VSL) is critical to the evaluation of many health and safety initiatives. To date, the large and rigorous VSL research literature has not explicitly accommodated publication selectivity bias (i.e., the reduced probability that insignificant or negative VSL values are reported). This study demonstrates that doing so is essential. For studies that employ hedonic wage equations to estimate VSL, correction for selection bias reduces the average value of a statistical life by 70-80%. Our meta-regression analysis also identifies several sources for the wide heterogeneity found among reported VSL estimates.