Reproducibility and replicability in the context of effective statistical and open science practice

ABSTRACT

As a consequence of the global COVID-19 pandemic, a large number of research publications have appeared at a rapid pace, as well as a large number of withdrawal/retractions. Because of this rapid throughput, reproducibility and replicability in research become even more important. Many published scientific findings either have not been or cannot be reproduced or replicated. While some non-reproducible or non-replicable findings are the nature of the scientific process and its inherent uncertainty, others are usually due to the lack of transparency. These observations motivated the discussion of whether there is a “reproducibility and replicability crisis” and how to address it in many research areas, including psychology and medicine. In 2019, the National Academies of Sciences, Engineering, and Medicine (NASEM) released a report titled “Reproducibility and Replicability in Science” and the American Statistical Association published a special issue on “Statistical Inference in the 21st Century A World Beyond p < 0.05.” The objective of this presentation is to discuss how effective statistical practice under the open science environment may help the scientific community, including addiction researchers, to disseminate high-quality research findings. We will start by defining reproducibility and replicability, and describing how they relate to transparency and generalizability. Next, we will examine two key statistical concepts uncertainty and variability, and highlight the differences between exploratory and confirmatory research. We will conclude the presentation with some possible solutions such as preregistration. References National Academies of Sciences, Engineering, and Medicine 2019. Reproducibility and Replicability in Science. Washington, D.C The National Academies Press. https//doi.org/10.17226/25303.;2019 special issue on Statistical Inference in the 21st Century A World Beyond p < 0.05. The American Statistician, 73(Supp 1) 1 - 401.