Toward a standardized evaluation of imputation methodology.

Developing new imputation methodology has become a very active field. Unfortunately, there is no consensus on how to perform simulation studies to evaluate the properties of imputation methods. In part, this may be due to different aims between fields and studies. For example, when evaluating imputation techniques aimed at prediction, different aims may be formulated than when statistical inference is of interest. The lack of consensus may also stem from different personal preferences or scientific backgrounds. All in all, the lack of common ground in evaluating imputation methodology may lead to suboptimal use in practice. In this paper, we propose a move toward a standardized evaluation of imputation methodology. To demonstrate the need for standardization, we highlight a set of possible pitfalls that bring forth a chain of potential problems in the objective assessment of the performance of imputation routines. Additionally, we suggest a course of action for simulating and evaluating missing data problems. Our suggested course of action is by no means meant to serve as a complete cookbook, but rather meant to incite critical thinking and a move to objective and fair evaluations of imputation methodology. We invite the readers of this paper to contribute to the suggested course of action.

Graphical and numerical diagnostic tools to assess multiple imputation models by posterior predictive checking.

Problem: The congenial of the imputation model is crucial for valid statistical inferences. Hence, it is important to develop methodologies for diagnosing imputation models. Aim: We propose and evaluate a new diagnostic method based on posterior predictive checking to diagnose the congeniality of fully conditional imputation models. Our method applies to multiple imputation by chained equations, which is widely used in statistical software. Methods: The proposed method compares the observed data with their replicates generated under the corresponding posterior predictive distributions to diagnose the performance of imputation models. The method applies to various imputation models, including parametric and semi-parametric approaches and continuous and discrete incomplete variables. We studied the validity of the method through simulation and application. Results: The proposed diagnostic method based on posterior predictive checking demonstrates its validity in assessing the performance of imputation models. The method can diagnose the consistency of imputation models with the substantive model and can be applied to a broad range of research contexts. Conclusion: The diagnostic method based on posterior predictive checking provides a valuable tool for researchers who use fully conditional specification to handle missing data. By assessing the performance of imputation models, our method can help researchers improve the accuracy and reliability of their analyzes. Furthermore, our method applies to different imputation models. Hence, it is a versatile and valuable tool for researchers identifying plausible imputation models.

Joint distribution properties of fully conditional specification under the normal linear model with normal inverse-gamma priors.

Fully conditional specification (FCS) is a convenient and flexible multiple imputation approach. It specifies a sequence of simple regression models instead of a potential complex joint density for missing variables. However, FCS may not converge to a stationary distribution. Many authors have studied the convergence properties of FCS when priors of conditional models are non-informative. We extend to the case of informative priors. This paper evaluates the convergence properties of the normal linear model with normal-inverse gamma priors. The theoretical and simulation results prove the convergence of FCS and show the equivalence of prior specification under the joint model and a set of conditional models when the analysis model is a linear regression with normal inverse-gamma priors.

Prevalence of responsible research practices among academics in The Netherlands.

Background: Traditionally, research integrity studies have focused on research misbehaviors and their explanations. Over time, attention has shifted towards preventing questionable research practices and promoting responsible ones. However, data on the prevalence of responsible research practices, especially open methods, open codes and open data and their underlying associative factors, remains scarce. Methods: We conducted a web-based anonymized questionnaire, targeting all academic researchers working at or affiliated to a university or university medical center in The Netherlands, to investigate the prevalence and potential explanatory factors of 11 responsible research practices. Results: A total of 6,813 academics completed the survey, the results of which show that prevalence of responsible practices differs substantially across disciplines and ranks, with 99 percent avoiding plagiarism in their work but less than 50 percent pre-registering a research protocol. Arts and humanities scholars as well as PhD candidates and junior researchers engaged less often in responsible research practices. Publication pressure negatively affected responsible practices, while mentoring, scientific norms subscription and funding pressure stimulated them. Conclusions: Understanding the prevalence of responsible research practices across disciplines and ranks, as well as their associated explanatory factors, can help to systematically address disciplinary- and academic rank-specific obstacles, and thereby facilitate responsible conduct of research.

Prevalence of questionable research practices, research misconduct and their potential explanatory factors: A survey among academic researchers in The Netherlands.

Prevalence of research misconduct, questionable research practices (QRPs) and their associations with a range of explanatory factors has not been studied sufficiently among academic researchers. The National Survey on Research Integrity targeted all disciplinary fields and academic ranks in the Netherlands. It included questions about engagement in fabrication, falsification and 11 QRPs over the previous three years, and 12 explanatory factor scales. We ensured strict identity protection and used the randomized response method for questions on research misconduct. 6,813 respondents completed the survey. Prevalence of fabrication was 4.3% (95% CI: 2.9, 5.7) and of falsification 4.2% (95% CI: 2.8, 5.6). Prevalence of QRPs ranged from 0.6% (95% CI: 0.5, 0.9) to 17.5% (95% CI: 16.4, 18.7) with 51.3% (95% CI: 50.1, 52.5) of respondents engaging frequently in at least one QRP. Being a PhD candidate or junior researcher increased the odds of frequently engaging in at least one QRP, as did being male. Scientific norm subscription (odds ratio (OR) 0.79; 95% CI: 0.63, 1.00) and perceived likelihood of detection by reviewers (OR 0.62, 95% CI: 0.44, 0.88) were associated with engaging in less research misconduct. Publication pressure was associated with more often engaging in one or more QRPs frequently (OR 1.22, 95% CI: 1.14, 1.30). We found higher prevalence of misconduct than earlier surveys. Our results suggest that greater emphasis on scientific norm subscription, strengthening reviewers in their role as gatekeepers of research quality and curbing the "publish or perish" incentive system promotes research integrity.

Nationwide epidemiological approach to identify associations between keratoconus and immune-mediated diseases.

BACKGROUND: The aetiology of keratoconus (KC) remains poorly understood. KC has typically been described as a non-inflammatory disorder of the cornea. Nonetheless, there is increasing presumptive evidence for the role of the immune system in the pathogenesis of KC. AIM: To evaluate the association between KC and immune-mediated diseases on a population level. We hypothesise that KC is immune-mediated rather than a predominantly degenerative disease. METHODS: Data were obtained from the largest health insurance provider in the Netherlands. Dutch residents are obligatorily insured. The data contained all medical claims and sociodemographic characteristics from all KC patients plus all those data from a 1:6 age-matched and sex-matched control group. The primary outcome was the association between KC and immune-mediated diseases, as assessed by conditional logistic regression. RESULTS: Based on our analysis of 2051 KC cases and 12 306 matched controls, we identified novel associations between KC and Hashimoto's thyroiditis (OR=2.89; 95% CI: 1.41 to 5.94) and inflammatory skin conditions (OR=2.20; 95% CI: 1.37 to 3.53). We confirmed known associations between KC and atopic conditions, including allergic rash (OR=3.00; 95% CI: 1.03 to 8.79), asthma and bronchial hyperresponsiveness (OR=2.51; 95% CI: 1.63 to 3.84), and allergic rhinitis (OR=2.20; 95% CI: 1.39 to 3.49). CONCLUSION: Keratoconus appears positively associated with multiple immune-mediated diseases, which provides a population-based argument that systemic inflammatory responses may influence its onset. The identification of these particular diseases might shed light on potential comparable pathways through which this proinflammatory state is achieved, paving the way for pharmacological treatment strategies.

Effectiveness and moderators of individual cognitive behavioral therapy versus treatment as usual in clinically depressed adolescents: a randomized controlled trial.

We examined if manualized cognitive behavioral therapy (CBT) was more effective than Treatment As Usual (TAU) for clinically depressed adolescents within routine care. This multisite Randomized controlled trail included 88 clinically depressed adolescents (aged 12-21 years) randomly assigned to CBT or TAU. Multiple assessments (pre-, post treatment and six-month follow-up) were done using semi-structured interviews, questionnaires and ratings and multiple informants. The primary outcome was depressive or dysthymic disorder based on the KSADS. Completers, CBT (n = 19) and TAU (n = 26), showed a significant reduction of affective diagnoses at post treatment (76% versus 76%) and after six months (90% versus 79%). Intention-to-treat analyses on depressive symptoms showed that 41.6% within CBT and 31.8% within the TAU condition was below clinical cut-off at post treatment and after six-months, respectively 61.4% and 47.7%. No significant differences in self-reported depressive symptoms between CBT and TAU were found. No prediction or moderation effects were found for age, gender, child/parent educational level, suicidal criteria, comorbidity, and severity of depression. We conclude that CBT did not outperform TAU in clinical practice in the Netherlands. Both treatments were found to be suitable to treat clinically referred depressed adolescents. CBT needs further improvement to decrease symptom levels below the clinical cut-off at post treatment.

The Effectiveness of Parent Management Training-Oregon Model in Clinically Referred Children with Externalizing Behavior Problems in The Netherlands.

The present study examined the effectiveness of parent management training-Oregon model (PMTO) as a treatment for children with externalizing behavior problems in The Netherlands. Clinically referred children (N = 146) aged 4-11 years and their parents were partly randomized to either PMTO (n = 91) or Care As Usual (CAU; n = 55). Families were assessed at four time points: at pretreatment, and after 6, 12, and 18 months. Results showed that both PMTO and CAU were effective in reducing child externalizing behavior, parenting stress and parental psychopathology, with no significant differences between the two treatment conditions. PMTO and CAU interventions also produced some improvements in self-reported parenting skills, but not in observed parenting skills. According to the Reliable Change Index, 16.9 and 45.8 % of the children within the PMTO group showed full recovery or improvement in externalizing behavior, respectively, versus 9.7 and 42.8 % in the CAU condition. Finally, the effect size of PMTO on parent-reported externalizing behavior problems as found in the present study was comparable to that found in previous studies evaluating PMTO as an intervention for this type of child psychopathology.