Evaluation of intervention reporting in published emergency medicine clinical trials.

OBJECTIVE: The extent of intervention reporting in emergency medicine journals remains unclear. The primary objective is to assess overall completion of the Template for Intervention Description and Replication (TIDieR) checklist described in emergency medicine randomized clinical trials (RCTs). The secondary outcomes were to (1) compare reporting before and after TIDieR publication; (2) evaluate factors associated with intervention reporting. METHODS: Our cross-sectional study used Google Scholar's metrics to identify seven emergency medicine journals; of which, we randomly sampled 300 articles. Using two PubMed searches, we extracted 150 RCTs before and after publications of TIDieR. Two investigators independently extracted data. The primary analysis to measure overall completion included descriptive statistics for each checklist item. Our secondary analysis used an interrupted time series analysis and generalized estimating equations to determine the effect of TIDieR publication on intervention reporting. RESULTS: Our initial search yielded 635 articles; from which, we randomly sampled 300 articles. We excluded 67 articles, leaving 233 for analysis. The mean number of TIDieR items reported was 5.4 (standard deviation = 1.18). Of the 233 trials, 42.9% provided information about materials, 67% provided intervention procedures, and 99.1% provided intervention delivery. The least reported items were intervention modifications (2.6%), intervention adherence assessment methods (3.4%), and intervention adherence assessment outcomes (2.2%). CONCLUSIONS: The completeness of intervention reporting is suboptimal in emergency medicine journals, necessitating improvement. The current state of adherence could be improved through the combined efforts of journal editors, major editorial organizations, and authors.

Reproducible and transparent research practices in published neurology research.

BACKGROUND: The objective of this study was to evaluate the nature and extent of reproducible and transparent research practices in neurology publications. METHODS: The NLM catalog was used to identify MEDLINE-indexed neurology journals. A PubMed search of these journals was conducted to retrieve publications over a 5-year period from 2014 to 2018. A random sample of publications was extracted. Two authors conducted data extraction in a blinded, duplicate fashion using a pilot-tested Google form. This form prompted data extractors to determine whether publications provided access to items such as study materials, raw data, analysis scripts, and protocols. In addition, we determined if the publication was included in a replication study or systematic review, was preregistered, had a conflict of interest declaration, specified funding sources, and was open access. RESULTS: Our search identified 223,932 publications meeting the inclusion criteria, from which 400 were randomly sampled. Only 389 articles were accessible, yielding 271 publications with empirical data for analysis. Our results indicate that 9.4% provided access to materials, 9.2% provided access to raw data, 0.7% provided access to the analysis scripts, 0.7% linked the protocol, and 3.7% were preregistered. A third of sampled publications lacked funding or conflict of interest statements. No publications from our sample were included in replication studies, but a fifth were cited in a systematic review or meta-analysis. CONCLUSIONS: Currently, published neurology research does not consistently provide information needed for reproducibility. The implications of poor research reporting can both affect patient care and increase research waste. Collaborative intervention by authors, peer reviewers, journals, and funding sources is needed to mitigate this problem.

Assessment of transparent and reproducible research practices in the psychiatry literature.

BACKGROUND: Reproducibility is a cornerstone of scientific advancement; however, many published works may lack the core components needed for study reproducibility. AIMS: In this study, we evaluate the state of transparency and reproducibility in the field of psychiatry using specific indicators as proxies for these practices. METHODS: An increasing number of publications have investigated indicators of reproducibility, including research by Harwicke et al, from which we based the methodology for our observational, cross-sectional study. From a random 5-year sample of 300 publications in PubMed-indexed psychiatry journals, two researchers extracted data in a duplicate, blinded fashion using a piloted Google form. The publications were examined for indicators of reproducibility and transparency, which included availability of: materials, data, protocol, analysis script, open-access, conflict of interest, funding and online preregistration. RESULTS: This study ultimately evaluated 296 randomly-selected publications with a 3.20 median impact factor. Only 107 were available online. Most primary authors originated from USA, UK and the Netherlands. The top three publication types were cohort studies, surveys and clinical trials. Regarding indicators of reproducibility, 17 publications gave access to necessary materials, four provided in-depth protocol and one contained raw data required to reproduce the outcomes. One publication offered its analysis script on request; four provided a protocol availability statement. Only 107 publications were publicly available: 13 were registered in online repositories and four, ten and eight publications included their hypothesis, methods and analysis, respectively. Conflict of interest was addressed by 177 and reported by 31 publications. Of 185 publications with a funding statement, 153 publications were funded and 32 were unfunded. CONCLUSIONS: Currently, Psychiatry research has significant potential to improve adherence to reproducibility and transparency practices. Thus, this study presents a reference point for the state of reproducibility and transparency in Psychiatry literature. Future assessments are recommended to evaluate and encourage progress.

Assessment of the completeness of intervention reporting of randomized clinical trials for alcohol use disorders: Effect of the TIDieR checklist and guide.

BACKGROUND/PURPOSE: Properly designed randomized controlled trials (RCTs) are the gold standard in patient-centered clinical research. Incomplete intervention reporting affects the readers' ability to evaluate treatment efficacy. Previous studies show that detailed descriptions of trial interventions remains insufficient for reliable replication. Understanding reporting areas in need of improvement can improve the quality of intervention reporting. METHODS: This cross-sectional review uses the Template for Intervention Description and Replication (TIDieR) checklist to evaluate the quality of intervention reporting in RCTs. The primary outcome was to investigate the completeness of intervention reporting of RCTs reporting outcomes for patients with alcohol use disorder (AUD) published in highly ranked addiction journals. The secondary outcomes were to: 1) evaluate whether publication of the TIDieR checklist resulted in better intervention reporting practices and 2) determine whether particular trial characteristics were associated with the completeness of intervention reporting. RESULTS: The final analysis included 56 records. The mean number of reported TIDieR items was 5.1 (SD = 1.47) of a possible 12. TIDieR checklist publication did not increase the average completion of the TIDieR checklist items (p = 0.76). Improved TIDieR adherence was associated with trials with double blinding, non-drug interventions, and CONSORT endorsement. DISCUSSION/CONCLUSIONS: We found the reporting of interventions to be inadequate in our sample of AUD-related RCTs. Fundamental details were often not reported, hampering both clinical and research reproducibility. Moving forward, it may be necessary to consider additional mechanisms to either improve TIDieR uptake or to find other solutions to improve intervention reporting.