Analysis of Systematic Reviews in Clinical Practice Guidelines for Head and Neck Cancer.

OBJECTIVE: Clinical practice guidelines (CPGs) are essential to clinical decision-making as their recommendations are supported by published literature. Systematic reviews are considered the highest quality of evidence used to underpin these guidelines. However, research to support these recommendations may lack compliance to quality reporting among systematic reviews (SRs). Here, we aim to evaluate the quality of SRs underpinning CPG recommendations for the management of head and neck cancer (HNC). STUDY DESIGN: Retrospective cross-sectional analysis. METHODS: Using PubMed, we searched for CPGs pertinent to the management of head and neck cancer published between January 2017 and May 2021. Relevant guidelines were analyzed for all SRs. Cited SRs in CPGs were evaluated using A Measurement Tool to Assess Systematic Reviews-2 (AMSTAR-2) and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) instruments. Study characteristics were extracted in a masked triplicate fashion. RESULTS: A total of 16 CPGs and 142 unique SRs were included in our study. PRISMA completion ranged from 67.15% to 87.65% across CPGs with a mean of 76.41% (SD = 16.9). AMSTAR-2 completion ranged from 34.38% to 84.38% across CPGs with a mean of 67.55% (SD = 20.9) among all SRs. The lowest rated items included funding sources and publication bias. A higher score was achieved in SR done by Cochrane group and it was only 2.11% (3/142) of all SR's quoted in CPG. CONCLUSION: Adherence to AMSTAR-2 and PRISMA items exhibits a variation among SRs cited in CPGs for the management of HNC. The mature underpinning CPG recommendation of SRs cited as supportive evidence could be enhanced in reporting quality. LEVEL OF EVIDENCE: NA Laryngoscope, 132:1976-1983, 2022.

An assessment of transparency and reproducibility-related research practices in otolaryngology.

OBJECTIVES/HYPOTHESIS: Clinical research serves as the foundation for evidence-based patient care, and reproducibility of results is consequently critical. We sought to assess the transparency and reproducibility of research studies in otolaryngology by evaluating a random sample of publications in otolaryngology journals between 2014 and 2018. STUDY DESIGN: Review of published literature for reproducible and transparent research practices. METHODS: We used the National Library of Medicine catalog to identify otolaryngology journals that met the inclusion criteria (available in the English language and indexed in MEDLINE). From these journals, we extracted a random sample of 300 publications using a PubMed search for records published between January 1, 2014 and December 31, 2018. Specific indicators of reproducible and transparent research practices were evaluated in a blinded, independent, and duplicate manner using a pilot-tested Google form. RESULTS: Our initial search returned 26,498 records, from which 300 were randomly selected for analysis. Of these 300 records, 286 met inclusion criteria and 14 did not. Among the empirical studies, 2% (95% confidence interval [CI]: 0.4%-3.5%) of publications indicated that raw data were available, 0.6% (95% CI: 0.3%-1.6%) reported an analysis script, 5.3% (95% CI: 2.7%-7.8%) were linked to an accessible research protocol, and 3.9% (95% CI: 1.7%-6.1%) were preregistered. None of the publications had a clear statement claiming to replicate, or to be a replication of, another study. CONCLUSIONS: Inadequate reproducibility practices exist in otolaryngology. Nearly all studies in our analysis lacked a data or material availability statement, did not link to an accessible protocol, and were not preregistered. Taking steps to improve reproducibility would likely improve patient care. LEVEL OF EVIDENCE: NA Laryngoscope, 130: 1894-1901, 2020.

The fragility of statistically significant results in otolaryngology randomized trials.

OBJECTIVES: The American Academy of Otolaryngology-Head and Neck Surgery regards randomized controlled trials as class A evidence. A novel method to determine the robustness of outcomes in trials is the fragility index. This index represents the number of patients whose status would have to change from a non-event to an event to make a statistically significant result non-significant. METHODS: Investigators included otolaryngology journals listed in the top 10 of one or both of Google Scholar Metrics and Clarivate Analytics' Journal rankings. For inclusion, a randomized controlled trial needed to report a one-to-one random assignment of participants to condition, contain two parallel arms or have used a two-by-two factorial design, and report at least one statistically significant dichotomous outcome. RESULTS: Sixty-nine trials met inclusion criteria. The median fragility index was three events (interquartile range 1-7.5). Median sample size was 72 (interquartile range 50-102.5). Modest correlations were observed between fragility index and total sample size (r = 0.27) and fragility index and event rate (r = 0.46). Investigators found no correlation between fragility index and impact factor or Science Citation Index. In 39% (27/69) of trials, the number lost to follow-up was equal to or greater than the fragility index. CONCLUSION: A median fragility index of 3 indicates that three people, on average, are needed to alter the outcomes in otolaryngology trials. This indicates that the results of two-group randomized controlled trials reporting binary endpoints published in otolaryngology journals may frequently be fragile.

Methodological and reporting quality of systematic reviews referenced in the clinical practice guideline for pediatric high-blood pressure.

OBJECTIVE: To examine the quality of evidence supporting the Clinical Practice Guideline for Screening and Management of High Blood Pressure in Children and Adolescents. METHODS: Methodological and reporting quality of systematic reviews in the Key Action Statements, Consensus Opinions, and in the reference list were assessed using AMSTAR and The PRISMA checklist. RESULTS: For the systematic reviews within the American Academy of Pediatrics' Guideline as a whole (n = 27), a mean of 47.8% of AMSTAR items were appropriately addressed, and a mean of 61% of PRISMA were appropriately reported. The means for items adhered to in the systematic reviews cited by Key Action Statements (n = 6) were 44.6% for AMSTAR and 67% for PRISMA. The means for items adhered to in the systematic reviews cited by consensus opinion (n = 10) were 41.6% for AMSTAR and 54% for PRISMA. Overall, the AMSTAR and PRISMA scores were highly correlated (r = 0.77). CONCLUSION: This study further substantiates the call for more quality evidence-based research in pediatric hypertension. The overall methodological and reporting quality of the systematic reviews were lacking in the American Academy of Pediatrics Guideline. Assessment of risk of bias across studies, identification of an accessible protocol, full disclosure of the role of funding entities, funding of included studies, and a complete PICOS statement were areas of greatest concern. These deficiencies should be addressed in future research in pediatric hypertension.