Lowering the statistical significance threshold of randomized controlled trials in three major general anesthesiology journals.

PURPOSE: The primary objective of our study was to determine how lowering a P value threshold from 0.05 to 0.005 would affect the statistical significance of previously published randomized controlled trials (RCTs) in major anesthesiology journals. METHODS: We searched the PubMed database for studies electronically published in 2020 within three major general anesthesiology journals as indexed by both Google Metrics and Scimago Journal & Country Rank. Studies included were RCTs published in 2020 in Anesthesiology, Anesthesia & Analgesia, and the British Journal of Anaesthesia; had a primary endpoint, and used a P value threshold to determine the effect of the intervention. We performed screening and data extraction in a masked duplicate fashion. RESULTS: Ninety-one RCTs met inclusion criteria. The most frequently studied type of intervention was drugs (44/91, 48%). From the 91 trials, 99 primary endpoints, and thus P values, were obtained. Fifty-eight (59%) endpoints had a P value < 0.05 and 41 (41%) had a P value ≥ 0.05. Of the 58 primary endpoints previously considered statistically significant, 21 (36%) P values would maintain statistical significance at P < 0.005, and 37 (64%) would be reclassified as "suggestive." CONCLUSIONS: Lowering a P value threshold of 0.05 to 0.005 would have altered one third of significance interpretations of RCTs in the surveyed anesthesiology literature. Thus, it is important for readers to consider post hoc probabilities when evaluating clinical trial results. Although the present study focused on the anesthesiology literature, we suggest that our results warrant further research within other fields of medicine to help avoid clinical misinterpretation of RCT findings and improve quality of care.

RéSUMé: OBJECTIF: L'objectif principal de notre étude était de déterminer comment l'abaissement d'un seuil de valeur P de 0,05 à 0,005 affecterait la signification statistique des études randomisées contrôlées (ERC) précédemment publiées dans certaines des principales revues d'anesthésiologie. MéTHODE: Nous avons réalisé des recherches dans la base de données PubMed pour trouver des études publiées électroniquement en 2020 dans trois des principales revues d'anesthésiologie générale et indexées par Google Metrics et Scimago Journal & Country Rank. Les études incluses étaient des ERC publiées en 2020 dans les revues Anesthesiology, Anesthesia & Analgesia, et le British Journal of Anaesthesia, qui avaient un critère d'évaluation principal et utilisaient un seuil de valeur P pour déterminer l'effet de l'intervention. Nous avons effectué la sélection et l'extraction des données de manière dupliquée masquée. RéSULTATS: Quatre-vingt-onze ERC remplissaient les critères d'inclusion. Le type d'intervention le plus fréquemment étudié était de nature médicamenteuse (44/91, 48 %). Sur les 91 études, 99 critères d'évaluation principaux, et donc valeurs P, ont été obtenus. Cinquante-huit (59 %) critères d'évaluation avaient une valeur P < 0,05 et 41 (41 %) avaient une valeur P ≥ 0,05. Sur les 58 critères d'évaluation principaux précédemment considérés comme statistiquement significatifs, 21 (36 %) valeurs P maintiendraient leur signification statistique à P < 0,005, et 37 (64 %) seraient reclassées comme étant « suggestives ¼. CONCLUSION: Le fait d'abaisser le seuil de valeur P de 0,05 à 0,005 aurait modifié un tiers des interprétations de signification des ERC dans la littérature anesthésiologique étudiée. Il est donc important que les lectrices et lecteurs tiennent compte des probabilités post hoc lors de l'évaluation des résultats d'études cliniques. Bien que la présente étude se soit concentrée sur la littérature en anesthésiologie, nous suggérons que nos résultats justifient des recherches supplémentaires dans d'autres domaines de la médecine afin d'éviter une mauvaise interprétation clinique des résultats des ERC et d'améliorer la qualité des soins.

Assessing the reporting of harms in systematic reviews focused on platelet-rich plasma therapy: a cross-sectional analysis.

Aim: Our objectives are: to evaluate the completeness of harms reporting in systematic reviews (SRs) on platelet-rich plasma therapy; to assess the overall methodological quality of the SR using AMSTAR-2 tool; to assess harms reporting overlap in primary studies between SRs. Materials & methods: The authors searched five database systems for relevant literature on platelet rich plasma therapy. The authors screened and extracted in masked, duplicate fashion. Results: All SRs reported less than 50% completeness in harms reporting. The most frequently reported item was harms being stated in the abstract or title (26/103, 25.2%). AMSTAR-2 assessed 96 SRs as 'critically low', 6 SRs as 'low' and 1 'moderate'. Conclusion: Our study highlights that reporting of harms should become more standardized and transparent.

This study looked at how well the negative effects (harms) of platelet-rich plasma therapy are reported in systematic reviews (SRs). The researchers carefully reviewed and extracted data of SRs from different databases. They found that less than 50% of the SRs reported the harms associated with platelet rich plasma therapy. The researchers also used a tool called AMSTAR to assess the value of each of the SRs, and most were found to be 'critically low' reviews. In conclusion, this study highlights the importance of platelet rich plasma therapy negative effect reporting by SRs.

Assessing the Reporting of Harms in Systematic Reviews Focused on the Therapeutic and Cosmetic Uses of Botulinum Toxin.

BACKGROUND AND OBJECTIVE: The expanding use of botulinum toxin (BoNT) in medical practice demonstrates the need to highlight whether there is adequate information regarding its safety profile. The aim of our study was to identify completeness of harms reporting for BoNT treatment within systematic reviews (SRs), assess quality of SRs using the AMSTAR-2 tool, and determine the degree of overlap among primary studies within each SR. METHODS: On May 31, 2022, we searched Embase, Epistemonikos, MEDLINE, and the Cochrane Database of Systematic Reviews for SRs on BoNT therapy. Screening and data extraction were performed in a masked, duplicate fashion. AMSTAR-2 was used to evaluate the methodological quality of included SRs. Corrected covered area (CCA) was calculated for SR dyads. RESULTS: Of the 90 included SRs, we found that 70 completed less than 50% of harms items. The most reported items were BoNT as a favorable intervention (73/90, 81.1%) and harms as a primary outcome (72/90, 80.0%). The least reported items were grades and severity scales used to classify harms (8/90, 8.9%) and number of treatment discontinuations in each arm (10/90, 11.1%). Eighty-three SRs were rated "critically low" (83/90, 92.2%), while 5 SRs were rated "high" (5/90, 5.6%) via AMSTAR-2 tool. Significant associations were found between completion of harms reporting and: (1) a "critically low" appraisal on AMSTAR-2 tool (p = 0.0060) and (2) whether harms was reported as a primary outcome (p = 0.0001). The total CCA overlap was determined to be 0.8%. CONCLUSION: Our results demonstrate that harms are underreported within BoNT SRs. Because healthcare professionals often refer to SRs to guide clinical decision making, it is important to continue to explore shortcomings among BoNT literature in future studies.

Harms reporting by systematic reviews for functional endoscopic sinus surgery: a cross-sectional analysis.

OBJECTIVES: To evaluate the completeness of harms reporting in systematic reviews (SRs) pertaining to functional endoscopic sinus surgery (FESS). METHODS: Using a cross-sectional study design, we performed a comprehensive search using MEDLINE (PubMed and Ovid), EMBASE, Epistemonikos, and the Cochrane Database of Systematic Reviews databases for SRs regarding FESS on May 15th, 2022. Returns were screened and data were extracted in a masked, duplicate manner. Following established methodology, we extracted general study characteristics, harms items, and overall methodological quality for each SR in our sample. Corrected covered area (CCA) was calculated for SR dyads. For data analysis, using Stata 16.1 we performed a bivariate analysis between variables. RESULTS: Fifty-five SR's were included in our sample after excluding 375 studies that did not meet our inclusion criteria. Of the included SRs, 19 (19/55, 34.5%) did not report harms and 39 (39/55, 70.9%) reported half of the harms items or fewer. Our study found that 23 (23/55, 41.8%) of SRs demonstrated a method of harms data collection, 26 (26/55, 47.3%) of SRs had patients available for harms analysis in their results, and 25 (25/55, 45.5%) of SRs had a balanced discussion of harms and benefits of FESS. Fifty-two SRs were appraised as "critically low" quality using AMSTAR-2. A significant association was found between completeness of harms reporting (Mahady) and whether harms were listed as a primary outcome. No other associations were statistically significant. Two SR dyads had CCAs between 20% and 50% overlap and were compared for unique and shared harms. CONCLUSIONS: Our study demonstrates gaps in harms reporting regarding FESS in SRs. We recommend future studies implement guidelines such as the STROCCS guidelines or the harms extension of the PRISMA guidelines to improve harms reporting. Accurate harms reporting may advance patient safety and promote a more objective risk-benefit analysis for physicians and patients.

Assessing the Reporting and Frequency of Harms in Systematic Reviews Focused on Minimally Invasive Hysterectomies: A Cross-sectional Analysis.

OBJECTIVE: To assess the quality of harms reporting in systematic reviews (SRs) regarding minimally invasive hysterectomies (MIHs). DATA SOURCES: We performed a cross-sectional analysis on SRs addressing MIHs to assess completeness of harms reporting. Hysterectomies are common gynecologic procedures and are associated with complications. Such adverse events can directly affect physician decision making and patient outcomes. Thus, it is important that SRs equally weigh the harms and benefits surrounding MIHs. METHODS OF STUDY SELECTION: On May 15, 2022, we searched MEDLINE (PubMed and Ovid), Embase, Epistemonikos, and the Cochrane Database of Systematic Reviews for SRs with or without a meta-analysis on MIH for any indication. Eligible studies underwent full-text screening, data extraction, harms reporting assessment, and A Measurement Tool to Assess Systematic Reviews-2 quality assessment in a masked, duplicate fashion. The corrected covered area was calculated to indicate any overlap between SR dyads. TABULATION, INTEGRATION, AND RESULTS: A total of 52 SRs met the inclusion criteria for data extraction. We found that >44 of included SRs (of 52; 84.6%) reported >50% of the harms items. Completeness of harms reporting was significantly associated with harms specification as a primary outcome (p <.05). The corrected covered area was 0.60%. CONCLUSION: The harms reporting was more complete than hypothesized, but still had deficiencies throughout, such as inconsistent use of severity scales to classify harms. Even though MIH harms reporting in sample SRs was more complete than other fields of medicine, it was still suboptimal overall and requires refinement.

An analysis of harms reporting in systematic reviews regarding ketorolac for management of perioperative pain.

BACKGROUND: Owing to the frequent perioperative use of ketorolac tromethamine and its ability to minimise postoperative opioid requirements, it is important to continually reassess harms associated with its use. Our primary objective was to investigate the extent of harms reporting in systematic reviews (SRs) on ketorolac for perioperative pain. METHODS: In May 2022, we conducted a search of major databases, MEDLINE (PubMed and Ovid), Embase, Epistemonikos, and the Cochrane Database of Systematic Reviews to identify eligible SRs on ketorolac for perioperative pain. Screening and data extraction were performed in masked, duplicate fashion. A MeaSurement Tool to Assess systematic Reviews-2 (AMSTAR-2) was used to appraise the methodological quality of included SRs. Corrected covered area (CCA) was calculated to determine overlap of primary studies between SR dyads. RESULTS: A total of 28 SRs evaluating 630 primary studies met the inclusion criteria. Seven SRs (7/28, 25%) reported no harms and 17 SRs (17/28, 60.7%) reported ≤50% of harms items. A significant association was found between completeness of harms reporting and whether harms were specified as a primary outcome (P<0.001). No other associations were statistically significant. Regarding methodological quality, 22 SRs were appraised as 'critically low' (22/28, 78.6%), 5 as 'low' (5/28, 17.9%), and 1 as 'high' (1/28, 3.6%). One SR dyad had a CCA >50% but neither reported harms. CONCLUSIONS: The extent of harms reporting in systematic reviews was inadequate. Given the importance that systematic reviews have on guiding perioperative decision-making, it is essential to improve the completeness of harms reporting.

The impact of COVID-19 on otolaryngology research: a cross-sectional analysis of discontinued trials.

CONTEXT: The COVID-19 pandemic has reduced the capacity to conduct medical research due to recruitment difficulties, supply chain shortages, and funding deficits. The clinical practice of otolaryngology was especially impacted due to a reduction in elective procedures, such as facial plastic surgeries and vocal fold injections. OBJECTIVES: The primary objective was to examine the extent of clinical trial (CTs) disruption secondary to the COVID-19 pandemic in the field of otolaryngology. METHODS: On August 1, 2021, we conducted a systematic search utilizing ClinicalTrials.gov for CTs related to common otolaryngology disorders. We utilized the date range January 1, 2020 through August 1, 2021 to identify all trials potentially affected by the COVID-19 pandemic. Investigators performed screening and data extraction in a duplicate, masked fashion. Trials resulting from the search were extracted for trial status, condition treated, enrollment number, funding, study type, study design, last update posted date, and trial location. Trials that explicitly mentioned COVID-19 as a reason for discontinuation or suspension were coded as such. For trials that did not explicitly mention COVID-19, we coded the reason provided from ClinicalTrials.gov. The Oklahoma State University Center for Health Science Institutional Review Board determined that this project did not qualify as human subject research. RESULTS: A total of 1,777 CTs met the inclusion criteria, and 223 CTs were discontinued between January 1, 2020 and August 1, 2021. Thirty-three (14.8%) of the 223 CTs reported discontinuation explicitly due to the COVID-19 pandemic. The 33 studies had 1,715 participants enrolled in total. Among the primary interventions, 11 (33.3%) were devices, 10 (30.3%) were drugs, 5 (15.2%) were behavioral, 4 (12.1%) were diagnostic tests, 1 (3.0%) was dietary, and 2 (6.1%) were labeled as "other." Regarding the CT location, 20 (60.6%) were conducted in the United States, and 13 (39.4%) were conducted internationally. Of the 33 CTs, 19 (57.6%) were suspended, 9 (27.3%) were terminated, and 5 (15.2%) were withdrawn. The overall most common reason for trial disruption was recruitment difficulties (24.2%). Median enrollment for discontinued trials due to COVID-19 was 37 (interquartile range [IQR], 19-71) and for other reasons was 6 (IQR, 0-27), for which the Mann-Whitney test showed a statistically significant difference between the two (z=-3.913, p<0.001). There were no significant associations between trial location, funding source, randomization, or whether a study involved masked vs unmasked participants. CONCLUSIONS: The COVID-19 pandemic has incited an impact on clinical research in the field of otolaryngology. To preserve trial continuation amid future threats to participant interaction and communication, we recommend further exploration of remote monitoring practices and virtual procedures-those that will maintain the effectiveness and accuracy needed to establish novel therapeutics. We encourage future trials to gauge which remote assessments show the greatest validity, with the long-term goal of establishing innovative study designs resilient to future pandemics.

Influence of the COVID-19 Pandemic on Clinical Trial Discontinuation in Anesthesiology: Cross-sectional Analysis.

BACKGROUND: The COVID-19 pandemic drastically altered perioperative medical practice owing to safety concerns, postponing elective or nonemergent procedures, supply chain shortages, and reallocating perioperative staff to care for patients with COVID-19. However, the impact of the pandemic on the conduct on anesthesiology clinical research is unknown. OBJECTIVE: The primary objective was to quantify the magnitude of the COVID-19 pandemic's impact on anesthesiology clinical research. METHODS: We performed a systematic search using ClinicalTrials.gov to identify clinical trials related to the practice of anesthesiology. We screened trials with status updates from January 1, 2020, through October 1, 2021, to capture trials potentially affected by the COVID-19 pandemic by the time of our search. Investigators screened for relevant studies and extracted trial characteristics along with the reason for discontinuation reported on the clinical trial registry. RESULTS: A total of 823 clinical trials met inclusion criteria, and 146 clinical trials were discontinued within the designated date range. In total, 24 (16.4%) of the 146 clinical trials were halted explicitly owing to the COVID-19 pandemic. A significant association existed between trial enrollment numbers and the likelihood of discontinuation due to the COVID-19 pandemic, as larger trials were more likely to be disrupted (z=-2.914, P=.004). CONCLUSIONS: The COVID-19 pandemic is reportedly associated with the discontinuation of anesthesiology-related clinical trials. With the uncertain course of the COVID-19 pandemic, developing anesthesia trial protocols to help minimize social interaction and prevent premature trial disruption are imperative.

The use of telecommunication and virtualization among ongoing and discontinued COVID-19 clinical trials: A cross-sectional analysis.

IMPORTANCE: The COVID-19 pandemic has impacted clinical research due to safety measures such as social distancing and lockdowns. However, developing treatments for COVID-19 relies on conducting clinical trials. Using telemedicine or virtual methods may support ongoing trials and limit the pandemic's impact on clinical research. OBJECTIVE: To examine the use of virtual methods among ongoing and discontinued COVID-19 clinical trials. DESIGN: In this cross-sectional analysis, we performed a systematic search of ClinicalTrials.gov for COVID-19 related trials registered since the pandemic began. In masked, duplicate fashion, authors extracted data from included studies, noting whether trialists reported using telecommunication, virtualization, or remote data collection to deliver interventions and monitor outcome measures. The authors also coded the use of virtual methods for recruitment, enrollment, or follow-up visits. Chi-square tests and Kruskal-Wallis tests were used to assess differences in the use of virtual methods between ongoing and discontinued studies and differences between intervention types. RESULTS: Our search returned 2549 clinical trials, of which 2383 were included. Of included studies, 2109 (88.5%) were ongoing and 274 (11.5%) were discontinued. Overall, 519 (24.6%) ongoing COVID-19 trials reported using virtual methods for trial conduct and 43 (15.7%) discontinued trials reported using virtual methods. There was a statistically significant difference in the rate of reporting virtual methods between discontinued and ongoing trials (X21 = 27.2, P < .001). Studies listed as Behavioral or Other were more likely to report using virtual methods for delivering interventions compared to other intervention types (X21 = 751.88, P < .001). CONCLUSIONS AND RELEVANCE: The COVID-19 pandemic has presented an unprecedented need for safe and efficient clinical trial conduct. Nearly a quarter of ongoing COVID-19 clinical trials in our sample reported using virtual methods for supporting trial progress. Ongoing trials were more likely to report virtual methods compared to discontinued trials. Developing strategies that allow for continuing trials during emergencies may limit trial disruption. Exploring and developing remote trial methods may continue to be valuable in light of emerging COVID-19 variants and may persist beyond the pandemic.

Tonsillar-related pathologies: An analysis of the evidence underpinning management recommendations.

OBJECTIVE: Evidence-based decision making is crucial in reducing the health and economic burdens imposed by tonsillar-related pathologies. Clinical practice guidelines are used to guide these decisions; however, uptake of recommendations in these guidelines is low. Systematic reviews are the highest level of evidence used to influence guideline recommendations; therefore, improving the reporting and methodological quality of systematic reviews related to tonsillar-related pathologies may improve guideline uptake and patient care. METHODS: We used PubMed to search for all clinical practice guidelines related to tonsillar-related pathologies from 2010 to 2020. Included guidelines were then searched for all systematic reviews and meta-analyses. Study characteristics were extracted from each cited systematic review/meta-analysis before being evaluated using the PRISMA (Preferred Reporting Instrument for Systematic Reviews and Meta-Analyses) and AMSTAR-2 (A Measurement Tool to Assess Systematic Reviews 2) instruments. We then compared systematic reviews conducted by a Cochrane systematic review group with non-Cochrane systematic reviews. RESULTS: Seven clinical practice guidelines were included in our study and within these guidelines 98 SRs/MAs were cited, 80 of which were unique and included. Systematic reviews composed 9.1% (98/1082) of all guideline citations. Guideline PRISMA scores ranged from 0.47 to 0.83 with a mean score of 0.71 (n = 80) and guideline AMSTAR-2 scores ranged from 0.52 to 0.83 with a mean of 0.56 (7.29/13) and 0.75 (11.94/16) (n = 80). Cochrane systematic reviews displayed greater PRISMA (0.88 vs. 0.64: p < 0.001) and AMSTAR-2 (0.90 vs. 0.57; p < 0.001) scores compared to the non-Cochrane studies. We found PRISMA and AMSTAR-2 scores were positively correlated across guidelines (r = 0.93). CONCLUSION: Wide variation exists in adherence to PRISMA and AMSTAR-2 guidelines among systematic reviews cited in clinical practice guidelines for tonsillar-related pathologies. Prior registration and adequate risk of bias assessment are two areas where improvements may be needed. Given the importance of guideline uptake, careful considerations to improve the methodological and reporting quality of evidence supporting tonsillar-related pathology recommendations are necessary.

Evaluation of Spin in the Abstracts of Systematic Reviews and Meta-Analyses Focused on Tinnitus.

HYPOTHESIS: The objective was to investigate the prevalence of spin in abstracts of systematic reviews and meta-analyses covering the treatment of tinnitus. We hypothesized that spin would be present in these articles and a significant relationship would exist between spin usage and extracted study characteristics. BACKGROUND: Spin, the misrepresentation of study findings, can alter a clinician's interpretation of a study's results, potentially affecting patient care. Previous work demonstrates that spin is present in abstracts of randomized clinical trials. METHODS: Using a cross-sectional analysis, we conducted a systematic search using MEDLINE and Embase databases on June 2, 2020, for systematic reviews focused on tinnitus treatment. Investigators performed screening and data extraction in a masked, duplicate fashion. RESULTS: Forty systematic reviews met inclusion criteria, and spin was identified in four of them. Spin in abstracts most frequently occurred when conclusions claimed the beneficial effect of the experimental treatment despite high risk of bias in primary studies (n = 3). The other form of spin found was the conclusion claims safety based on nonstatistically significant results with a wide confidence interval (n = 1). There was no significant association between spin and any of our extracted study characteristics. CONCLUSION: Spin was observed in 10% of abstracts of systematic reviews and meta-analyses covering the treatment of tinnitus. Although this percentage may be small, we recommend that medical journals provide a more detailed framework for abstract structure and require the inclusion of risk of bias assessment results in abstracts to prevent the incorporation of spin.