What's in a P-Value? A Fragility Analysis of RCTs in the AUA Guidelines for Benign Prostatic Hyperplasia.

OBJECTIVE: To use the fragility index (FI) and fragility quotient (FQ) to assess the strength of statistically significant findings for randomized controlled trials (RCTs) cited in the American Urological Association (AUA) guidelines for benign prostatic hyperplasia. METHODS: Two investigators independently screened the AUA guidelines for management of benign prostatic hyperplasia for RCTs cited as evidence for recommendations. Investigators extracted data related to event rate per group and loss to follow-up which was compared against the FI. Stata 17.0 was used to calculate the FI and FQ which was then summarized and reported according to primary or secondary endpoints. RESULTS: Among the 373 citations in the AUA guidelines, 24 RCTs met inclusion criteria with 29 distinct outcomes analyzed. The median fragility index was 12 (IQR = 4-38), indicating that twelve alternative events to either study arm would nullify statistical significance. Six studies had a FI of ≤2, indicating that only 1-2 outcomes would need to be changed in order to render nonsignificance of results. In 10/24 RCTs, the number of patients lost to follow-up was greater than the FI. CONCLUSION: The AUA Clinical Practice Guidelines for management of benign prostatic hyperplasia cite RCTs with more robust findings when compared to previous studies assessing fragility in the field of Urology. While several included studies had high fragility, the median FI in our analysis was approximately 4-5 times higher than comparable studies of urologic RCTs. However, there are areas where improvement is necessary to support the highest quality of evidence-based medicine.

Spin in Abstracts of Systematic Reviews and Meta-analyses of Melanoma Therapies: Cross-sectional Analysis.

BACKGROUND: Spin is defined as the misrepresentation of a study's results, which may lead to misperceptions or misinterpretation of the findings. Spin has previously been found in randomized controlled trials and systematic reviews of acne vulgaris treatments and treatments of various nondermatological conditions. OBJECTIVE: The purpose of this study was to quantify the presence of spin in abstracts of systematic reviews and meta-analyses of melanoma therapies and identify any related secondary characteristics of these articles. METHODS: We used a cross-sectional approach on June 2, 2020, to search the MEDLINE and Embase databases from their inception. To meet inclusion criteria, a study was required to be a systematic review or meta-analysis pertaining to the treatment of melanoma in human subjects, and reported in English. We used the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) definition of systematic reviews and meta-analyses. Data were extracted in a masked, duplicate fashion. We conducted a powered bivariate linear regression and calculated odds ratios for each study characteristic. RESULTS: A total of 200 systematic reviews met the inclusion criteria. We identified spin in 38% (n=76) of the abstracts. The most common type of spin found was type 3 (selective reporting of or overemphasis on efficacy outcomes or analysis favoring the beneficial effect of the experimental intervention), occurring 40 times; the least common was type 2 (title claims or suggests a beneficial effect of the experimental intervention not supported by the findings), which was not present in any included abstracts. We found that abstracts pertaining to pharmacologic interventions were 3.84 times more likely to contain spin. The likelihood of an article containing spin has decreased annually (adjusted odds ratio 0.91, 95% CI 0.84-0.99). No significant correlation between funding source or other study characteristics and the presence of spin was identified. CONCLUSIONS: We have found that spin is fairly common in the abstracts of systematic reviews of melanoma treatments, but the prevalence of spin in these abstracts has been declining from 1992-2020.

Evaluation of Industry Relationships Among Authors of Systematic Reviews and Meta-analyses Regarding Ménières Disease.

OBJECTIVES: To quantify the presence of conflicts of interest (COI) in SRs and MAs of Ménières disease treatment and identify any related secondary characteristics of these articles. METHODS: A search was conducted on May 28, 2020 to search MEDLINE and Embase databases for SRs or MAs pertaining to Ménières disease published between September 1, 2016 and June 2, 2020. A risk of bias assessment was performed using the Cochrane Collaboration risk of bias assessment criteria. RESULTS: A total of 13 systematic reviews conducted by 49 authors met the inclusion criteria. Of the 49 authors, 7 (14.3%) were found to have some form of COI. Of these 7 authors, 1 (14.3%) completely disclosed all COI within the SR, 1 (14.3%) disclosed one or more COI but were found to have an additional undisclosed COI, and 5 (71.4%) were found to have only undisclosed COI. One of 2 industry funded SRs (50%) had a high risk of bias, and 1 (50%) of the non-industry sponsored SRs were found to have a high risk of bias. CONCLUSIONS: Overall authors of SRs pertaining to Ménières disease appear to be properly disclosing COI at higher rates than other fields of medicine; however, further room for improvement has been noted.

Tonsillar-related pathologies: An analysis of the evidence underpinning management recommendations.

OBJECTIVE: Evidence-based decision making is crucial in reducing the health and economic burdens imposed by tonsillar-related pathologies. Clinical practice guidelines are used to guide these decisions; however, uptake of recommendations in these guidelines is low. Systematic reviews are the highest level of evidence used to influence guideline recommendations; therefore, improving the reporting and methodological quality of systematic reviews related to tonsillar-related pathologies may improve guideline uptake and patient care. METHODS: We used PubMed to search for all clinical practice guidelines related to tonsillar-related pathologies from 2010 to 2020. Included guidelines were then searched for all systematic reviews and meta-analyses. Study characteristics were extracted from each cited systematic review/meta-analysis before being evaluated using the PRISMA (Preferred Reporting Instrument for Systematic Reviews and Meta-Analyses) and AMSTAR-2 (A Measurement Tool to Assess Systematic Reviews 2) instruments. We then compared systematic reviews conducted by a Cochrane systematic review group with non-Cochrane systematic reviews. RESULTS: Seven clinical practice guidelines were included in our study and within these guidelines 98 SRs/MAs were cited, 80 of which were unique and included. Systematic reviews composed 9.1% (98/1082) of all guideline citations. Guideline PRISMA scores ranged from 0.47 to 0.83 with a mean score of 0.71 (n = 80) and guideline AMSTAR-2 scores ranged from 0.52 to 0.83 with a mean of 0.56 (7.29/13) and 0.75 (11.94/16) (n = 80). Cochrane systematic reviews displayed greater PRISMA (0.88 vs. 0.64: p < 0.001) and AMSTAR-2 (0.90 vs. 0.57; p < 0.001) scores compared to the non-Cochrane studies. We found PRISMA and AMSTAR-2 scores were positively correlated across guidelines (r = 0.93). CONCLUSION: Wide variation exists in adherence to PRISMA and AMSTAR-2 guidelines among systematic reviews cited in clinical practice guidelines for tonsillar-related pathologies. Prior registration and adequate risk of bias assessment are two areas where improvements may be needed. Given the importance of guideline uptake, careful considerations to improve the methodological and reporting quality of evidence supporting tonsillar-related pathology recommendations are necessary.

United States internet searches for "infertility" following COVID-19 vaccine misinformation.

CONTEXT: On December 1, 2020, Drs. Wolfgang Wodarg and Micheal Yeadon petitioned to withhold emergency use authorization of the BNT162b2 messenger ribonucleic acid vaccine for coronavirus disease 2019 (COVID-19) manufactured by BioNTech and Pfizer, raising concern for female infertility risks but acknowledging the lack of evidence. The European Medicines Agency and the US Food and Drug Administration ultimately issued emergency use authorizations, but misinformation claiming that COVID-19 vaccines cause female infertility began circulating on social media, potentially influencing public perception and medical decision making among pregnant patients or those seeking to become pregnant. OBJECTIVES: To determine the potential influence misinformation may have had on public interest in infertility related topics, as analyzed through internet search statistics in the US. METHODS: The Google Trends tool was used to analyze results for the search terms "infertility," "infertility AND vaccine," and "infertility AND COVID vaccine" in the US from February 4, 2020 to February 3, 2021. We applied autoregressive integrated moving average models to forecast expected values, comparing them with actual observed values. RESULTS: At peak interest (100), the forecasted relative search volumes interest for the search terms "infertility," "infertility AND vaccine," and "infertility AND COVID vaccine" were 45.47 (95% CI, 33.27-57.66; p<0.001), 0.88 (95% CI, 2.87-4.63; p<0.001), and 0.29 (95% CI, -2.25-2.82; p<0.001). The actual relative search volumes at peak searching represented 119.9, 11,251, and 34,900% increases, respectively, when compared with forecasted values. CONCLUSIONS: COVID-19 vaccine misinformation corresponded with increased internet searches for topics related to infertility in the US. Dispelling misinformation and informing patients about the risks and benefits of COVID-19 vaccination may prevent unnecessary vaccine hesitancy or refusal, contributing to successful vaccination efforts.