ABSTRACT
PurposeTo assess the trustworthiness and impact of preprint trial reports during the COVID-19 pandemic. Data sourcesWHO COVID-19 database and the L-OVE COVID-19 platform by the Epistemonikos Foundation (up to August 3rd, 2021) DesignWe compare the characteristics of COVID-19 trials with and without preprints, estimate time to publication of COVID-19 preprint reports, describe discrepancies in key methods and results between preprint and published trial reports, report the number of retracted preprints and publications, and assess whether including versus excluding preprint reports affects meta-analytic estimates and the certainty of evidence. For the effects of eight therapies on mortality and mechanical ventilation, we performed meta-analyses including preprints and excluding preprints at 1 month, 3 months, and 6 months after the first trial addressing the therapy became available either as a preprint or publication (120 meta-analyses in total). ResultsWe included 356 trials, 101 of which are only available as preprints, 181 as journal publications, and 74 as preprints first and subsequently published in journals. Half of all preprints remain unpublished at six months and a third at one year. There were few important differences in key methods and results between trial preprints and their subsequent published reports. We identified four retracted trials, three of which were published in peer-reviewed journals. With two exceptions (2/60; 3.3%), point estimates were consistent between meta-analyses including versus excluding preprints as to whether they indicated benefit, no appreciable effect, or harm. There were nine comparisons (9/60; 15%) for which the rating of the certainty of evidence differed when preprints were included versus excluded, for four of these comparisons the certainty of evidence including preprints was higher and for five of these comparisons the certainty of evidence including preprints was lower. LimitationsThe generalizability of our results is limited to COVID-19. Preprints that are subsequently published in journals may be the most rigorous and may not represent all trial preprints. ConclusionWe found no compelling evidence that preprints provide less trustworthy results than published papers. We show that preprints remain the only source of findings of many trials for several months, a length of time that is unacceptable in a health emergency. We show that including preprints may affect the results of meta-analyses and the certainty of evidence. We encourage evidence users to consider data from preprints in contexts in which decisions are being made rapidly and evidence is being produced faster than can be peer-reviewed. O_TEXTBOXSummary Box 1O_ST_ABSWhat is already known on this topicC_ST_ABSO_LIClinicians and decision-makers need rapidly available and credible data addressing the comparative effectiveness of treatments and prophylaxis for COVID-19. C_LIO_LIInvestigators have adopted preprint servers, which allow the rapid dissemination of research findings before publication in peer-reviewed journals. C_LI What this study addsO_LIWe found no compelling evidence that preprints provide less trustworthy results than published papers. C_LIO_LIWe show that including preprints may affect the results of meta-analyses and the certainty of evidence and we encourage evidence users to consider data from preprints in contexts in which decisions are being made rapidly and evidence is being produced faster than can be peer-reviewed. C_LI C_TEXTBOX
ABSTRACT
ObjectiveTo compare the effects of interleukin-6 (IL-6) receptor blockers, with or without corticosteroids, on mortality in patients with COVID-19. DesignSystematic review and network meta-analysis Data sourcesWHO COVID-19 database, a comprehensive multilingual source of global covid-19 literature, and two prospective meta-analyses Study selectionTrials in which people with suspected, probable, or confirmed COVID-19 were randomized to IL-6 receptor blockers (with or without corticosteroids), corticosteroids, placebo, or standard care. ResultsWe assessed the risk of bias of included trials using a modification of the Cochrane risk of bias tool. We performed a Bayesian fixed effect network meta-analysis and assessed the certainty of evidence using the GRADE approach. We identified 45 eligible trials (20,650 patients), 36 (19,350 patients) of which could be included in the network meta-analysis. 27 of 36 trials were rated at high risk of bias, primarily due to lack of blinding. Tocilizumab (20 more per 1000, 15 fewer to 59 more; low certainty) and sarilumab (11 more per 1000, 38 fewer to 55 more; low certainty) alone may not reduce the risk of death. Tocilizumab, in combination with corticosteroids, probably reduces the risk of death compared to corticosteroids alone (35 fewer per 1000, 52 fewer to 18 more; moderate certainty) and sarilumab, in combination with corticosteroids, may reduce the risk of death compared to corticosteroids alone (43 fewer, 73 fewer to 12 more; low certainty). Tocilizumab and sarilumab, both in combination with corticosteroids, may have similar effects (8 more per 1000, 20 fewer to 35 more; low certainty). ConclusionIL-6 receptor blockers, when added to standard care that includes corticosteroids, in patients with severe or critical COVID-19, probably reduce mortality. Tocilizumab and sarilumab may have similar effectiveness. Systematic review registrationNA What is already known on this topic?O_LIIL-6 receptor blockers have immunosuppressive effects that may be important in COVID-19 patients with immune system dysfunction and inflammation C_LIO_LICorticosteroids reduce the risk of death in patients with severe or critical COVID-19 C_LI What this study addsO_LIOur systematic review and network meta-analysis provides a comprehensive review of the evidence addressing the effects of IL-6 receptor blockers, alone or in combination with corticosteroids, in COVID-19 C_LIO_LIIL-6 receptor blockers when added to a standard care that includes corticosteroids, in patients with severe or critical COVID-19, probably reduce mortality. C_LIO_LITocilizumab and sarilumab in combination with corticosteroids may have similar effectiveness for reducing mortality. C_LI
