Systematic reviews on interventions for COVID-19 have rarely graded the certainty of the evidence

ABSTRACT BACKGROUND: Numerous systematic reviews on coronavirus disease-19 (COVID-19) treatment have been developed to provide syntheses of the large volume of primary studies. However, the methodological quality of most of these reviews is questionable and the results provided may therefore present bias. OBJECTIVE: To investigate how many systematic reviews on the therapeutic or preventive options for COVID-19 assessed the certainty of the evidence through the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach. METHODS: We conducted a sensitive search in MEDLINE (via PubMed) and included all systematic reviews that assessed any intervention for COVID-19. The systematic reviews included were examined to identify any planned and/or actual assessment using the GRADE approach (or absence thereof) regarding the certainty of the evidence. RESULTS: We included 177 systematic reviews and found that only 37 (21%; 37/177) assessed and reported the certainty of the evidence using the GRADE approach. This number reduced to 27 (16.2%; 27/167) when Cochrane reviews (n = 10), in which an evaluation using GRADE is mandatory, were excluded. CONCLUSION: Most of the systematic reviews on interventions relating to COVID-19 omitted assessment of the certainty of the evidence. This is a critical methodological omission that must not be overlooked in further research, so as to improve the impact and usefulness of syntheses relating to COVID-19.

Determinants Used to Justify the Strength of Recommendations among Korean Clinical Practice Guidelines

BACKGROUND: A standardized systematic approach to grade evidence and the strength of recommendations is important for guideline users to minimize bias and help interpret the most suitable decisions at the point of care. The study aims to identify and classify determinants used to make judgement for the strength of recommendations among 56 Korean clinical practice guidelines (CPGs), and explore strong recommendations based on low quality of evidence. METHODS: Determinants used in the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach among 34 CPGs which have reported both strength of recommendations and level of evidence were reviewed. RESULTS: Five of 34 CPGs (14.7%) considered quality of evidence, benefits and harms, patients' values and preferences, and costs. And 24 of 34 CPGs (70.6%) considered both magnitude of effect and feasibility as additional determinants. Judgement table was not widely provided for use to translate evidence into recommendations. Eighty-two of 121 recommendations (67.8%, ranged 20.0% to 100.0%) among 11 CPGs using the same judgement scheme showed ‘strong’ strength of recommendations based on low or very low quality of evidence. Among 5 paradigmatic situations that justify strong recommendations based on low or very low evidence, situation classified as ‘potential equivalence, one option clearly less risky or costly’ was 87.8% for 82 strong recommendations. Situation classified as ‘uncertain benefit, certain harm’ was 4.9%. CONCLUSION: There is a need to introduce and systematize an evidence-based grading system. Using judgement table to justify the strength of recommendations and applying the 5 paradigmatic situations mentioned above is also recommended in the near future.