Strong recommendations from low certainty evidence: a cross-sectional analysis of a suite of national guidelines.

BACKGROUND: Clinical guidelines should be based on a thorough evaluation of the evidence and generally include a rating of the quality of evidence and assign a strength to recommendations. Grading of Recommendations Assessment, Development and Evaluation (GRADE) guidance warns against making strong recommendations when the certainty of the evidence is low or very low, but has identified five paradigmatic situations (e.g. life-threatening situations) where this may be justified. AIMS AND OBJECTIVES: We aimed to characterize the strength of recommendations and certainty of the evidence in Irish National Clinical Guidelines using the GRADE approach. METHODS: All National Clinical Guidelines from the National Clinical Effectiveness Committee (NCEC) website using the GRADE approach (fully or partially) were included. All recommendations and their corresponding certainty of the evidence, strength of recommendations and justifications were extracted. Authors classified instances of strong recommendations with low certainty evidence (referred to as discordant recommendations) into one of the five paradigmatic situations. Descriptive statistics were calculated. RESULTS: From the 29 NCEC Clinical Guidelines available at the time of analysis, we identified 8 guidelines using GRADE with a total of 240 recommendations; 38 recommendations did not use the GRADE approach and were excluded. Half of the included guidelines focused on emergency situations. In the final dataset of 202 recommendations, 151 (74.7%) were classified as strong and 51 (25.3%) as conditional. Of the 151 strong recommendations, 55 (36.4%) were supported by high or moderate certainty evidence and 96 (63.6%) by low or very low certainty evidence and were considered discordant. Of these 96 discordant recommendations, 55 (73.7%) were consistent with one of the five paradigmatic situations. However, none were specifically described as such within the guidelines. CONCLUSIONS: The proportion of discordant recommendations identified in this analysis was higher than some previous international studies (range of all strong recommendations being discordant 30-50%), but similar to other guidelines focused on emergency situations. The majority of discordant recommendations could be mapped to one of the five situations, but no National Clinical Guideline explicitly referenced this. Guideline developers require further guidance to enable greater transparency in the reporting of the reasons for discordant recommendations.

No Recommendation Is (at Least Presently) the Best Recommendation: An Updating Quality Appraisal of Recommendations on Screening for Scoliosis.

Recommendations addressing screening for scoliosis differ substantially. Systematically developed guidelines are confronted by consensus and opinion-based statements. This paper elaborates on the issue of the standards of development and reporting of current guidelines and recommendation statements, as well as on the methodological quality of the corresponding evidence syntheses. The SORT classification and the AMSTAR 2 tool were used for those purposes, respectively. Publications included in the analysis differed substantially in terms of their methodological quality. Based on the SORT and AMSTAR 2 scores, the 2018 US PSTF recommendation statement and systematic review on screening for scoliosis are trustworthy and high-quality sources of evidence and aid for decision making. The recommendation statement on insufficient evidence to formulate any recommendations is, paradoxically, very informative. Significantly, updated opinion-based position statements supporting screening for scoliosis acknowledged the importance of research evidence as a basis for recommendation formulation and are more cautious and balanced than formerly. Expert opinions, not built on properly presented analyses of evidence, are at odds with evidence-based practice. Nonetheless, contemporary principles of screening programs, especially those addressing people's values and preferences, and the possible harms of screening, remain underrepresented in both research and recommendations addressing screening for scoliosis.

The Grading of Recommendations, assessment, development, and evaluation approach was rarely followed and inconsistently applied in pressure injury prevention guidelines development: A cross-sectional survey.

INTRODUCTION: The Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) approach was developed to assess the certainty (or quality) of evidence and strength of recommendations in guidelines and endorsed internationally as a standard. Some guidelines had been developed to promote pressure injury prevention. AIMS: We explored whether and to what extent the development of pressure injury prevention guidelines had followed or been informed by the GRADE approach. If this approach was not used, we examined which other methods were used instead. METHODS: A cross-sectional study of pressure injury prevention guidelines was conducted. PubMed, Embase, CINAHL, and Chinese databases as well as guideline repositories and websites of professional bodies were searched for guidelines from 1990 to 2020. The grading systems of the certainty (or quality) of evidence and strength of recommendations of included guidelines were extracted. For the GRADE approach guidelines, compliance was assessed with the GRADE application criteria. RESULTS: Twenty guidelines were identified. Among them, four guidelines (20%) indicated the use of the GRADE approach. The compliance rate ranged from 33.3%-94.4%. Other approaches, such as the Scottish Intercollegiate Guidelines Network (SIGN) approach, were also used. CONCLUSION: The GRADE approach is rarely followed and inconsistently applied in pressure injury prevention guidelines. Other systems, such as the SIGN approach, are being used despite being outdated or inconsistent. Strategies for further uptake and appropriate application of the GRADE approach among guideline developers are needed in the future.

GRADE Methods in traditional medicine.

The GRADE (Grading of Recommendation, Assessment, Development and Evaluation) methods was developed to evaluate the quality of evidence and make recommendations, which has been widely adopted in clinical practice guidelines. The GRADE methods address the classification of outcomes, systematic collection, appraisal and synthesis of research evidence based on each outcome, evaluate the overall quality of the evidence, and making recommendations. This essay summarized the GRADE methods and its use in clinical practice guidelines of traditional, complementary, and integrative medicine as well as highlighting some of the challenges.

Visual transformation for guidelines presentation of the strength of recommendations and the certainty of evidence.

OBJECTIVE: The objective of this paper is to propose an approach to visual unification of adapted guidelines and transformation of classifications of certainty of evidence (CoE) and strength of recommendations (SoR) into the approach suggested by the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) working group. STUDY DESIGN AND SETTING: We carried out a literature search in MEDLINE and Epistemonikos, an analysis of selected guidelines, and an iterative discussion to decide on a consistent visual presentation and CoE and SoR depictions. RESULTS: The results of the literature search suggested this issue had not been addressed yet. The analysis of the chosen eight guidelines showed significant heterogeneity in the visual presentation of recommendations. Recommendations were often worded similarly to whether or not they were strong or conditional. Many guidelines contained "statements," almost all of which did not fulfill the good practice statement (GPS) criteria. We proposed an approach for transforming recommendations that are being adapted and which use various classification systems for CoE and SoR into GRADE and a consistent visual style. CONCLUSION: Guideline developers should aim for unification in the formulation of recommendations to improve transferability.

Systematic reviews on interventions for COVID-19 have rarely graded the certainty of the evidence

ABSTRACT BACKGROUND: Numerous systematic reviews on coronavirus disease-19 (COVID-19) treatment have been developed to provide syntheses of the large volume of primary studies. However, the methodological quality of most of these reviews is questionable and the results provided may therefore present bias. OBJECTIVE: To investigate how many systematic reviews on the therapeutic or preventive options for COVID-19 assessed the certainty of the evidence through the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach. METHODS: We conducted a sensitive search in MEDLINE (via PubMed) and included all systematic reviews that assessed any intervention for COVID-19. The systematic reviews included were examined to identify any planned and/or actual assessment using the GRADE approach (or absence thereof) regarding the certainty of the evidence. RESULTS: We included 177 systematic reviews and found that only 37 (21%; 37/177) assessed and reported the certainty of the evidence using the GRADE approach. This number reduced to 27 (16.2%; 27/167) when Cochrane reviews (n = 10), in which an evaluation using GRADE is mandatory, were excluded. CONCLUSION: Most of the systematic reviews on interventions relating to COVID-19 omitted assessment of the certainty of the evidence. This is a critical methodological omission that must not be overlooked in further research, so as to improve the impact and usefulness of syntheses relating to COVID-19.

Guideline developers in the United States were inconsistent in applying criteria for appropriate Grading of Recommendations, Assessment, Development and Evaluation use.

OBJECTIVES: We assessed whether guidelines published by organizations based in the United States comply with published criteria for the use of the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) approach. STUDY DESIGN AND SETTING: We performed a cross-sectional study of all clinical practice guidelines that indicated the use of the GRADE approach, were published between 2011 and 2018, and listed in the National Guidelines Clearinghouse. RESULTS: We included 67 guideline documents from 44 of 135 (32.6%) US-based organizations that indicated the use of the GRADE approach. The majority (60/67, 89.6%) of guidelines defined the certainty of evidence consistent with GRADE, but only approximately 1 in 10 (7/67, 10.4%) explicitly reported consideration of all eight criteria to assess the certainty in the evidence for rating down and up. A majority of guidelines (36/67, 53.7%) provided a summary of the evidence, described explicit consideration of all four central domains (36/67, 53.7%), and rated the strength of recommendation consistent with GRADE (36/67, 53.7%). CONCLUSION: Approximately one in three US-based organizations developing evidence-based guidelines report the use of GRADE, but adherence to published criteria is inconsistent. As uptake of the GRADE approach increases in the United States, continued efforts to train guideline methodologists and panel members are important.

Clinical practice guidelines in pediatric anesthesia: What constitutes high-quality guidance?

The explosion of scientific evidence has outstripped the ability of individual clinicians to acquire, process, and apply it within a clinical context. Clinical practice guidelines bridge the gap between this large body of evidence and clinical practice by translating evidence into recommendations. As such, they are an important extension of the evidence-based medicine paradigm. Guidelines promise to translate evidence into actionable clinical recommendations. However, this promise can only be realized if they are both useful and trustworthy. As the number of guidelines increases users' question their role, the quality of the scientific evidence behind them and the strength of the recommendations made. In this article, we provide definitions and outline the features of the common forms of clinical guidance. We then describe the essential characteristics of good quality clinical guidance and outline initiatives aimed at improving quality. Specific issues and limitations related to guideline development in perioperative care of children are discussed. Finally, two clinical guidance documents, published recently in Pediatric Anesthesia, are discussed based upon these criteria.

Determinants Used to Justify the Strength of Recommendations among Korean Clinical Practice Guidelines.

BACKGROUND: A standardized systematic approach to grade evidence and the strength of recommendations is important for guideline users to minimize bias and help interpret the most suitable decisions at the point of care. The study aims to identify and classify determinants used to make judgement for the strength of recommendations among 56 Korean clinical practice guidelines (CPGs), and explore strong recommendations based on low quality of evidence. METHODS: Determinants used in the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach among 34 CPGs which have reported both strength of recommendations and level of evidence were reviewed. RESULTS: Five of 34 CPGs (14.7%) considered quality of evidence, benefits and harms, patients' values and preferences, and costs. And 24 of 34 CPGs (70.6%) considered both magnitude of effect and feasibility as additional determinants. Judgement table was not widely provided for use to translate evidence into recommendations. Eighty-two of 121 recommendations (67.8%, ranged 20.0% to 100.0%) among 11 CPGs using the same judgement scheme showed 'strong' strength of recommendations based on low or very low quality of evidence. Among 5 paradigmatic situations that justify strong recommendations based on low or very low evidence, situation classified as 'potential equivalence, one option clearly less risky or costly' was 87.8% for 82 strong recommendations. Situation classified as 'uncertain benefit, certain harm' was 4.9%. CONCLUSION: There is a need to introduce and systematize an evidence-based grading system. Using judgement table to justify the strength of recommendations and applying the 5 paradigmatic situations mentioned above is also recommended in the near future.

Determinants Used to Justify the Strength of Recommendations among Korean Clinical Practice Guidelines

BACKGROUND: A standardized systematic approach to grade evidence and the strength of recommendations is important for guideline users to minimize bias and help interpret the most suitable decisions at the point of care. The study aims to identify and classify determinants used to make judgement for the strength of recommendations among 56 Korean clinical practice guidelines (CPGs), and explore strong recommendations based on low quality of evidence. METHODS: Determinants used in the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach among 34 CPGs which have reported both strength of recommendations and level of evidence were reviewed. RESULTS: Five of 34 CPGs (14.7%) considered quality of evidence, benefits and harms, patients' values and preferences, and costs. And 24 of 34 CPGs (70.6%) considered both magnitude of effect and feasibility as additional determinants. Judgement table was not widely provided for use to translate evidence into recommendations. Eighty-two of 121 recommendations (67.8%, ranged 20.0% to 100.0%) among 11 CPGs using the same judgement scheme showed ‘strong’ strength of recommendations based on low or very low quality of evidence. Among 5 paradigmatic situations that justify strong recommendations based on low or very low evidence, situation classified as ‘potential equivalence, one option clearly less risky or costly’ was 87.8% for 82 strong recommendations. Situation classified as ‘uncertain benefit, certain harm’ was 4.9%. CONCLUSION: There is a need to introduce and systematize an evidence-based grading system. Using judgement table to justify the strength of recommendations and applying the 5 paradigmatic situations mentioned above is also recommended in the near future.

UpToDate adherence to GRADE criteria for strong recommendations: an analytical survey.

INTRODUCTION: UpToDate is widely used by clinicians worldwide and includes more than 9400 recommendations that apply the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework. GRADE guidance warns against strong recommendations when certainty of the evidence is low or very low (discordant recommendations) but has identified five paradigmatic situations in which discordant recommendations may be justified. OBJECTIVES: Our objective was to document the strength of recommendations in UpToDate and assess the frequency and appropriateness of discordant recommendations. DESIGN: Analytical survey of all recommendations in UpToDate. METHODS: We identified all GRADE recommendations in UpToDate and examined their strength (strong or weak) and certainty of the evidence (high, moderate or low certainty). We identified all discordant recommendations as of January 2015, and pairs of reviewers independently classified them either into one of the five appropriate paradigms or into one of three categories inconsistent with GRADE guidance, based on the evidence presented in UpToDate. RESULTS: UpToDate included 9451 GRADE recommendations, of which 6501 (68.8%) were formulated as weak recommendations and 2950 (31.2%) as strong. Among the strong, 844 (28.6%) were based on high certainty in effect estimates, 1740 (59.0%) on moderate certainty and 366 (12.4%) on low certainty. Of the 349 discordant recommendations 204 (58.5%) were judged appropriately (consistent with one of the five paradigms); we classified 47 (13.5%) as good practice statements; 38 (10.9%) misclassified the evidence as low certainty when it was at least moderate and 60 (17.2%) warranted a weak rather than a strong recommendation. CONCLUSION: The proportion of discordant recommendations in UpToDate is small (3.7% of all recommendations) and the proportion that is truly problematic (strong recommendations that would best have been weak) is very small (0.6%). Clinicians should nevertheless be cautious and look for clear explanations-in UpToDate and elsewhere-when guidelines offer strong recommendations based on low certainty evidence.

GRADE guidelines system is reproducible when instructions are clearly operationalized even among the guidelines panel members with limited experience with GRADE.

OBJECTIVES: The Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) is a widely used methodology for the development of clinical practice guideline. Although its reproducibility is good for evaluating the quality of evidence, it has not been tested in context of developing recommendations. The objective of this study was to assess the reproducibility of all GRADE factors that determine the direction and strength of a recommendation among the guideline panel members with limited exposure to GRADE methodology. STUDY DESIGN AND SETTING: The study was conducted as part of the clinical practice guideline development process of American Association of Blood Banking for the use of prophylactic vs. therapeutic platelet transfusion in patients with thrombocytopenia. The results from the systematic review and meta-analysis for each question were summarized as a GRADE evidence profile. Interrater agreement for all GRADE factors and strength of recommendations was summarized using a weighted kappa statistic with 95% confidence intervals (CI). RESULTS: Eighteen members of the panel participated in the deliberation of making recommendations and completed the online questionnaire. They were given two 1-hour lectures about GRADE. The agreement for all domains was better than chance. The interrater agreement for the domain of quality of evidence was good (kappa value: 0.68; 95% CIs: 0.54, 0.84), and fair for balance of benefit and harms (kappa value: 0.4; 95% CIs: 0.25, 0.57) and use of resources (kappa value: 0.28: 95% CIs: 0.12, 0.42). The interrater agreement was moderate for the GRADE domain of patients' values and preferences (kappa value: 0.44; 95% CI: 0.31, 0.56). The interrater agreement for making a for/against recommendation was good (kappa value: 0.74; 95% CIs: 0.33, 0.91) and fair for strong/weak recommendation (kappa value: 0.39; 95% CIs: 0.18, 0.68). CONCLUSIONS: Although not all elements of GRADE system had good agreement, the interrater agreement for assessing the quality of evidence and issuing a recommendation of for vs. against among panel members who had limited exposure to GRADE methodology was good. This is probably because GRADE has operationalized these two areas in more detail than other domains. Further operationalization of all GRADE domains such as with the GRADE evidence to decision frameworks would likely improve its reproducibility.

Treatment of adult patients with schizophrenia and complex mental health needs - A national clinical guideline.

BACKGROUND AND AIM: The Danish Health and Medicines Authority assembled a group of experts to develop a national clinical guideline for patients with schizophrenia and complex mental health needs. Within this context, ten explicit review questions were formulated, covering several identified key issues. METHODS: Systematic literature searches were performed stepwise for each review question to identify relevant guidelines, systematic reviews/meta-analyses, and randomized controlled trials. The quality of the body of evidence for each review question was assessed using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) system. Clinical recommendations were developed on the basis of the evidence, assessment of the risk-benefit ratio, and perceived patient preferences. RESULTS: Based on the identified evidence, a guideline development group (GDG) recommended that the following interventions should be offered routinely: antipsychotic maintenance therapy, family intervention and assertive community treatment. The following interventions should be considered: long-acting injectable antipsychotics, neurocognitive training, social cognitive training, cognitive behavioural therapy for persistent positive and/or negative symptoms, and the combination of cognitive behavioural therapy and motivational interviewing for cannabis and/or central stimulant abuse. SSRI or SNRI add-on treatment for persistent negative symptoms should be used only cautiously. Where no evidence was available, the GDG agreed on a good practice recommendation. CONCLUSIONS: The implementation of this guideline in daily clinical practice can facilitate good treatment outcomes within the population of patients with schizophrenia and complex mental health needs. The guideline does not cover all available interventions and should be used in conjunction with other relevant guidelines.

Treatment of adult patients with schizophrenia and complex mental health needs: a national clinical guideline

BACKGROUND AND AIM: The Danish Health and Medicines Authority assembled a group of experts to develop a national clinical guideline for patients with schizophrenia and complex mental health needs. Within this context, ten explicit review questions were formulated, covering several identified key issues. METHODS: Systematic literature searches were performed stepwise for each review question to identify relevant guidelines, systematic reviews/meta-analyses, and randomized controlled trials. The quality of the body of evidence for each review question was assessed using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) system. Clinical recommendations were developed on the basis of the evidence, assessment of the risk-benefit ratio, and perceived patient preferences. RESULTS: Based on the identified evidence, a guideline development group (GDG) recommended that the following interventions should be offered routinely: antipsychotic maintenance therapy, family intervention and assertive community treatment. The following interventions should be considered: long-acting injectable antipsychotics, neurocognitive training, social cognitive training, cognitive behavioural therapy for persistent positive and/or negative symptoms, and the combination of cognitive behavioural therapy and motivational interviewing for cannabis and/or central stimulant abuse. SSRI or SNRI add-on treatment for persistent negative symptoms should be used only cautiously. Where no evidence was available, the GDG agreed on a good practice recommendation. CONCLUSIONS: The implementation of this guideline in daily clinical practice can facilitate good treatment outcomes within the population of patients with schizophrenia and complex mental health needs. The guideline does not cover all available interventions and should be used in conjunction with other relevant guidelines.(AU)

A practical approach to evidence-based dentistry: VII: How to use patient management recommendations from clinical practice guidelines.

BACKGROUND AND OVERVIEW: Clinical practice guidelines represent highly processed evidence with associated recommendations to inform clinical practice and optimize patient care. Appropriately developed, evidence-based recommendations will integrate the best evidence regarding benefits and harms, the certainty of the evidence, patients' values and preferences, and resource utilization. PRACTICAL IMPLICATIONS: The authors provide a structure for clinicians to critically appraise clinical practice guidelines to determine whether the guidelines offer trustworthy recommendations.

A review of clinical practice guidelines found that they were often based on evidence of uncertain relevance to primary care patients.

OBJECTIVES: Primary care patients typically have less severe illness than those in hospital and may be overtreated if clinical guideline evidence is inappropriately generalized. We aimed to assess whether guideline recommendations for primary care were based on relevant research. STUDY DESIGN AND SETTING: Literature review of all publications cited in support of National Institute for Health and Care Excellence (NICE) recommendations for primary care. The relevance to primary care of all 45 NICE clinical guidelines published in 2010 and 2011, and their recommendations, was assessed by an expert panel. RESULTS: Twenty-two of 45 NICE clinical guidelines published in 2010 and 2011 were relevant to primary care. These 22 guidelines contained 1,185 recommendations, of which 495 were relevant to primary care, and cited evidence from 1,573 research publications. Of these cited publications, 590 (38%, range by guideline 6-74%) were based on patients typical of primary care. CONCLUSION: Nearly two-third (62%) of publications cited to support primary care recommendations were of uncertain relevance to patients in primary care. Guideline development groups should more clearly identify which recommendations are intended for primary care and uncertainties about the relevance of the supporting evidence to primary care patients, to avoid potential overtreatment.