[Methodological Standard for the Development of Quality Indicators within Clinical Practice Guidelines - Results of a structured consensus process]. / Methodischer Standard für die Entwicklung von Qualitätsindikatoren im Rahmen von S3-Leitlinien ­ Ergebnisse einer strukturierten Konsensfindung.

BACKGROUND: Recommendations of evidence- and formally consensus-based clinical practice guidelines (CPGs) represent a valuable source of quality indicators (QIs). Nevertheless, a standardized methodological procedure for developing QIs in the context of CPGs does not yet exist in Germany for all CPGs. For this reason, a methodological standard for the guideline-based development of QIs (QI Standard) was developed based on a structured consensus process involving multiple key stakeholders. METHODS: The proposed content of the QI Standard was derived from evidence, drawing upon results of reviews and qualitative studies, and considered German manuals for guideline-based QI development of two guideline programs. A multi-perspective consensus panel, broadly representing key stakeholders from the German healthcare system with expertise in CPGs and/or quality management, was nominated to vote on recommendations for guideline-based development of QIs. The iterative, structured consensus process included a two-stage online survey based on the Delphi method ("preliminary voting") and a moderated final stakeholder conference where all those recommendations were definitely included in the QI Standard that received approval of more than 75 % (consensus criterion) of the consensus panel. RESULTS: Based on the agreed QI Standard, the QI development process starts with a criteria-based selection of "potential" QIs which - in case of adoption - are published in CPGs as "preliminary" QIs and can achieve the status "final" after successful testing. The QI Standard is composed of a total of 30 recommendations, which are allocated to six areas: A) preparatory work steps for the guideline-based recommendation of QIs, B) QI development group and cooperation with the CPG group, C) development of potential QIs, D) critical appraisal of potential QIs, E) formal adoption and publication as well as F) piloting/testing of preliminary QIs and conversion into final QIs. DISCUSSION: Before the QI Standard can be recommended for implementation in future CPGs, it should have been successfully tested in selected German CPG projects. In addition to methodological requirements for the QI development, it must be ensured that guideline groups have adequate resources for the implementation of the QI Standard. CONCLUSION: By using the QI Standard, scientifically sound and healthcare-relevant QIs can be expected.

Correction to: Guideline-based quality indicators-a systematic comparison of German and international clinical practice guidelines.

An amendment to this paper has been published and can be accessed via the original article.

[(How) Are quality indicators for measuring and appraising the quality of healthcare derived from evidence-based clinical practice guidelines? A review]. / (Wie) Erfolgt die Ableitung von Qualitätsindikatoren zur Messung und Bewertung der Versorgungsqualität im Rahmen von S3-Leitlinien? Eine Übersichtsarbeit.

BACKGROUND: Evidence-based and formally consensus-based clinical practice guidelines (CPGs) offer potential for the development of quality indicators (QIs). Although QIs are recommended as part of some CPGs, there is no accepted gold standard for the specific development process of guideline-based QIs. The purpose of this review, which is embedded in a mixed-methods research project, was to analyze the current state of methodological approaches for QI development in German CPGs to derive insights for the development of a national evidence-guided and consensus-based standard for guideline-based development of QIs. METHODS: In order to identify valid CPGs containing recommendations for QIs, a search was carried out (July 31, 2016) via the guideline database of the German Association of the Scientific Medical Societies (AWMF). Based on a stratified random sample per guideline program (guidelines published by medical societies, National Program for Disease Management Guidelines (DMG), and the German Guideline Program in Oncology [GGPO]), 11 CPGs were selected. With regard to QIs, the specific development methodology, indications on their psychometric properties and how the quality of care should be examined by recommended QIs were extracted and compared by using the guideline documents. RESULTS: In 35 of the 109 (16/85 medical societies, 4/8 DMG, 15/16 GGPO) (32 %) valid CPGs, a total of 372 QIs were recommended. Based on 11 randomly selected guidelines (5 published by medical societies, 1 DMG, 5 GGPO; a total of 109 QIs), the QI development methodology was inconsistent in all five medical societies guidelines (including QI presentation, usage and selection of guideline recommendations for QI derivation) compared to DMG and GGPO. Based on all 109 QIs, 2 (2 %) were presented as a quantitative measure with a reference range, and quality objectives were formulated for 17 (16 %). There was no guideline explicitly reporting about the results of a pilot study or data-based analysis of the psychometric properties of the recommended QIs. The GGPO guideline documents were the only ones providing information on the assessment of the quality of care based on recommended QIs. DISCUSSION: The usage of the QI manuals of the DMG and GGPO leads to a largely standardized development of guideline-based QIs. In the CPGs of the medical societies - if at all - QIs are developed inconsistently and mostly unsystematically. Due to largely missing reference ranges and quality objectives, the identified QIs cannot yet be used to transparently identify potential quality deficits in health care. This requires results of pilot studies and further development of guideline-based QI. CONCLUSIONS: A standard for QI development is needed for German guideline authors to seize the opportunity and develop clinically relevant, widely accepted and evidence-based QIs in the guideline development process. In addition, it must be ensured that appropriate structures are used or set up in order to be able to apply the recommended QIs in the German healthcare system.

Guideline-based quality indicators-a systematic comparison of German and international clinical practice guidelines.

BACKGROUND: Evidence-based clinical practice guidelines (CPGs) are relevant sources for generating quality indicators (QIs). The objective of this study was to compare guideline-based QIs of German and international CPGs and their underlying methodological approaches. METHODS: We conducted systematic searches in the guideline databases of G-I-N (Guidelines International Network) and NGC (National Guideline Clearinghouse) between February and June 2017 to identify international CPGs matching the topics of German evidence-based CPGs (n = 35) that report QIs, which were identified in a preceding study. Additionally, we searched the websites of the particular CPG providers for separate documents with regard to QIs. We included evidence-based CPGs which report QIs. Reported QIs, the underlying guideline recommendations, and information on methods of development were extracted. The selection and extraction of CPGs were conducted by one reviewer and checked by another. For each matched pair of CPGs, we assessed whether the suggested QIs matched or were not directly comparable. RESULTS: Twenty-five international CPGs, originating from seven CPG providers in total, met the criteria for inclusion. They matched the topics of 18 German CPGs. This resulted in 30 CPG pairs for the comparison of QIs (some of the international CPGs matched the topic of more than one German CPG). We found 27 QI pairs with QIs "not different or slightly different", corresponding to 13% (27 of 212) of the QIs in German CPGs and 16% (27 of 166) in international CPGs. Only two QI pairs were judged to be "different/inconsistent". For 183 of 212 (86%) QIs from German CPGs and 137 of 166 (83%) QIs from international CPGs, no direct comparison could be made. An explicit link to one or more guideline recommendations was found for 136 of 152 (89%) QIs from German CPGs and 82 of 166 (49%) QIs from international CPGs. Some information on methods for the development of QIs existed for 12 of 18 (67%) German CPGs and 8 of 25 (32%) international CPGs. CONCLUSIONS: The majority of QIs in German and international CPGs were not comparable. Various reasons for this are conceivable. More transparent reporting of the underlying methods for generating guideline-based QIs is needed.