Validation of the Antifungal National Antimicrobial Prescribing Survey (AF-NAPS) quality assessment tool.

BACKGROUND: The Antifungal National Antimicrobial Prescribing Survey (AF-NAPS) was developed to undertake streamlined quality audits of antifungal prescribing. The validity and reliability of such tools is not characterized. OBJECTIVES: To assess the validity and reliability of the AF-NAPS quality assessment tool. METHODS: Case vignettes describing antifungal prescribing were prepared. A steering group was assembled to determine gold-standard classifications for appropriateness and guideline compliance. Infectious diseases physicians, antimicrobial stewardship (AMS) and specialist pharmacists undertook a survey to classify appropriateness and guideline compliance of prescriptions utilizing the AF-NAPS tool. Validity was measured as accuracy, sensitivity and specificity compared with gold standard. Inter-rater reliability was measured using Fleiss' kappa statistics. Assessors' responses and comments were thematically analysed to determine reasons for incorrect classification. RESULTS: Twenty-eight clinicians assessed 59 antifungal prescriptions. Overall accuracy of appropriateness assessment was 77.0% (sensitivity 85.3%, specificity 68.0%). Highest accuracy was seen amongst specialist (81%) and AMS pharmacists (79%). Prescriptions with lowest accuracy were in the haematology setting (69%), use of echinocandins (73%), mould-active azoles (75%) and for prophylaxis (71%). Inter-rater reliability was fair overall (0.3906), with moderate reliability amongst specialist pharmacists (0.5304). Barriers to accurate classification were incorrect use of the appropriateness matrix, knowledge gaps and lack of guidelines for some indications. CONCLUSIONS: The AF-NAPS is a valid tool, assisting assessors to correctly classify appropriate prescriptions more accurately than inappropriate prescriptions. Specialist and AMS pharmacists had similar performance, providing confidence that both can undertake AF-NAPS audits to a high standard. Identified reasons for incorrect classification will be targeted in the online tool and educational materials.

Establishing essential metrics for antifungal stewardship in hospitals: the results of an international Delphi survey.

BACKGROUND: Guidance on assessment of the quantity and appropriateness of antifungal prescribing is required to assist hospitals to interpret data effectively and structure quality improvement programmes. OBJECTIVES: To achieve expert consensus on a core set of antifungal stewardship (AFS) metrics and to determine their feasibility for implementation. METHODS: A literature review was undertaken to develop a list of candidate metrics. International experts were invited to participate in sequential web-based surveys to evaluate the importance and feasibility of metrics in the area of AFS using Delphi methodology. Three surveys were completed. Consensus was predefined as ≥80% agreement on the importance of each metric. RESULTS: Eighty-two experts consented to participate from 17 different countries. Response rate for each survey was >80%. The panel included adult and paediatric physicians, microbiologists and pharmacists with diverse content expertise. Consensus was achieved for 38 metrics considered important to routinely include in AFS programmes, and related to antifungal consumption (n = 5), quality of antifungal prescribing and management of invasive fungal infection (IFI) (n = 24), and clinical outcomes (n = 9). Twenty-one consensus metrics were considered to have moderate to high feasibility for routine collection. CONCLUSIONS: The identified core AFS metrics will provide a framework to comprehensively assess the quantity and quality of antifungal prescribing within hospitals to develop quality improvement programmes aimed at improving IFI prevention, management and patient-centred outcomes. A standardized approach will support collaboration and benchmarking to monitor the efficacy of current prophylaxis and treatment guidelines, and will provide important feedback to guideline developers.