Performance monitoring in cardiac surgery: application of statistical process control to a single-site database.

BACKGROUND: Graphical Statistical Process Control (SPC) tools have been shown to promptly identify significant variations in clinical outcomes in a range of health care settings. We explored the application of these techniques to quantitatively inform the routine cardiac surgical (CAS) morbidity and mortality (M&M) review processes at a single site. METHODS: Baseline clinical and procedural data relating to 5265 consecutive cardiac surgical procedures, performed at St Andrew's War Memorial Hospital (SAWMH) between the 1st January 2003 and the 30th April 2012, were retrospectively evaluated. A range of appropriate clinical outcome indicators (COIs) were developed and evaluated using a combination of Cumulative Sum charts, Exponentially Weighted Moving Average charts and Funnel Plots. Charts were updated regularly and discussed at the cardiac surgery unit's bi-monthly M&M meetings. Risk adjustment (RA) for the COIs was developed and validated for incorporation into the charts to improve monitoring performance. RESULTS: Discrete and aggregated measures, including blood product/reoperation, major acute post-procedural complications, cardiopulmonary bypass duration and Length of Stay/Readmission < 28 days have proved to be valuable measures for monitoring outcomes. Instances of variation in performance identified using the charts were examined thoroughly and could be related to changes in clinical practice (e.g. antifibrinolytic use) as well as differences in individual operator performance (in some instances, driven by case mix). CONCLUSIONS: SPC tools can promptly detect meaningful changes in clinical outcome thereby allowing early intervention to address altered performance. Careful interpretation of charts for group and individual operators has proven helpful in detecting and differentiating systemic versus individual variation.

Use of graphical statistical process control tools to monitor and improve outcomes in cardiac surgery.

BACKGROUND: Graphical Statistical Process Control (SPC) tools have been shown to promptly identify significant variations in clinical outcomes in a range of health care settings. We explored the application of these techniques to qualitatively inform the routine cardiac surgical morbidity and mortality (M&M) review process at a single site. METHODS: Baseline clinical and procedural data relating to 4774 consecutive cardiac surgical procedures, performed between the 1st January 2003 and the 30th April 2011, were retrospectively evaluated. A range of appropriate performance measures and benchmarks were developed and evaluated using a combination of CUmulative SUM (CUSUM) charts, Exponentially Weighted Moving Average (EWMA) charts and Funnel Plots. Charts have been discussed at the unit's routine M&M meetings. Risk adjustment (RA) based on EuroSCORE has been incorporated into the charts to improve performance. RESULTS: Discrete and aggregated measures, including Blood Product/Reoperation, major acute post-procedural complications and Length of Stay/Readmission<28 days have proved to be usable measures for monitoring outcomes. Monitoring trends in minor morbidities provides a valuable warning of impending changes in significant events. Instances of variation in performance have been examined and could be related to differences in individual operator performance via individual operator curves. CONCLUSION: SPC tools facilitate near "real-time" performance monitoring allowing early detection and intervention in altered performance. Careful interpretation of charts for group and individual operators has proven helpful in detecting and differentiating systemic vs. individual variation.

Impact of quantitative feedback and benchmark selection on radiation use by cardiologists performing cardiac angiography.

Audit of and feedback on both group and individual data provided immediately after the point of care and compared with realistic benchmarks of excellence have been demonstrated to drive change. This study sought to evaluate the impact of immediate benchmarked quantitative case-based performance feedback on the clinical practice of cardiologists practicing at a private hospital in Brisbane, Australia. The participating cardiologists were assigned to one of two groups: Group 1 received patient and procedural details for review and Group 2 received Group 1 data plus detailed radiation data relating to the procedures and comparative benchmarks. In Group 2, Linear-by-Linear Association analysis suggests a link between change in radiation use and initial radiation dose category (p=0.014) with only those initially 'challenged' by the benchmarks showing improvement. Those not 'challenged' by the benchmarks deteriorated in performance compared with those starting well below the benchmarks showing greatest increase in radiation use. Conversely, those blinded to their radiation use (Group 1) showed general improvement in radiation use throughout the study compared with those performing initially close to the benchmarks showing greatest improvement. This study shows that use of non-challenging benchmarks in case-based radiation risk feedback does not promote a reduction in radiation use; indeed, it may contribute to increased doses. Paradoxically, cardiologists who are aware of performance monitoring but blinded to individual case data appear to maintain, if not reduce, their radiation use.

Data Quality Improvement in Clinical Databases Using Statistical Quality Control: Review and Case Study.

Ensuring the quality of data being collected in clinical and medical contexts is a concern for data managers and users. Quality assurance frameworks, systematic audits, and correction procedures have been proposed to enhance the accuracy and completeness of databases. Following an overview of the undertaken approaches, particularly statistical methods, the authors promote acceptance sampling plans (ASPs) and statistical process control (SPC) tools, including control charts and root cause analysis, as the technical core of the data quality improvement mechanism. They review ASP and SPC techniques and discuss their implementation in data quality evaluation and improvement. Two case studies are presented in which the authors apply some of the techniques to databases maintained by a local hospital. Finally, guidelines are proposed for which techniques are appropriate with regard to dataflow and database specifications.

The role of quantitative feedback in coronary angiography radiation reduction.

OBJECTIVE: To evaluate the benefits of radiation education with and without feedback reporting in altering clinician radiation use behaviour in performing coronary angiography (CA). DESIGN: A retrospective review of radiation use (fluoroscopy time) in coronary angiograms performed between July 1996 and December 2005 by 10 cardiologists to assess the impact of various interventions aimed at minimizing radiation risk. The impact of interventions such as education and audit/feedback was correlated against radiation use using cumulative sum and cumulative expected minus observed charts. SETTING: Private Hospital in Brisbane, Australia. PARTICIPANTS: Ten cardiologists. INTERVENTION: Education and audit/feedback. RESULTS: Baseline radiation use subject to standard guidelines was stable. Group performance charts show a modest transient improvement in radiation use associated with an education intervention alone. However, regular detailed personalized feedback comparing an individual's radiation use to group and external benchmarks was successful in achieving sustained reduction in overall radiation use. For individual participants, significant improvement was noted in 7 of 10 cardiologists. CONCLUSION: Although an improved theoretical understanding of effective radiation hygiene strategies might contribute to reduced radiation use, this study suggests that regular detailed quantitative feedback supporting education is an effective tool in altering radiation use in CA. Understanding triggers that stimulate change in clinician behaviour is critical to the design of systems to optimize clinical performance. Confidentially reported benchmarking systems may be a useful tool to alter clinician behaviour.