How best to measure surgical quality? Comparison of the Agency for Healthcare Research and Quality Patient Safety Indicators (AHRQ-PSI) and the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) postoperative adverse events at a single institution.

BACKGROUND: Evaluating surgical outcomes is an important tool to compare providers and institutions and to drive process improvements. Differing methodologies, however, may provide conflicting measurements of similar clinical outcomes making comparisons difficult. ACS-NSQIP is a validated, risk-adjusted, clinically derived data methodology to compare observed to expected outcomes after a wide variety of operations. The AHRQ-PSI are a set of computer algorithms to identify potential adverse in-patient events using secondary ICD-9-CM diagnosis and procedure codes from hospital discharge abstracts. METHODS: We compared the ACS-NSQIP and AHRQ-PSI methods for hospital general surgical (n = 6565) or vascular surgical inpatients procedures (n = 1041) at a tertiary-care academic institution from April 2006 to June 2009 on 7 adverse event types. RESULTS: ACS-NSQIP inpatient adverse events were identified in 564 (7.4%) patients. AHRQ-PSIs were identified in 268 (3.5%) patients. Only 159 (2.1%) patients had inpatient events identified by both methods. Using ACS-NSQIP as the clinically based standard the sensitivity of the specific AHRQ-PSI ranged from 0.030 for infections to 0.535 for PE/DVT. Positive predictive values of AHRQ-PSI ranged from 18% for hemorrhage/hematoma to 89% for renal failure. Greater agreement at greater ASA class and wound classification was observed. CONCLUSION: AHRQ-PSI algorithms identified less than a third of the ACS-NSQIP clinically important adverse events. Furthermore, the AHRQ-PSI identified a large number of events with no corresponding clinically important adverse outcomes. The sensitivity of the AHRQ-PSI for detecting clinically relevant adverse events identified by the ACS-NSQIP varied widely. The AHRQ-PSI as applied to postoperative patients is a poor measure of quality performance.

Measuring hospital adverse events: assessing inter-rater reliability and trigger performance of the Global Trigger Tool.

OBJECTIVE: To determine the inter-rater reliability of the Institute for Healthcare Improvement's Global Trigger Tool (GTT) in a practice setting, and explore the value of individual triggers. DESIGN: Prospective assessment of application of the GTT to monthly random samples of hospitalized patients at four hospitals across three regions in the USA. SETTING: Mayo Clinic campuses are in Minnesota, Arizona and Florida. PARTICIPANTS: A total of 1138 non-pediatric inpatients from all units across the hospital. INTERVENTION: GTT was applied to randomly selected medical records with independent assessments of two registered nurses with a physician review for confirmation. MAIN OUTCOME MEASURE: The Cohen Kappa coefficient was used as a measure of inter-rater agreement. The positive predictive value was assessed for individual triggers. RESULTS: Good levels of reliability were obtained between independent nurse reviewers at the case-level for both the occurrence of any trigger and the identification of an adverse event. Nurse reviewer agreement for individual triggers was much more varied. Higher agreement appears to occur among triggers that are objective and consistently recorded in selected portions of the medical record. Individual triggers also varied on their yield to detect adverse events. Cases with adverse events had significantly more triggers identified (mean 4.7) than cases with no adverse events (mean 1.8). CONCLUSIONS: The trigger methodology appears to be a promising approach to the measurement of patient safety. However, automated processes could make the process more efficient in identifying adverse events and has a greater potential of improving care delivery and patient 'outcomes'.

Effect of discharge instructions on readmission of hospitalised patients with heart failure: do all of the Joint Commission on Accreditation of Healthcare Organizations heart failure core measures reflect better care?

BACKGROUND: Most nationally standardised quality measures use widely accepted evidence-based processes as their foundation, but the discharge instruction component of the United States standards of Joint Commission on Accreditation of Healthcare Organizations heart failure core measure appears to be based on expert opinion alone. OBJECTIVE: To determine whether documentation of compliance with any or all of the six required discharge instructions is correlated with readmissions to hospital or mortality. RESEARCH DESIGN: A retrospective study at a single tertiary care hospital was conducted on randomly sampled patients hospitalised for heart failure from July 2002 to September 2003. PARTICIPANTS: Applying the Joint Commission on Accreditation of Healthcare Organizations criteria, 782 of 1121 patients were found eligible to receive discharge instructions. Eligibility was determined by age, principal diagnosis codes and discharge status codes. MEASURES: The primary outcome measures are time to death and time to readmission for heart failure or readmission for any cause and time to death. RESULTS: In all, 68% of patients received all instructions, whereas 6% received no instructions. Patients who received all instructions were significantly less likely to be readmitted for any cause (p = 0.003) and for heart failure (p = 0.035) than those who missed at least one type of instruction. Documentation of discharge instructions is correlated with reduced readmission rates. However, there was no association between documentation of discharge instructions and mortality (p = 0.521). CONCLUSIONS: Including discharge instructions among other evidence-based heart failure core measures appears justified.