Identifying Risks and Opportunities in Outpatient Surgical Patient Safety: A Qualitative Analysis of Veterans Health Administration Staff Perceptions.

OBJECTIVES: Little is known about patient safety risks in outpatient surgery. Inpatient surgical adverse events (AEs) risk factors include patient- (e.g., advanced age), process- (e.g., inadequate preoperative assessment), or structure-related characteristics (e.g., low surgical volume); however, these factors may differ from outpatient care where surgeries are often elective and in younger/healthier patients. We undertook an exploratory qualitative research project to identify risk factors for AEs in outpatient surgery. METHODS: We developed a conceptual framework of patient, process, and structure factors associated with surgical AEs on the basis of a literature review. This framework informed our semistructured interview guide with (1) open-ended questions about a specific outpatient AE that the participant experienced and (2) outpatient surgical patient safety risk factors in general. We interviewed nationwide Veterans Health Administration surgical staff. Results were coded on the basis of categories in the conceptual framework, and additional themes were identified using content analysis. RESULTS: Fourteen providers representing diverse surgical roles participated. Ten reported witnessing an AE, and everyone provided input on risk factors in our conceptual framework. We did not find evidence that patient race/age, surgical technique, or surgical volume affected patient safety. Emerging factors included patient compliance, postoperative patient assessments/instruction, operating room equipment needs, and safety culture. CONCLUSIONS: Surgical staff are familiar with AEs and patient safety problems in outpatient surgery. Our results show that processes of care undertaken by surgical providers, as opposed to immutable patient characteristics, may affect the occurrence of AEs. The factors we identified may facilitate more targeted research on outpatient surgical AEs.

Caveats for high reliability in healthcare.

Purpose Given the pace of industry change and the rapid diffusion of high reliability organization (HRO) approaches, lags and divergences have arisen between research and practice in healthcare. The purpose of this paper is to explore several of these theory-practice gaps and propose implications for research and practice. Design/methodology/approach Classic and cutting-edge HRO literature is applied to analyze two industry trends: delivery system integration, and the confluence of patient-as-consumer and patient-centered care. Findings Highly reliable integrated delivery systems will likely function very differently from classic HRO organizations. Both practitioners and researchers should address conditions such as how a system is bounded, how reliable the system should be and how interdependencies are handled. Additionally, systems should evaluate the added uncertainty and variability introduced by enhanced agency on the part of patients/families in decision making and in processes of care. Research limitations/implications Dramatic changes in the sociotechnical environment are influencing the coupling and interactivity of system elements in healthcare. Researchers must address the maintenance of reliability across organizations and the migration of decision-making power toward patients and families. Practical implications As healthcare systems integrate, managers attempting to apply HRO principles must recognize how these systems present new and different reliability-related challenges and opportunities. Originality/value This paper provides a starting point for the advancement of research and practice in high-reliability healthcare by providing an in-depth exploration of the implications of two major industry trends.

Development of an Adverse Event Surveillance Model for Outpatient Surgery in the Veterans Health Administration.

OBJECTIVE: Develop and validate a surveillance model to identify outpatient surgical adverse events (AEs) based on previously developed electronic triggers. DATA SOURCES: Veterans Health Administration's Corporate Data Warehouse. STUDY DESIGN: Six surgical AE triggers, including postoperative emergency room visits and hospitalizations, were applied to FY2012-2014 outpatient surgeries (n = 744,355). We randomly sampled trigger-flagged and unflagged cases for nurse chart review to document AEs and measured positive predictive value (PPV) for triggers. Next, we used chart review data to iteratively estimate multilevel logistic regression models to predict the probability of an AE, starting with the six triggers and adding in patient, procedure, and facility characteristics to improve model fit. We validated the final model by applying the coefficients to FY2015 outpatient surgery data (n = 256,690) and reviewing charts for cases at high and moderate probability of an AE. PRINCIPAL FINDINGS: Of 1,730 FY2012-2014 reviewed surgeries, 350 had an AE (20 percent). The final surveillance model c-statistic was 0.81. In FY2015 surgeries with >0.8 predicted probability of an AE (n = 405, 0.15 percent), PPV was 85 percent; in surgeries with a 0.4-0.5 predicted probability of an AE, PPV was 38 percent. CONCLUSIONS: The surveillance model performed well, accurately identifying outpatient surgeries with a high probability of an AE.

The Nature and Severity of Adverse Events in Select Outpatient Surgical Procedures in the Veterans Health Administration.

BACKGROUND: Research on adverse events (AEs) in outpatient surgery has been limited. As part of a Veterans Health Administration (VA) project on AE surveillance, we chart-reviewed selected outpatient surgical cases to characterize the nature and severity of AEs. METHODS: We abstracted financial year 2012-2015 VA outpatient surgery cases selected with high (n = 1185) and low (n = 1072) likelihood of an AE based on postoperative health care utilization. The abstraction tool included established AE definitions and validated harm and severity scales. RESULTS: We found AEs in 608 high-likelihood (51%) and 126 low-likelihood outpatient surgical procedures (12%). Among 1010 unique AEs, the most common were wound issues (n = 261, 26%), urinary retention (23%), and urinary tract infections (12%). While 63% of all AEs involved minimal harm, 28% required hospitalization, and 9% were severely harmful including 8 AEs requiring intervention to sustain life and 2 deaths. Overall, 102 AEs (10%) required, at minimum, a repeat surgery to treat. CONCLUSIONS: Among VA outpatient surgical procedures selected based on likelihood of an AE, nearly 40% of identified events carried more than minimal patient harm, undermining the claim that outpatient surgery is relatively safe. Prevalent and preventable AEs such as wound dehiscence and urinary retention may be useful targets for quality improvement.

Tailoring an educational program on the AHRQ Patient Safety Indicators to meet stakeholder needs: lessons learned in the VA.

BACKGROUND: Given that patient safety measures are increasingly used for public reporting and pay-for performance, it is important for stakeholders to understand how to use these measures for improvement. The Agency for Healthcare Research and Quality (AHRQ) Patient Safety Indicators (PSIs) are one particularly visible set of measures that are now used primarily for public reporting and pay-for-performance among both private sector and Veterans Health Administration (VA) hospitals. This trend generates a strong need for stakeholders to understand how to interpret and use the PSIs for quality improvement (QI). The goal of this study was to develop an educational program and tailor it to stakeholders' needs. In this paper, we share what we learned from this program development process. METHODS: Our study population included key VA stakeholders involved in reviewing performance reports and prioritizing and initiating quality/safety initiatives. A pre-program formative evaluation through telephone interviews and web-based surveys assessed stakeholders' educational needs/interests. Findings from the formative evaluation led to development and implementation of a cyberseminar-based program, which we tailored to stakeholders' needs/interests. A post-program survey evaluated program participants' perceptions about the PSI educational program. RESULTS: Interview data confirmed that the concepts we had developed for the interviews could be used for the survey. Survey results informed us on what program delivery mode and content topics were of high interest. Six cyberseminars were developed-three of which focused on two content areas that were noted of greatest interest: learning how to use PSIs for monitoring trends and understanding how to interpret PSIs. We also used snapshots of VA PSI reports so that participants could directly apply learnings. Although initial interest in the program was high, actual attendance was low. However, post-program survey results indicated that perceptions about the program were positive. CONCLUSIONS: Conducting a formative evaluation was a highly important process in program development. The useful information that we collected through the interviews and surveys allowed us to tailor the program to stakeholders' needs and interests. Our experiences, particularly with the formative evaluation process, yielded valuable lessons that can guide others when developing and implementing similar educational programs.

Comparing definitions of outpatient surgery: Implications for quality measurement.

BACKGROUND: Adverse event (AE) rates in outpatient surgery are inconsistently reported, partly because of the lack of a standard definition of outpatient surgery. We compared the types and rates of surgical procedures defined by two national healthcare agencies: Health Care Cost Institute (HCCI) and the Healthcare Cost and Utilization Project (HCUP) and considered implications for quality measurement. METHODS: We used HCCI and HCUP definitions to identify FY2012-14 VA outpatient surgeries. RESULTS: There were six times as many HCCI surgeries as HCUP (6,575,830 versus 1,086,640). Ninety-nine percent of HCUP-defined surgeries were also identified by HCCI. More HCUP surgeries had higher average Medicare Relative Value Units then HCCI surgeries [5.3 (SD = 4.4) versus 1.6 (SD = 2.3) RVUs]. CONCLUSIONS: Rates and types of procedures vary widely between definitions. Quality measurement using HCCI versus HCUP may produce significantly lower AE rates because many of the surgeries included reflect low complexity and potentially low risk of AEs.

Applying the High Reliability Health Care Maturity Model to Assess Hospital Performance: A VA Case Study.

BACKGROUND: The lack of a tool for categorizing and differentiating hospitals according to their high reliability organization (HRO)-related characteristics has hindered progress toward implementing and sustaining evidence-based HRO practices. Hospitals would benefit both from an understanding of the organizational characteristics that support HRO practices and from knowledge about the steps necessary to achieve HRO status to reduce the risk of harm and improve outcomes. The High Reliability Health Care Maturity (HRHCM) model, a model for health care organizations' achievement of high reliability with zero patient harm, incorporates three major domains critical for promoting HROs-Leadership, Safety Culture, and Robust Process Improvement ®. A study was conducted to examine the content validity of the HRHCM model and evaluate whether it can differentiate hospitals' maturity levels for each of the model's components. METHODS: Staff perceptions of patient safety at six US Department of Veterans Affairs (VA) hospitals were examined to determine whether all 14 HRHCM components were present and to characterize each hospital's level of organizational maturity. RESULTS: Twelve of the 14 components from the HRHCM model were detected; two additional characteristics emerged that are present in the HRO literature but not represented in the model-teamwork culture and system-focused tools for learning and improvement. Each hospital's level of organizational maturity could be characterized for 9 of the 14 components. DISCUSSION: The findings suggest the HRHCM model has good content validity and that there is differentiation between hospitals on model components. Additional research is needed to understand how these components can be used to build the infrastructure necessary for reaching high reliability.

Partnering With VA Stakeholders to Develop a Comprehensive Patient Safety Data Display: Lessons Learned From the Field.

Health care systems are increasingly burdened by the large numbers of safety measures currently being reported. Within the Veterans Administration (VA), most safety reporting occurs within organizational silos, with little involvement by the frontline users of these measures. To provide a more integrated picture of patient safety, the study team partnered with multiple VA stakeholders and engaged potential frontline users at 2 hospitals to develop a Guiding Patient Safety (GPS) tool. The GPS is currently in its fourth generation; once approval is obtained from senior leadership, implementation will begin. Stakeholders were enthusiastic about the GPS's user-friendly format, comprehensive content, and potential utility for improving safety. These findings suggest that stakeholder engagement is a critical first step in the development of tools that will more likely be used by frontline users. Policy makers and researchers may consider adopting this innovative partnered-research model in developing future national initiatives to deliver meaningful programs to frontline users.

Assessing the potential adoption and usefulness of concurrent, action-oriented, electronic adverse drug event triggers designed for the outpatient setting.

BACKGROUND: Adverse drug event (ADE) detection is an important priority for patient safety research. Trigger tools have been developed to help identify ADEs. In previous work we developed seven concurrent, action-oriented, electronic trigger algorithms designed to prompt clinicians to address ADEs in outpatient care. OBJECTIVES: We assessed the potential adoption and usefulness of the seven triggers by testing the positive predictive validity and obtaining stakeholder input. METHODS: We adapted ADE triggers, "bone marrow toxin-white blood cell count (BMT-WBC)," "bone marrow toxin - platelet (BMT-platelet)," "potassium raisers," "potassium reducers," "creatinine," "warfarin," and "sedative hypnotics," with logic to suppress flagging events with evidence of clinical intervention and applied the triggers to 50,145 patients from three large health care systems. Four pharmacists assessed trigger positive predictive value (PPV) with respect to ADE detection (conservatively excluding ADEs occurring during clinically appropriate care) and clinical usefulness (i.e., whether the trigger alert could change care to prevent harm). We measured agreement between raters using the free kappa and assessed positive PPV for the trigger's detection of harm, clinical usefulness, and both. Stakeholders from the participating health care systems rated the likelihood of trigger adoption and the perceived ease of implementation. FINDINGS: Agreement between pharmacist raters was moderately high for each ADE trigger (kappa free > 0.60). Trigger PPVs for harm ranged from 0 (Creatinine, BMT-WBC) to 17 percent (potassium raisers), while PPV for care change ranged from 0 (WBC) to 60 percent (Creatinine). Fifteen stakeholders rated the triggers. Our assessment identified five of the seven triggers as good candidates for implementation: Creatinine, BMT-Platelet, Potassium Raisers, Potassium Reducers, and Warfarin. CONCLUSIONS: At least five outpatient ADE triggers performed well and merit further evaluation in outpatient clinical care. When used in real time, these triggers may promote care changes to ameliorate patient harm.

Examining the validity of AHRQ's patient safety indicators (PSIs): is variation in PSI composite score related to hospital organizational factors?

Increasing use of Agency for Healthcare Research and Quality's Patient Safety Indicators (PSIs) for hospital performance measurement intensifies the need to critically assess their validity. Our study examined the extent to which variation in PSI composite score is related to differences in hospital organizational structures or processes (i.e., criterion validity). In site visits to three Veterans Health Administration hospitals with high and three with low PSI composite scores ("low performers" and "high performers," respectively), we interviewed a cross-section of hospital staff. We then coded interview transcripts for evidence in 13 safety-related domains and assessed variation across high and low performers. Evidence of leadership and coordination of work/communication (organizational process domains) was predominantly favorable for high performers only. Evidence in the other domains was either mixed, or there were insufficient data to rate the domains. While we found some evidence of criterion validity, the extent to which variation in PSI rates is related to differences in hospitals' organizational structures/processes needs further study.

Improving patient care through leadership engagement with frontline staff: a Department of Veterans Affairs case study.

BACKGROUND: Leveraging Frontline Expertise (LFLE) is a patient safety intervention for engaging senior managers with the work-systems challenges faced by frontline workers and ensuring follow-up and accountability for systemic change. A study was conducted to assess the ability to refine, implement, and demonstrate the effectiveness of LFLE, which was designed for and tested in private-sector hospitals, in a Department of Veterans Affairs medical center (VAMC), typically a more hierarchical setting. METHODS: LFLE was pilot tested in an urban, East coast-based VAMC, which implemented LFLE in its emergency department and operating room, with the medical/surgical ward and ICU serving as controls. A 20-month multimethod evaluation involved interviews, observation, data-tracking forms, and surveys to measure participant perceptions of the program, operational benchmarks of effectiveness, and longitudinal change in safety climate. RESULTS: Implementation showed fidelity to program design. Participating units identified 22 improvement opportunities, 16 (73%) of which were fully or partially resolved. Senior managers' attitudes toward LFLE were more positive than those of frontline staff, whose attitudes were mixed. Perceptions of safety climate deteriorated during the study period in the implementation units relative to controls. DISCUSSION: LFLE can be implemented in the VA, yield work-system improvements, and increase alignment of improvement aims and actions across hierarchical levels. Yet the results also warn against dangers inherent in adapting improvement programs to new settings. Findings suggest the need for active listening and learning from frontline staff by senior managers and trust building across hierarchical

Quality improvement for patient safety: project-level versus program-level learning.

BACKGROUND: Improving quality and patient safety is of increasing strategic importance to health care organizations. However, simply increasing the volume of quality improvement (QI) activity does not necessarily improve patient outcomes. There is a need for greater understanding of QI success factors. PURPOSE: This study looked for differences in QI implementation across hospitals with a range of performance on Patient Safety Indicators. METHODS: We conducted an exploratory comparative case study of 4 Veterans Health Administration hospitals including site visits and interviews with leaders and staff. FINDINGS: Two themes emerged. Project-level QI learning is assessing and modifying specific QI projects relative to expectations. Program-level QI learning is assessing and modifying the overall QI endeavor. The nature of project-level QI learning was similar across sites, whereas we identified qualitative differences across organizations in program-level QI learning. The highest performing organization was evaluating and refining its overall approach to QI, whereas the others were learning how to build and control QI programs. PRACTICE IMPLICATIONS: Program-level QI learning may be key if a QI program is to succeed in improving patient outcomes. This type of organizational learning entails a big-picture, organization-wide view of QI. It also entails second-order organizational learning based on assessment not only of whether QI is being done correctly but also whether the right QI activities are being done, for the right reasons. The organization is "learning to learn." In addition to gaining mastery and control of QI, leaders regularly engage with staff in rethinking QI and experimenting with new approaches. Leaders also assess how QI activity fits in the organization's developmental journey and how it supports realization of strategy.

Consensus building for development of outpatient adverse drug event triggers.

OBJECTIVES: Adverse drug event (ADE) detection is an important priority of patient safety research. Trigger tools have been developed to help identify ADEs. As part of a larger study, we developed complex and specific trigger algorithms intended for concurrent use with clinical care to detect outpatient ADEs. This article assesses the use of a modified Delphi process to obtain expert consensus on the value of these triggers. METHODS: We selected a panel of distinguished clinical and research experts to participate in the modified Delphi process. We created a set of outpatient ADE triggers based on literature review, clinical input, and methodological expertise. The importance of the targeted ADEs, associated drug classes, and trigger logic was used to rate each trigger. Specific criteria were developed to establish consensus. RESULTS: The modified Delphi process established consensus on 6 outpatient ADE triggers to test with patient-level data based on high ratings of utility for patient-level interventions. These triggers focused on detecting ADEs caused by the following drugs or drug classes: bone marrow toxins, potassium raisers, potassium reducers, creatinine, warfarin, and sedative hypnotics. Participants reported including all aspects of the trigger in their ratings, despite our efforts to separate evaluation of clinical need and trigger logic. Participants' expertise affected the evaluation of trigger rules, leading to contradictory feedback on how to improve trigger design. CONCLUSIONS: The efficiency of the modified Delphi method could be improved by allowing participants to produce an overall summary score that incorporates both the clinical value and the general logic of the trigger. Revising and improving trigger design should be conducted in a separate process limited only to trigger experts.

Development of trigger tools for surveillance of adverse events in ambulatory surgery.

BACKGROUND: The trigger tool methodology uses clinical algorithms applied electronically to 'flag' medical records where adverse events (AEs) have most likely occurred. The authors sought to create surgical triggers to detect AEs in the ambulatory care setting. METHODS: Four consecutive steps were used to develop ambulatory surgery triggers. First, the authors conducted a comprehensive literature review for surgical triggers. Second, a series of multidisciplinary focus groups (physicians, nurses, pharmacists and information technology specialists) provided user input on trigger selection. Third, a clinical advisory panel designed an initial set of 10 triggers. Finally, a three-phase Delphi process (surgical and trigger tool experts) evaluated and rated the suggested triggers. RESULTS: The authors designed an initial set of 10 surgical triggers including five global triggers (flagging medical records for the suspicion of any AE) and five AE-specific triggers (flagging medical records for the suspicion of specific AEs). Based on the Delphi rating of the trigger's utility for system-level interventions, the final triggers were: (1) emergency room visit(s) within 21 days from surgery; (2) unscheduled readmission within 30 days from surgery; (3) unscheduled procedure (interventional radiological, urological, dental, cardiac or gastroenterological) or reoperation within 30 days from surgery; (4) unplanned initial hospital length of stay more than 24 h; and (5) lower-extremity Doppler ultrasound order entry and ICD code for deep vein thrombosis or pulmonary embolus within 30 days from surgery. CONCLUSION: The authors therefore propose a systematic methodology to develop trigger tools that takes into consideration previously published work, end-user preferences and expert opinion.

Testing the association between patient safety indicators and hospital structural characteristics in VA and nonfederal hospitals.

This study tested the association between hospital structural characteristics-teaching status, bedsize, and nurse staffing-and potentially preventable adverse events. The authors calculated 14 Agency for Healthcare Research and Quality Patient Safety Indicators (PSIs) and a PSI composite, using discharge databases from VA and nonfederal hospitals. This study compared the likelihood of PSI events in hospitals, controlling for structural and other characteristics, including patients' case-mix. Additional controls were employed to account for differences in VA versus nonfederal patients and data. The study found some associations, most notably a positive (unfavorable) association between status as a major teaching hospital and six PSIs. However, for most PSIs, the authors found no association between the structural characteristics tested and likelihood of PSI events. The study's findings extend previous research showing a lack of consistent relationship between structural characteristics and patient safety. However, the results also suggest continued need for examination of the relationship between teaching status and potentially preventable adverse events.

Validity of selected AHRQ patient safety indicators based on VA National Surgical Quality Improvement Program data.

OBJECTIVES: To examine the criterion validity of the Agency for Health Care Research and Quality (AHRQ) Patient Safety Indicators (PSIs) using clinical data from the Veterans Health Administration (VA) National Surgical Quality Improvement Program (NSQIP). DATA SOURCES: Fifty five thousand seven hundred and fifty two matched hospitalizations from 2001 VA inpatient surgical discharge data and NSQIP chart-abstracted data. STUDY DESIGN: We examined the sensitivities, specificities, positive predictive values (PPVs), and positive likelihood ratios of five surgical PSIs that corresponded to NSQIP adverse events. We created and tested alternative definitions of each PSI. DATA COLLECTION: FY01 inpatient discharge data were merged with 2001 NSQIP data abstracted from medical records for major noncardiac surgeries. PRINCIPAL FINDINGS: Sensitivities were 19-56 percent for original PSI definitions; and 37-63 percent using alternative PSI definitions. PPVs were 22-74 percent and did not improve with modifications. Positive likelihood ratios were 65-524 using original definitions, and 64-744 using alternative definitions. "Postoperative respiratory failure" and "postoperative wound dehiscence" exhibited significant increases in sensitivity after modifications. CONCLUSIONS: PSI sensitivities and PPVs were moderate. For three of the five PSIs, AHRQ has incorporated our alternative, higher sensitivity definitions into current PSI algorithms. Further validation should be considered before most of the PSIs evaluated herein are used to publicly compare or reward hospital performance.

Using patient safety indicators to estimate the impact of potential adverse events on outcomes.

The authors estimated the impact of potentially preventable patient safety events, identified by Agency for Healthcare Research and Quality (AHRQ) Patient Safety Indicators (PSIs), on patient outcomes: mortality, length of stay (LOS), and cost. The PSIs were applied to all acute inpatient hospitalizations at Veterans Health Administration (VA) facilities in fiscal 2001. Two methods-regression analysis and multivariable case matching- were used independently to control for patient and facility characteristics while predicting the effect of the PSI on each outcome. The authors found statistically significant (p < .0001) excess mortality, LOS, and cost in all groups with PSIs. The magnitude of the excess varied considerably across the PSIs. These VA findings are similar to those from a previously published study of nonfederal hospitals, despite differences between VA and non-VA systems. This study contributes to the literature measuring outcomes of medical errors and provides evidence that AHRQ PSIs may be useful indicators for comparison across delivery systems.

Enhancing patient safety through organizational learning: Are patient safety indicators a step in the right direction?

OBJECTIVE: To assess the potential contribution of the Agency for Healthcare Research and Quality Patient Safety Indicators (PSIs) to organizational learning for patient safety improvement. PRINCIPAL FINDINGS: Patient safety improvement requires organizational learning at the system level, which entails changes in organizational routines that cut across divisions, professions, and levels of hierarchy. This learning depends on data that are varied along a number of dimensions, including structure-process-outcome and from granular to high-level; and it depends on integration of those varied data. PSIs are inexpensive, easy to use, less subject to bias than some other sources of patient safety data, and they provide reliable estimates of rates of preventable adverse events. CONCLUSIONS: From an organizational learning perspective, PSIs have both limitations and potential contributions as sources of patient safety data. While they are not detailed or timely enough when used alone, their simplicity and reliability make them valuable as a higher-level safety performance measure. They offer one means for coordination and integration of patient safety data and activity within and across organizations.