Does adding clinical data to administrative data improve agreement among hospital quality measures?

BACKGROUND: Hospital performance measures based on patient mortality and readmission have indicated modest rates of agreement. We examined if combining clinical data on laboratory tests and vital signs with administrative data leads to improved agreement with each other, and with other measures of hospital performance in the nation's largest integrated health care system. METHODS: We used patient-level administrative and clinical data, and hospital-level data on quality indicators, for 2007-2010 from the Veterans Health Administration (VA). For patients admitted for acute myocardial infarction (AMI), heart failure (HF) and pneumonia we examined changes in hospital performance on 30-d mortality and 30-d readmission rates as a result of adding clinical data to administrative data. We evaluated whether this enhancement yielded improved measures of hospital quality, based on concordance with other hospital quality indicators. RESULTS: For 30-d mortality, data enhancement improved model performance, and significantly changed hospital performance profiles; for 30-d readmission, the impact was modest. Concordance between enhanced measures of both outcomes, and with other hospital quality measures - including Joint Commission process measures, VA Surgical Quality Improvement Program (VASQIP) mortality and morbidity, and case volume - remained poor. CONCLUSIONS: Adding laboratory tests and vital signs to measure hospital performance on mortality and readmission did not improve the poor rates of agreement across hospital quality indicators in the VA. INTERPRETATION: Efforts to improve risk adjustment models should continue; however, evidence of validation should precede their use as reliable measures of quality.

How pooling fragmented healthcare encounter data affects hospital profiling.

OBJECTIVES: People receiving healthcare from multiple payers (eg, Medicare and the Veterans Health Administration [VA]) have fragmented health records. How the use of more complete data affects hospital profiling has not been examined. STUDY DESIGN: Retrospective cohort study. METHODS: We examined 30-day mortality following acute myocardial infarction at 104 VA hospitals for veterans 66 years and older from 2006 through 2010 who were also Medicare beneficiaries. Using VA-only data versus combined VA/Medicare data, we calculated 2 risk-standardized mortality rates (RSMRs): 1 based on observed mortality (O/E) and the other from CMS' Hospital Compare program, based on model-predicted mortality (P/E). We also categorized hospital outlier status based on RSMR relative to overall VA mortality: average, better than average, and worse than average. We tested whether hospitals whose patients received more of their care through Medicare would look relatively better when including those data in risk adjustment, rather than including VA data alone. RESULTS: Thirty-day mortality was 14.8%. Adding Medicare data caused both RSMR measures to significantly increase in about half the hospitals and decrease in the other half. O/E RSMR increased in 53 hospitals, on average, by 2.2%, and decreased in 51 hospitals by -2.6%. P/E RSMR increased, on average, by 1.2% in 56 hospitals, and decreased in the others by -1.3%. Outlier designation changed for 4 hospitals using O/E measure, but for no hospitals using P/E measure. CONCLUSIONS: VA hospitals vary in their patients' use of Medicare-covered care and completeness of health records based on VA data alone. Using combined VA/Medicare data provides modestly different hospital profiles compared with those using VA-alone data.

Improving patient care through leadership engagement with frontline staff: a Department of Veterans Affairs case study.

BACKGROUND: Leveraging Frontline Expertise (LFLE) is a patient safety intervention for engaging senior managers with the work-systems challenges faced by frontline workers and ensuring follow-up and accountability for systemic change. A study was conducted to assess the ability to refine, implement, and demonstrate the effectiveness of LFLE, which was designed for and tested in private-sector hospitals, in a Department of Veterans Affairs medical center (VAMC), typically a more hierarchical setting. METHODS: LFLE was pilot tested in an urban, East coast-based VAMC, which implemented LFLE in its emergency department and operating room, with the medical/surgical ward and ICU serving as controls. A 20-month multimethod evaluation involved interviews, observation, data-tracking forms, and surveys to measure participant perceptions of the program, operational benchmarks of effectiveness, and longitudinal change in safety climate. RESULTS: Implementation showed fidelity to program design. Participating units identified 22 improvement opportunities, 16 (73%) of which were fully or partially resolved. Senior managers' attitudes toward LFLE were more positive than those of frontline staff, whose attitudes were mixed. Perceptions of safety climate deteriorated during the study period in the implementation units relative to controls. DISCUSSION: LFLE can be implemented in the VA, yield work-system improvements, and increase alignment of improvement aims and actions across hierarchical levels. Yet the results also warn against dangers inherent in adapting improvement programs to new settings. Findings suggest the need for active listening and learning from frontline staff by senior managers and trust building across hierarchical

Hospital safety climate and safety outcomes: is there a relationship in the VA?

Strengthening safety climate is recognized as a necessary strategy for improving patient safety. Yet there is little empirical evidence linking hospitals' safety climate with safety outcomes.The authors explored the potential relationship between safety climate and Veterans Health Administration hospital safety performance using the Patient Safety Indicator (PSI) rates. Safety climate survey data were merged with hospital discharge data to calculate PSIs. Linear regressions examined the relationship between hospitals' safety climate and dimensions of safety climate with individual PSIs and a PSI composite measure, controlling for organizational-level variables. Safety climate overall was not related to the PSIs or to the PSI composite, although a few individual dimensions of safety climate were associated with specific PSIs. Perceptions of frontline staff were more closely aligned with PSIs than those of senior managers.

Comparing safety climate between two populations of hospitals in the United States.

OBJECTIVE: To compare safety climate between diverse U.S. hospitals and Veterans Health Administration (VA) hospitals, and to explore the factors influencing climate in each setting. DATA SOURCES: Primary data from surveys of hospital personnel; secondary data from the American Hospital Association's 2004 Annual Survey of Hospitals. STUDY DESIGN: Cross-sectional study of 69 U.S. and 30 VA hospitals. DATA COLLECTION: For each sample, hierarchical linear models used safety-climate scores as the dependent variable and respondent and facility characteristics as independent variables. Regression-based Oaxaca-Blinder decomposition examined differences in effects of model characteristics on safety climate between the U.S. and VA samples. PRINCIPAL FINDINGS: The range in safety climate among U.S. and VA hospitals overlapped substantially. Characteristics of individuals influenced safety climate consistently across settings. Working in southern and urban facilities corresponded with worse safety climate among VA employees and better safety climate in the U.S. sample. Decomposition results predicted 1.4 percentage points better safety climate in U.S. than in VA hospitals: -0.77 attributable to sample-characteristic differences and 2.2 due to differential effects of sample characteristics. CONCLUSIONS: Results suggest that safety climate is linked more to efforts of individual hospitals than to participation in a nationally integrated system or measured characteristics of workers and facilities.

Relationship of hospital organizational culture to patient safety climate in the Veterans Health Administration.

Improving safety climate could enhance patient safety, yet little evidence exists regarding the relationship between hospital characteristics and safety climate. This study assessed the relationship between hospitals' organizational culture and safety climate in Veterans Health Administration (VA) hospitals nationally. Data were collected from a sample of employees in a stratified random sample of 30 VA hospitals over a 6-month period (response rate = 50%; n = 4,625). The Patient Safety Climate in Healthcare Organizations (PSCHO) and the Zammuto and Krakower surveys were used to measure safety climate and organizational culture, respectively. Higher levels of safety climate were significantly associated with higher levels of group and entrepreneurial cultures, while lower levels of safety climate were associated with higher levels of hierarchical culture. Hospitals could use these results to design specific interventions aimed at improving safety climate.

Recruitment of hospitals for a safety climate study: facilitators and barriers.

BACKGROUND: Despite increasing emphasis on safety culture assessment, little is known about the factors that affect hospitals' participation in such studies. Factors affecting recruitment of 30 Department of Veterans Affairs (VA) hospitals into a study to evaluate perceptions of safety culture, or safety "climate," were examined. METHODS: To minimize selection bias, hospitals were recruited that represented the spectrum of safety performance on the basis of Patient Safety Indicator scores. Invitations and additional mailings, informational conference calls, and personal contact with hospitals were used to encourage participation. Investigators worked closely with hospitals' key stakeholders to obtain support and buy-in for the study. Relationships among safety performance, organizational culture, and other hospital characteristics with hospitals' participation and ease of recruitment were examined. Findings were compared with those of a companion study in the non-VA setting. RESULTS: Despite attempts to optimize recruitment, it was necessary to contact more than 90 hospitals to obtain a 30-hospital sample. Having a more entrepreneurial culture (associated with risk-taking, innovation, and quality improvement) was significantly related to shorter recruitment time in VA and non-VA settings. Safety performance was significantly related to participation in the VA (that is, "better-performing" hospitals were more likely to be recruited than "lower-performing" hospitals), but not in the non-VA study, where recruitment was based on size and region. DISCUSSION: Researchers should recruit representative samples of hospitals based on measures of safety performance. Hospital selection bias could lead to erroneous findings, ultimately impeding efforts to improve safety within organizations.

An overview of patient safety climate in the VA.

OBJECTIVE: To assess variation in safety climate across VA hospitals nationally. STUDY SETTING: Data were collected from employees at 30 VA hospitals over a 6-month period using the Patient Safety Climate in Healthcare Organizations survey. STUDY DESIGN: We sampled 100 percent of senior managers and physicians and a random 10 percent of other employees. At 10 randomly selected hospitals, we sampled an additional 100 percent of employees working in units with intrinsically higher hazards (high-hazard units [HHUs]). DATA COLLECTION: Data were collected using an anonymous survey design. PRINCIPAL FINDINGS: We received 4,547 responses (49 percent response rate). The percent problematic response--lower percent reflecting higher levels of patient safety climate--ranged from 12.0-23.7 percent across hospitals (mean=17.5 percent). Differences in safety climate emerged by management level, clinician status, and workgroup. Supervisors and front-line staff reported lower levels of safety climate than senior managers; clinician responses reflected lower levels of safety climate than those of nonclinicians; and responses of employees in HHUs reflected lower levels of safety climate than those of workers in other areas. CONCLUSIONS: This is the first systematic study of patient safety climate in VA hospitals. Findings indicate an overall positive safety climate across the VA, but there is room for improvement.

Development and validation of a psychiatric case-mix system.

BACKGROUND: Although difficulties in applying risk-adjustment measures to mental health populations are increasingly evident, a model designed specifically for patients with psychiatric disorders has never been developed. OBJECTIVE: Our objective was to develop and validate a case-mix classification system, the "PsyCMS," for predicting concurrent and future mental health (MH) and substance abuse (SA) healthcare costs and utilization. SUBJECTS: Subjects included 914,225 veterans who used Veterans Administration (VA) healthcare services during fiscal year 1999 (FY99) with any MH/SA diagnosis (International Classification of Diseases, 9th Revision, Clinical Modification [ICD-9-CM] codes 290.00-312.99, 316.00-316.99). METHODS: We derived diagnostic categories from ICD-CM codes using Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition definitions, clinical input, and empiric analyses. Weighted least-squares regression models were developed for concurrent (FY99) and prospective (FY00) MH/SA costs and utilization. We compared the predictive ability of the PsyCMS with several case-mix systems, including adjusted clinical groups, diagnostic cost groups, and the chronic illness and disability payment system. Model performance was evaluated using R-squares and mean absolute prediction errors (MAPEs). RESULTS: Patients with MH/SA diagnoses comprised 29.6% of individuals seen in the VA during FY99. The PsyCMS accounted for a distinct proportion of the variance in concurrent and prospective MH/SA costs (R=0.11 and 0.06, respectively), outpatient MH/SA utilization (R=0.25 and 0.07), and inpatient MH/SA utilization (R=0.13 and 0.05). The PsyCMS performed better than other case-mix systems examined with slightly higher R-squares and lower MAPEs. CONCLUSIONS: The PsyCMS has clinically meaningful categories, demonstrates good predictive ability for modeling concurrent and prospective MH/SA costs and utilization, and thus represents a useful method for predicting mental health costs and utilization.