Detecting adverse events in surgery: comparing events detected by the Veterans Health Administration Surgical Quality Improvement Program and the Patient Safety Indicators.

BACKGROUND: The Patient Safety Indicators (PSIs) use administrative data to screen for select adverse events (AEs). In this study, VA Surgical Quality Improvement Program (VASQIP) chart review data were used as the gold standard to measure the criterion validity of 5 surgical PSIs. Independent chart review was also used to determine reasons for PSI errors. METHODS: The sensitivity, specificity, and positive predictive value of PSI software version 4.1a were calculated among Veterans Health Administration hospitalizations (2003-2007) reviewed by VASQIP (n = 268,771). Nurses re-reviewed a sample of hospitalizations for which PSI and VASQIP AE detection disagreed. RESULTS: Sensitivities ranged from 31% to 68%, specificities from 99.1% to 99.8%, and positive predictive values from 31% to 72%. Reviewers found that coding errors accounted for some PSI-VASQIP disagreement; some disagreement was also the result of differences in AE definitions. CONCLUSIONS: These results suggest that the PSIs have moderate criterion validity; however, some surgical PSIs detect different AEs than VASQIP. Future research should explore using both methods to evaluate surgical quality.

Examining the impact of the AHRQ Patient Safety Indicators (PSIs) on the Veterans Health Administration: the case of readmissions.

BACKGROUND: By focusing primarily on outcomes in the inpatient setting one may overlook serious adverse events that may occur after discharge (eg, readmissions, mortality) as well as opportunities for improving outpatient care. OBJECTIVE: Our overall objective was to examine whether experiencing an Agency for Healthcare Research and Quality Patient Safety Indicator (PSI) event in an index medical or surgical hospitalization increased the likelihood of readmission. METHODS: We applied the Agency for Healthcare Research and Quality PSI software (version 4.1.a) to 2003-2007 Veterans Health Administration inpatient discharge data to generate risk-adjusted PSI rates for 9 individual PSIs and 4 aggregate PSI measures: any PSI event and composite PSIs reflecting "Technical Care," "Continuity of Care," and both surgical and medical care (Mixed). We estimated separate logistic regression models to predict the likelihood of 30-day readmission for individual PSIs, any PSI event, and the 3 composites, adjusting for age, sex, comorbidities, and the occurrence of other PSI(s). RESULTS: The odds of readmission were 23% higher for index hospitalizations with any PSI event compared with those with no event [confidence interval (CI), 1.19-1.26], and ranged from 22% higher for Iatrogenic Pneumothorax (CI, 1.03-1.45) to 61% higher for Postoperative Wound Dehiscence (CI, 1.27-2.05). For the composites, the odds of readmission ranged from 15% higher for the Technical Care composite (CI, 1.08-1.22) to 37% higher for the Continuity of Care composite (CI, 1.26-1.50). CONCLUSIONS: Our results suggest that interventions that focus on minimizing preventable inpatient safety events as well as improving coordination of care between and across settings may decrease the likelihood of readmission.

Validating the patient safety indicators in the Veterans Health Administration: do they accurately identify true safety events?

BACKGROUND: The Agency for Healthcare Research and Quality (AHRQ) Patient Safety Indicators (PSIs) use administrative data to detect potentially preventable in-hospital adverse events. However, few studies have determined how accurately the PSIs identify true safety events. OBJECTIVES: We examined the criterion validity, specifically the positive predictive value (PPV), of 12 selected PSIs using clinical data abstracted from the Veterans Health Administration (VA) electronic medical record as the gold standard. METHODS: We identified PSI-flagged cases from 28 representative hospitals by applying the AHRQ PSI software (v.3.1a) to VA fiscal year 2003 to 2007 administrative data. Trained nurse-abstractors used standardized abstraction tools to review a random sample of flagged medical records (112 records per PSI) for the presence of true adverse events. Interrater reliability was assessed. We evaluated PPVs and associated 95% confidence intervals of each PSI and examined false positive (FP) cases to determine why they were incorrectly flagged and gain insight into how each PSI might be improved. RESULTS: PPVs ranged from 28% (95% CI, 15%-43%) for Postoperative Hip Fracture to 87% (95% CI, 79%-92%) for Postoperative Wound Dehiscence. Common reasons for FPs included conditions that were present on admission (POA), coding errors, and lack of coding specificity. PSIs with the lowest PPVs had the highest proportion of FPs owing to POA. CONCLUSIONS: Overall, PPVs were moderate for most of the PSIs. Implementing POA codes and using more specific ICD-9-CM codes would improve their validity. Our results suggest that additional coding improvements are needed before the PSIs evaluated herein are used for hospital reporting or pay for performance.

Validity of selected Patient Safety Indicators: opportunities and concerns.

BACKGROUND: The Agency for Healthcare Research and Quality (AHRQ) recently designed the Patient Safety Indicators (PSIs) to detect potential safety-related adverse events. The National Quality Forum has endorsed several of these ICD-9-CM-based indicators as quality-of-care measures. We examined the positive predictive value (PPV) of 3 surgical PSIs: postoperative pulmonary embolus and deep vein thrombosis (pPE/DVT), iatrogenic pneumothorax (iPTX), and accidental puncture and laceration (APL). STUDY DESIGN: We applied the AHRQ PSI software (v.3.1a) to fiscal year 2003 to 2007 Veterans Health Administration (VA) administrative data to identify (flag) patients suspected of having a pPE/DVT, iPTX, or APL. Two trained nurse abstractors reviewed a sample of 336 flagged medical records (112 records per PSI) using a standardized instrument. Inter-rater reliability was assessed. RESULTS: Of 2,343,088 admissions, 6,080 were flagged for pPE/DVT (0.26%), 1,402 for iPTX (0.06%), and 7,203 for APL (0.31%). For pPE/DVT, the PPV was 43% (95% CI, 34% to 53%); 21% of cases had inaccurate coding (eg, arterial not venous thrombosis); and 36% featured thromboembolism present on admission or preoperatively. For iPTX, the PPV was 73% (95% CI, 64% to 81%); 18% had inaccurate coding (eg, spontaneous pneumothorax), and 9% were pneumothoraces present on admission. For APL, the PPV was 85% (95% CI, 77% to 91%); 10% of cases had coding inaccuracies and 5% indicated injuries present on admission. However, 27% of true APLs were minor injuries requiring no surgical repair (eg, small serosal bowel tear). Inter-rater reliability was >90% for all 3 PSIs. CONCLUSIONS: Until coding revisions are implemented, these PSIs, especially pPE/DVT, should be used primarily for screening and case-finding. Their utility for public reporting and pay-for-performance needs to be reassessed.

Comparison of in-hospital versus 30-day mortality assessments for selected medical conditions.

BACKGROUND: In-hospital mortality measures such as the Agency for Healthcare Research and Quality (AHRQ) Inpatient Quality Indicators (IQIs) are easily derived using hospital discharge abstracts and publicly available software. However, hospital assessments based on a 30-day postadmission interval might be more accurate given potential differences in facility discharge practices. OBJECTIVES: To compare in-hospital and 30-day mortality rates for 6 medical conditions using the AHRQ IQI software. METHODS: We used IQI software (v3.1) and 2004-2007 Veterans Health Administration (VA) discharge and Vital Status files to derive 4-year facility-level in-hospital and 30-day observed mortality rates and observed/expected ratios (O/Es) for admissions with a principal diagnosis of acute myocardial infarction, congestive heart failure, stroke, gastrointestinal hemorrhage, hip fracture, and pneumonia. We standardized software-calculated O/Es to the VA population and compared O/Es and outlier status across sites using correlation, observed agreement, and kappas. RESULTS: Of 119 facilities, in-hospital versus 30-day mortality O/E correlations were generally high (median: r = 0.78; range: 0.31-0.86). Examining outlier status, observed agreement was high (median: 84.7%, 80.7%-89.1%). Kappas showed at least moderate agreement (k > 0.40) for all indicators except stroke and hip fracture (k ≤ 0.22). Across indicators, few sites changed from a high to nonoutlier or low outlier, or vice versa (median: 10, range: 7-13). CONCLUSIONS: The AHRQ IQI software can be easily adapted to generate 30-day mortality rates. Although 30-day mortality has better face validity as a hospital performance measure than in-hospital mortality, site assessments were similar despite the definition used. Thus, the measure selected for internal benchmarking should primarily depend on the healthcare system's data linkage capabilities.

Trends in the inpatient quality indicators: the Veterans Health Administration experience.

BACKGROUND: The Agency for Healthcare Research and Quality Inpatient Quality Indicators (IQIs), which include in-hospital mortality and utilization rates, have received little attention in the Veterans Health Administration (VA), despite extensive private sector use for quality improvement. OBJECTIVES: We examined the following: the feasibility of applying the IQIs to VA data; temporal trends in national VA IQI rates; temporal and regional IQI trends in geographic areas defined by Veterans Integrated Service Networks' (VISNs); and VA versus non-VA (Nationwide Inpatient Sample) temporal trends. METHODS: We derived VA- and VISN-level IQI observed rates, risk-adjusted rates, and observed to expected ratios (O/Es), using VA inpatient data (2004-2007). We examined the trends in VA- and VISN-level rates using weighted linear regression, variation in VISN-level O/Es, and compared VA to non-VA trends. RESULTS: VA in-hospital mortality rates from selected medical conditions (stroke, hip fracture, pneumonia) decreased significantly over time; procedure-related mortality rates were unchanged. Laparoscopic cholecystectomy rates increased significantly. A few VISNs were consistently high or low outliers for the medical-related mortality IQIs. Within any given year, utilization indicators, especially cardiac catheterization and cholecystectomy, showed the most inter-VISN variation. Compared with the non-VA, VA medical-related mortality rates for the above-mentioned conditions decreased more rapidly, whereas laparascopic cholecystectomy rates rose more steeply. CONCLUSIONS: The IQIs are easily applied to VA administrative data. They can be useful to tracks rate trends over time, reveal variation between sites, and for trend comparisons with other healthcare systems. By identifying potential quality events related to mortality and utilization, they may complement existing VA quality improvement initiatives.

Effects of resident duty hour reform on surgical and procedural patient safety indicators among hospitalized Veterans Health Administration and Medicare patients.

OBJECTIVE: Improving patient safety was a strong motivation behind duty hour regulations implemented by Accreditation Council for Graduate Medical Education on July 1, 2003. We investigated whether rates of patient safety indicators (PSIs) changed after these reforms. RESEARCH DESIGN: Observational study of patients admitted to Veterans Health Administration (VA) (N = 826,047) and Medicare (N = 13,367,273) acute-care hospitals from July 1, 2000 to June 30, 2005. We examined changes in patient safety events in more versus less teaching-intensive hospitals before (2000-2003) and after (2003-2005) duty hour reform, using conditional logistic regression, adjusting for patient age, gender, comorbidities, secular trends, baseline severity, and hospital site. MEASURES: Ten PSIs were aggregated into 3 composite measures based on factor analyses: "Continuity of Care," "Technical Care," and "Other" composites. RESULTS: Continuity of Care composite rates showed no significant changes postreform in hospitals of different teaching intensity in either VA or Medicare. In the VA, there were no significant changes postreform for the technical care composite. In Medicare, the odds of a Technical Care PSI event in more versus less teaching-intensive hospitals in postreform year 1 were 1.12 (95% CI; 1.01-1.25); there were no significant relative changes in postreform year 2. Other composite rates increased in VA in postreform year 2 in more versus less teaching-intensive hospitals (odds ratio, 1.63; 95% CI; 1.10-2.41), but not in Medicare in either postreform year. CONCLUSIONS: Duty hour reform had no systematic impact on PSI rates. In the few cases where there were statistically significant increases in the relative odds of developing a PSI, the magnitude of the absolute increases were too small to be clinically meaningful.

Validity of selected AHRQ patient safety indicators based on VA National Surgical Quality Improvement Program data.

OBJECTIVES: To examine the criterion validity of the Agency for Health Care Research and Quality (AHRQ) Patient Safety Indicators (PSIs) using clinical data from the Veterans Health Administration (VA) National Surgical Quality Improvement Program (NSQIP). DATA SOURCES: Fifty five thousand seven hundred and fifty two matched hospitalizations from 2001 VA inpatient surgical discharge data and NSQIP chart-abstracted data. STUDY DESIGN: We examined the sensitivities, specificities, positive predictive values (PPVs), and positive likelihood ratios of five surgical PSIs that corresponded to NSQIP adverse events. We created and tested alternative definitions of each PSI. DATA COLLECTION: FY01 inpatient discharge data were merged with 2001 NSQIP data abstracted from medical records for major noncardiac surgeries. PRINCIPAL FINDINGS: Sensitivities were 19-56 percent for original PSI definitions; and 37-63 percent using alternative PSI definitions. PPVs were 22-74 percent and did not improve with modifications. Positive likelihood ratios were 65-524 using original definitions, and 64-744 using alternative definitions. "Postoperative respiratory failure" and "postoperative wound dehiscence" exhibited significant increases in sensitivity after modifications. CONCLUSIONS: PSI sensitivities and PPVs were moderate. For three of the five PSIs, AHRQ has incorporated our alternative, higher sensitivity definitions into current PSI algorithms. Further validation should be considered before most of the PSIs evaluated herein are used to publicly compare or reward hospital performance.

Use of VA and Medicare services by dually eligible veterans with psychiatric problems.

OBJECTIVE: To examine how service accessibility measured by geographic distance affects service sector choices for veterans who are dually eligible for veterans affairs (VA) and Medicare services and who are diagnosed with mental health and/or substance abuse (MH/SA) disorders. DATA SOURCES: Primary VA data sources were the Patient Treatment (acute care), Extended Care (long-term care), and Outpatient Clinic files. VA cost data were obtained from (1) inpatient and outpatient cost files developed by the VA Health Economics and Resource Center and (2) outpatient VA Decision Support System files. Medicare data sources were the denominator, Medicare Provider Analysis Review (MEDPAR), Provider-of-Service, Outpatient Standard Analytic and Physician/Supplier Standard Analytic files. Additional sources included the Area Resource File and Census Bureau data. STUDY DESIGN: We identified dually eligible veterans who had either an inpatient or outpatient MH/SA diagnosis in the VA system during fiscal year (FY)'99. We then estimated one- and two-part regression models to explain the effects of geographic distance on both VA and Medicare total and MH/SA costs. PRINCIPAL FINDINGS: Results provide evidence for substitution between the VA and Medicare, demonstrating that poorer geographic access to VA inpatient and outpatient clinics decreased VA expenditures but increased Medicare expenditures, while poorer access to Medicare-certified general and psychiatric hospitals decreased Medicare expenditures but increased VA expenditures. CONCLUSIONS: As geographic distance to VA medical facility increases, Medicare plays an increasingly important role in providing mental health services to veterans.

Using patient safety indicators to estimate the impact of potential adverse events on outcomes.

The authors estimated the impact of potentially preventable patient safety events, identified by Agency for Healthcare Research and Quality (AHRQ) Patient Safety Indicators (PSIs), on patient outcomes: mortality, length of stay (LOS), and cost. The PSIs were applied to all acute inpatient hospitalizations at Veterans Health Administration (VA) facilities in fiscal 2001. Two methods-regression analysis and multivariable case matching- were used independently to control for patient and facility characteristics while predicting the effect of the PSI on each outcome. The authors found statistically significant (p < .0001) excess mortality, LOS, and cost in all groups with PSIs. The magnitude of the excess varied considerably across the PSIs. These VA findings are similar to those from a previously published study of nonfederal hospitals, despite differences between VA and non-VA systems. This study contributes to the literature measuring outcomes of medical errors and provides evidence that AHRQ PSIs may be useful indicators for comparison across delivery systems.

Performance of statistical models to predict mental health and substance abuse cost.

BACKGROUND: Providers use risk-adjustment systems to help manage healthcare costs. Typically, ordinary least squares (OLS) models on either untransformed or log-transformed cost are used. We examine the predictive ability of several statistical models, demonstrate how model choice depends on the goal for the predictive model, and examine whether building models on samples of the data affects model choice. METHODS: Our sample consisted of 525,620 Veterans Health Administration patients with mental health (MH) or substance abuse (SA) diagnoses who incurred costs during fiscal year 1999. We tested two models on a transformation of cost: a Log Normal model and a Square-root Normal model, and three generalized linear models on untransformed cost, defined by distributional assumption and link function: Normal with identity link (OLS); Gamma with log link; and Gamma with square-root link. Risk-adjusters included age, sex, and 12 MH/SA categories. To determine the best model among the entire dataset, predictive ability was evaluated using root mean square error (RMSE), mean absolute prediction error (MAPE), and predictive ratios of predicted to observed cost (PR) among deciles of predicted cost, by comparing point estimates and 95% bias-corrected bootstrap confidence intervals. To study the effect of analyzing a random sample of the population on model choice, we re-computed these statistics using random samples beginning with 5,000 patients and ending with the entire sample. RESULTS: The Square-root Normal model had the lowest estimates of the RMSE and MAPE, with bootstrap confidence intervals that were always lower than those for the other models. The Gamma with square-root link was best as measured by the PRs. The choice of best model could vary if smaller samples were used and the Gamma with square-root link model had convergence problems with small samples. CONCLUSION: Models with square-root transformation or link fit the data best. This function (whether used as transformation or as a link) seems to help deal with the high comorbidity of this population by introducing a form of interaction. The Gamma distribution helps with the long tail of the distribution. However, the Normal distribution is suitable if the correct transformation of the outcome is used.

Tracking rates of Patient Safety Indicators over time: lessons from the Veterans Administration.

BACKGROUND: The Patient Safety Indicators (PSIs), developed by the Agency for Healthcare Research and Quality, are useful screening tools for highlighting areas in which quality should be further investigated and providing useful benchmarks for tracking progress. OBJECTIVES: Our objectives were to: 1) provide a descriptive analysis of the incidence of PSI events from 2001 to 2004 in the Veterans Health Administration (VA); 2) examine trends in national PSI rates at the hospital discharge level over time; and 3) assess whether hospital characteristics (eg, teaching status, number of beds, and degree of quality improvement implementation) and baseline safety-related hospital performance predict future hospital safety-related performance. METHODS: We examined changes in risk-adjusted PSI rates at the discharge level, calculated the correlation between hospitals' risk-adjusted PSI rates in 2001 with subsequent years, and developed generalized linear models to examine predictors of hospitals' 2004 risk-adjusted PSI rates. RESULTS: Risk-adjusted rates of 2 of the 15 PSIs demonstrated significant trends over time. Rates of iatrogenic pneumothorax increased over time, whereas rates of failure to rescue decreased. Most PSIs demonstrated consistent rates over time. After accounting for patient and hospital characteristics, hospitals' baseline risk-adjusted PSI rates were the most important predictors of their 2004 risk-adjusted rates for 8 PSIs. CONCLUSIONS: The PSIs are useful tools for tracking and monitoring patient safety events in the VA. Future research should investigate whether trends reflect better or worse care or increased attention to documenting patient safety events.

Development and validation of a psychiatric case-mix system.

BACKGROUND: Although difficulties in applying risk-adjustment measures to mental health populations are increasingly evident, a model designed specifically for patients with psychiatric disorders has never been developed. OBJECTIVE: Our objective was to develop and validate a case-mix classification system, the "PsyCMS," for predicting concurrent and future mental health (MH) and substance abuse (SA) healthcare costs and utilization. SUBJECTS: Subjects included 914,225 veterans who used Veterans Administration (VA) healthcare services during fiscal year 1999 (FY99) with any MH/SA diagnosis (International Classification of Diseases, 9th Revision, Clinical Modification [ICD-9-CM] codes 290.00-312.99, 316.00-316.99). METHODS: We derived diagnostic categories from ICD-CM codes using Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition definitions, clinical input, and empiric analyses. Weighted least-squares regression models were developed for concurrent (FY99) and prospective (FY00) MH/SA costs and utilization. We compared the predictive ability of the PsyCMS with several case-mix systems, including adjusted clinical groups, diagnostic cost groups, and the chronic illness and disability payment system. Model performance was evaluated using R-squares and mean absolute prediction errors (MAPEs). RESULTS: Patients with MH/SA diagnoses comprised 29.6% of individuals seen in the VA during FY99. The PsyCMS accounted for a distinct proportion of the variance in concurrent and prospective MH/SA costs (R=0.11 and 0.06, respectively), outpatient MH/SA utilization (R=0.25 and 0.07), and inpatient MH/SA utilization (R=0.13 and 0.05). The PsyCMS performed better than other case-mix systems examined with slightly higher R-squares and lower MAPEs. CONCLUSIONS: The PsyCMS has clinically meaningful categories, demonstrates good predictive ability for modeling concurrent and prospective MH/SA costs and utilization, and thus represents a useful method for predicting mental health costs and utilization.

Health status among 28,000 women veterans. The VA Women's Health Program Evaluation Project.

BACKGROUND: Male veterans receiving Veterans Health Administration (VA) care have worse health than men in the general population. Less is known about health status in women veteran VA patients, a rapidly growing population. OBJECTIVE: To characterize health status of women (vs men) veteran VA patients across age cohorts, and assess gender differences in the effect of social support upon health status. DESIGN AND PATIENTS: Data came from the national 1999 Large Health Survey of Veteran Enrollees (response rate 63%) and included 28,048 women and 651,811 men who used VA in the prior 3 years. MEASUREMENTS: Dimensions of health status from validated Veterans Short Form-36 instrument; social support (married, living arrangement, have someone to take patient to the doctor). RESULTS: In each age stratum (18 to 44, 45 to 64, and > or =65 years), Physical Component Summary (PCS) and Mental Component Summary (MCS) scores were clinically comparable by gender, except that for those aged > or =65, mean MCS was better for women than men (49.3 vs 45.9, P<.001). Patient gender had a clinically insignificant effect upon PCS and MCS after adjusting for age, race/ethnicity, and education. Women had lower levels of social support than men; in patients aged <65, being married or living with someone benefited MCS more in men than in women. CONCLUSIONS: Women veteran VA patients have as heavy a burden of physical and mental illness as do men in VA, and are expected to require comparable intensity of health care services. Their ill health occurs in the context of poor social support, and varies by age.

Evaluating the patient safety indicators: how well do they perform on Veterans Health Administration data?

BACKGROUND: The Patient Safety Indicators (PSIs), an administrative data-based tool developed by the Agency for Healthcare Research and Quality, are increasingly being used to screen for potential in-hospital patient safety problems. Although the Veterans Health Administration (VA) is a national leader in patient safety, accurate information on the epidemiology of patient safety events in the VA is still unavailable. OBJECTIVES: Our objectives were to: (1) apply the AHRQ PSI software to VA administrative data to identify potential instances of compromised patient safety; (2) determine occurrence rates of PSI events in the VA; and (3) examine the construct validity of the PSIs. METHODS: We examined differences between observed and risk-adjusted PSI rates in the VA, compared VA and non-VA PSI rates, and investigated the construct validity of the PSIs by examining correlations of the PSIs with other outcomes of VA hospitalizations. RESULTS: We identified 11,411 PSI events in the VA nationwide in FY'01. Observed PSI rates per 1000 discharges ranged from 0.007 for "transfusion reaction" to 155.5 for "failure to rescue." There were significant, although small, differences between VA and non-VA risk-adjusted PSI rates. Hospitalizations with PSI events had longer lengths of stay, higher mortality, and higher costs than those without PSI events. CONCLUSIONS: Our results suggest that the PSIs may be useful as a patient safety screening tool in the VA. Our PSI rates were consistent with the national incidence of low rates; however, differences between VA and non-VA rates suggest that inadequate case-mix adjustment may be contributing to these findings.

Dual-system use: are there implications for risk adjustment and quality assessment?

The authors examined the implications of dual-system use for risk adjustment and quality assessment. The sample (n = 34 151) included all veterans dually enrolled in the Veterans Health Administration (VA) and the private sector in 1998 with (1) an inpatient discharge from either a VA or Medicare setting for 1 of 6 conditions/procedures and (2) inpatient and/or outpatient use in both the VA and private sector. The authors used the Diagnostic Cost Groups risk-adjustment system to obtain concurrent and prospective health status (relative risk scores) using veterans' Medicare diagnoses only, VA diagnoses only, and diagnoses from both systems. Both concurrent and prospective relative risk scores increased when diagnoses from both systems were used; the population's disease profile also was affected. The authors conclude that it is important to capture the true disease burden of the population by obtaining diagnoses from all health care systems providing care to facilitate meaningful comparisons of performance.

Burden of medical illness in women with depression and posttraumatic stress disorder.

BACKGROUND: Depression and posttraumatic stress disorder (PTSD) are important women's health issues. Depression is known to be associated with poor physical health; however, associations between physical health and PTSD, a common comorbidity of depression, have received less attention. OBJECTIVES: To examine number of medical symptoms and physical health status in women with PTSD across age strata and benchmark them against those of women with depression alone or with neither depression nor PTSD. METHODS: A random sample of Veterans Health Administration enrollees received a mailed survey in 1999-2000 (response rate, 63%). The 30 865 women respondents were categorized according to whether a health care provider had ever told them that they had PTSD, depression (without PTSD), or neither. Outcomes were self-reported medical conditions and physical health status measured with the Veterans SF-36 instrument, a version of the Medical Outcomes Study 36-Item Short-Form Health Survey (SF-36) modified for use in veteran populations. RESULTS: Across age strata, women with PTSD (n = 4348) had more medical conditions and worse physical health status (physical functioning, role limitations due to physical problems, bodily pain, and energy/vitality scales from the Veterans SF-36) than women with depression alone (n = 7580) or neither (n = 18 937). In age-adjusted analyses, the Physical Component Summary score was on average 3.4 points lower in women with depression alone and 6.3 points lower in women with PTSD than in women with neither (P<.001). CONCLUSIONS: Posttraumatic stress disorder is associated with a greater burden of medical illness than is seen with depression alone. The presence of PTSD may account for an important component of the excess medical morbidity and functional status limitations seen in women with depression.

Case-mix adjusting performance measures in a veteran population: pharmacy- and diagnosis-based approaches.

OBJECTIVE: To compare the rankings for health care utilization performance measures at the facility level in a Veterans Health Administration (VHA) health care delivery network using pharmacy- and diagnosis-based case-mix adjustment measures. DATA SOURCES/STUDY SETTING: The study included veterans who used inpatient or outpatient services in Veterans Integrated Service Network (VISN) 20 during fiscal year 1998 (October 1997 to September 1998; N = 126,076). Utilization and pharmacy data were extracted from VHA national databases and the VISN 20 data warehouse. STUDY DESIGN: We estimated concurrent regression models using pharmacy or diagnosis information in the base year (FY1998) to predict health service utilization in the same year. Utilization measures included bed days of care for inpatient care and provider visits for outpatient care. PRINCIPAL FINDINGS: Rankings of predicted utilization measures across facilities vary by case-mix adjustment measure. There is greater consistency within the diagnosis-based models than between the diagnosis- and pharmacy-based models. The eight facilities were ranked differently by the diagnosis- and pharmacy-based models. CONCLUSIONS: Choice of case-mix adjustment measure affects rankings of facilities on performance measures, raising concerns about the validity of profiling practices. Differences in rankings may reflect differences in comparability of data capture across facilities between pharmacy and diagnosis data sources, and unstable estimates due to small numbers of patients in a facility.

Do different case-mix measures affect assessments of provider efficiency? Lessons from the Department of Veterans Affairs.

Although case-mix adjustment is critical for provider profiling, little is known regarding whether different case-mix measures affect assessments of provider efficiency. We examine whether two case-mix measures, Adjusted Clinical Groups (ACGs) and Diagnostic Cost Groups (DCGs), result in different assessments of efficiency across service networks within the Department of Veterans Affairs (VA). Three profiling indicators examine variation in resource use. Although results from the ACGs and DCGs generally agree on which networks have greater or lesser efficiency than average, assessments of individual network efficiency vary depending upon the case-mix measure used. This suggests that caution should be used so that providers are not misclassified based on reported efficiency.

Applying diagnostic cost groups to examine the disease burden of VA facilities: comparing the six "Evaluating VA Costs" study sites with other VA sites and Medicare.

OBJECTIVES: To compare the disease burden of Veterans Health Administration (VA) patients at six study sites with all other VA patients and the Medicare population. DESIGN: A 60% random sample of all VA veteran patients during federal fiscal year 1997 was obtained from administrative databases. A split-sample technique provided a 40% sample (n = 1,046,803) for development and a 20% sample (n = 524,461) for validation. We selected the six study sites from the 40% sample, yielding a total of 50,080 patients in those sites. METHODS: We used Diagnostic Cost Groups to classify patients into clinical groupings based on age, gender, and International Classification of Diseases, Ninth Revision, Clinical Modification diagnoses. The Diagnostic Cost Group model produces relative risk scores that describe patients' expected resource use normalized to the Medicare population. We compared the severity of the six sites with each other and with all other VA facilities and the severity of VA patients with that of Medicare beneficiaries. RESULTS: There were minor statistically significant differences between the study sites and all other VA facilities. Compared with the Medicare population, VA's population was younger and had lower expected resource use (relative risk scores were 1.0 and 0.76, respectively). CONCLUSIONS: Disease burden of the six study sites is representative of all other VA facilities. Although lower relative risk scores suggest that VA patients are healthier than Medicare beneficiaries, when age is taken into account, scores are more comparable. Interpreting the expected resource utilization of the VA population against other benchmarks should be performed carefully.