P.Re.Val.E.: outcome research program for the evaluation of health care quality in Lazio, Italy.

BACKGROUND: P.Re.Val.E. is the most comprehensive comparative evaluation program of healthcare outcomes in Lazio, an Italian region, and the first Italian study to make health provider performance data available to the public. The aim of this study is to describe the P.Re.Val.E. and the impact of releasing performance data to the public. METHODS: P.Re.Val.E. included 54 outcome/process indicators encompassing many different clinical areas. Crude and adjusted rates were estimated for the 2006-2009 period. Multivariate regression models and direct standardization procedures were used to control for potential confounding due to individual characteristics. Variable life-adjusted display charts were developed, and 2008-2009 results were compared with those from 2006-2007. RESULTS: Results of 54 outcome indicators were published online at http://www.epidemiologia.lazio.it/prevale10/index.php. Public disclosure of the indicators' results caused mixed reactions but finally promoted discussion and refinement of some indicators. Based on the P.Re.Val.E. experience, the Italian National Agency for Regional Health Services has launched a National Outcome Program aimed at systematically comparing outcomes in hospitals and local health units in Italy. CONCLUSIONS: P.Re.Val.E. highlighted aspects of patient care that merit further investigation and monitoring to improve healthcare services and equity.

Spie charts, target plots, and radar plots for displaying comparative outcomes of health care.

OBJECTIVE: Comparative evaluations of clinical outcomes (e.g., in-hospital mortality, complications after a surgical procedure) or health care processes involve the definition of several indicators for each study unit. Graphical displays are best suited for highlighting the main patterns in the data. The aim of this study was to compare different graphical techniques, including target plots, radar plots, and "spie" charts, for comparing the performances of different health care providers. STUDY DESIGN AND SETTING: Thirteen indicators were calculated and combined in eight composite indices for eight clinical categories of interest. The indices were displayed with target plots, radar plots, and "spie" charts. RESULTS: All the three techniques had an immediate interpretation and were easy to implement. However, target plots failed to highlight small differences between indicators, whereas radar plots were strongly influenced by the order in which the indicators were displayed. Both target and radar plots assumed equal weights for the indicators, and did not allow predetermined judgments on the relative importance of the indicators. "Spie" charts overcame the primary limitations of the other two techniques. Furthermore, they are well suited to summarize the overall performance of a health care provider with a single score. CONCLUSION: "Spie" charts represented the best graphical tool for displaying multivariate health care data in comparative evaluations of clinical outcomes and processes of care among health care providers.

Effectiveness of invasive reperfusion therapy and standard medical treatment in AMI.

OBJECTIVE: Coronary revascularization methods are among the most common major medical procedures performed in industrialized countries. The short- and long-term comparative effectiveness of different techniques remains undetermined. METHODS AND RESULTS: The study population included all adult patients (94,864 subjects) admitted for acute myocardial infarction (AMI) in 2004 in Italy, as recorded in the National Hospital Discharge Records. Invasive reperfusion therapy (coronary artery bypass graft, percutaneous coronary intervention), and standard medical treatments were compared. End points were short-term (30-day) in-hospital mortality and any combination of new revascularizations, re-AMI, stroke, or death (MACCE) occurring within 6 and 12 months from initial treatment. Risk factors and comorbidities were used to define patients' health status. The Cox model was applied to evaluate risk-adjusted hazard ratios (HR) for different approaches. Medical treatment was used as the reference category. Propensity score matching was performed to evaluate selection bias in the allocation to CABG or percutaneous coronary intervention (PCI). Both short-term mortality and 6- and 12-month MACCE risk for invasive reperfusion therapy was significantly lower than the reference (HR: 0.50, P < 0.001; 0.54, P < 0.001; 0.64, P < 0.001, respectively). In the matched population, while short-term mortality was significantly lower for PCI (HR 0.36, P < 0.001), long-term MACCE risk was lower for CABG (6-month HR 2.93, P < 0.001; 12-month HR 3.01, P < 0.001). CONCLUSIONS: In spite of adjustments for patients' health status, medical treatment resulted in a higher mortality and increased MACCE risk. While PCI gave the best short-term results, after 1 year the risk of re-interventions or further events was significantly lower for patients undergoing CABG.

Risk adjustment for inter-hospital comparison of primary cesarean section rates: need, validity and parsimony.

BACKGROUND: Cesarean section rates is often used as an indicator of quality of care in maternity hospitals. The assumption is that lower rates reflect in developed countries more appropriate clinical practice and general better performances. Hospitals are thus often ranked on the basis of caesarean section rates. The aim of this study is to assess whether the adjustment for clinical and sociodemographic variables of the mother and the fetus is necessary for inter-hospital comparisons of cesarean section (c-section) rates and to assess whether a risk adjustment model based on a limited number of variables could be identified and used. METHODS: Discharge abstracts of labouring women without prior cesarean were linked with abstracts of newborns discharged from 29 hospitals of the Emilia-Romagna Region (Italy) from 2003 to 2004. Adjusted ORs of cesarean by hospital were estimated by using two logistic regression models: 1) a full model including the potential confounders selected by a backward procedure; 2) a parsimonious model including only actual confounders identified by the "change-in-estimate" procedure. Hospital rankings, based on ORs were examined. RESULTS: 24 risk factors for c-section were included in the full model and 7 (marital status, maternal age, infant weight, fetopelvic disproportion, eclampsia or pre-eclampsia, placenta previa/abruptio placentae, malposition/malpresentation) in the parsimonious model. Hospital ranking using the adjusted ORs from both models was different from that obtained using the crude ORs. The correlation between the rankings of the two models was 0.92. The crude ORs were smaller than ORs adjusted by both models, with the parsimonious ones producing more precise estimates. CONCLUSION: Risk adjustment is necessary to compare hospital c-section rates, it shows differences in rankings and highlights inappropriateness of some hospitals. By adjusting for only actual confounders valid and more precise estimates could be obtained.

Risk adjustment and outcome research. Part I.

OBJECTIVE: The increasing demand for comparative evaluation of outcomes requires the development and diffusion of epidemiologic research, the ability to correctly formulate hypotheses, to conduct analyses and to interpret the results. The purpose of this paper is to provide a detailed but easy-reading review of epidemiologic methods to compare healthcare outcomes, particularly risk-adjustment methods. METHODS: The paper is divided into three parts. Part I describes confounding in observational studies, the ways confounding is identified and controlled (propensity adjustment and risk adjustment), and the methods for constructing the severity measures in risk-adjustment procedures. CONCLUSIONS: It is becoming increasingly important for policy makers and planners to identify which factors may improve or worsen the effectiveness of treatments and services and to compare the performances of providers. Politicians, managers, epidemiologists, and clinicians should make their decisions based on the validity and precision of study results, by using the best scientific knowledge available. The statistical methods described in this review cannot measure 'reality' as it 'truly' is, but can produce 'images' of it, defining limits and uncertainties in terms of validity and precision. Studies that use credible risk-adjustment strategies are more likely to yield reliable and applicable findings.