Factors affecting appropriateness of hospital utilization in two hospitals in Turkey.

To determine the extent of inappropriate hospital use, to investigate factors related to variations in appropriateness, and to identify reasons for inappropriateness, the Appropriateness Evaluation Protocol (AEP) was applied to 2,067 patient days in two hospitals between March 1997 and 1998 in Ankara, Turkey. A substantial amount of inappropriate utilization was found in both hospitals (34.2%, 24.6%). Factors affecting the appropriateness of hospital utilization and reasons for inappropriateness were varied and presented by internal medicine, general surgery, and gynecology services. In general, results of the logistic regression analysis indicated that inappropriateness was significantly associated with admission number (first admission/readmission), admission route (emergent/non-emergent), and day of the week. The most common reason for inappropriateness was diagnostic procedures and/or treatments that could have been carried out on an ambulatory basis. This study demonstrates that the AEP can be used as a tool to improve the efficiency of the Turkish hospitals.

Liability and validity of the Appropriateness Evaluation Protocol in Turkey.

OBJECTIVE: To assess the inter-rater reliability between nurses and the convergent validity of the Appropriateness Evaluation Protocol (AEP) in the Turkish context. METHODS: Two nurses applied the original AEP concurrently to a random subsample of 335 patient-days in internal medicine, general surgery, and gynaecology departments at a university hospital and a government teaching hospital, as a part of a larger study. Inter-rater reliability was tested by calculating overall agreement and specific agreements between nurse reviewers' AEP assessments. Validity was tested by comparing the assessments of the nurses based on the AEP with the implicit judgements of five expert physicians on a random subsample of 818 patient-days. Sensitivity, specificity, positive and negative predictive values of the AEP were calculated. Reliability and validity were also evaluated by the K statistic. RESULTS: In the reliability test, there was a high level of agreement between the two independent raters applying the AEP in the three departments studied: overall agreement = 90.7-97.6%; specific inappropriate agreement = 69.1-92.3%; specific appropriate agreement = 88.3-96.6%. In validity testing, the AEP had a sensitivity of 0.83-0.97, specificity of 0.62-0.80, and positive and negative predictive values of 0.84-0.88 and 0.73-0.95 respectively. Kappa coefficients in internal medicine and gynaecology indicated almost perfect agreement in reliability testing and moderate agreement in validity testing. In general surgery, the K coefficients showed substantial agreement in both tests. CONCLUSION: These results indicate that the AEP is a reliable and valid instrument to assess appropriateness of patient-days in Turkey.

Perceived quality of care in pregnancy assessment before and after delivery.

OBJECTIVE: To assess the perceived quality of care in a group of pregnant women attended in a public Hospital. STUDY DESIGN: All pregnant women seen at the Hospital Clinic of Barcelona in 1996. Two study groups were defined: group A, women seen at the outpatient clinic as a regular follow-up visit for pregnancy, and group B, women seen at the outpatient clinic for follow-up after delivery. A satisfaction questionnaire survey was used in a random sample of both groups of women. RESULTS: Total number of interviews performed was 174. Both groups, A and B were comparable. Scores recorded in both groups were significantly different (P<0.01) for the clinical follow-up and privacy variables (regarded as better than expected). The difference in scores for the information supplied was also statistically significant (P<0.01), but regarded as worst than expected. CONCLUSIONS: The analysis of satisfaction does not seem to follow a linear, straightforward explanation. The differences seen strongly suggest the need of patient's satisfaction surveys to be specific by dimension and tailored to patients' expectations.

Reliability study of the European appropriateness evaluation protocol.

OBJECTIVE: To help to co-ordinate and harmonize research on utilization review in Europe, the US Appropriateness Evaluation Protocol (f¿EP) was adapted for use in the European setting. The aim of this paper is to assess the reliability of the European version of the AEP (EU-AEP). DESIGN: Nineteen English-language medical records were reviewed by a physician reviewer from each of six participating countries: Austria, France, Italy, Spain, Switzerland and the UK. Each of the six reviewers was asked to assess the appropriateness of the 19 admissions and 31 hospitalization days (19 admission days and 12 randomly selected days of hospital stay, excluding days of discharge) using the revised review instrument. To evaluate inter-rater reliability, the kappa statistic was used to measure overall and pair-wise agreement for the assessment of appropriateness of admission and of day of care, respectively. RESULTS: For admission, the overall kappa statistic among the six reviewers was 0.64, with kappa values for each pair of reviewers in the range 0.46-0.86. For day of care, the kappa was 0.59, with pair-wise kappa coefficients in the range 0.25-0.95. CONCLUSION: The observed agreement could be considered substantial, especially if the fact that medical records were hand-written in a language native to only one of the reviewers is considered. Besides all the study limitations, this finding provides at least preliminary support for the application of the EU-AEP as a reliable instrument in the European setting, including application in comparative studies involving two or more countries.

Competition and quality among managed care plans in the USA. Boston University Health Care Management Program Group.

This paper examines the popular idea that competition among managed care plans will lead not only to lower prices, but also to improved quality. We explore the likelihood that competition based on quality will occur and that better quality care will result. First, we discuss key elements of competitive theory and then we attempt to apply them to markets for health care coverage and care. We identify the conditions necessary for competition to have the desired effects and assess the extent to which those conditions do or can exist. We conclude that in the USA, many consumers have no choice among plans and, therefore, cannot select one based on quality. Moreover, the evidence suggests that as long as price varies among health plans, consumers who do have a choice will tend to emphasize price, not quality, in making their selections. We conclude with suggestions to increase the likelihood that quality can improve as a result of competition.

Applications of statistical quality control to cardiac surgery.

BACKGROUND: Although originally developed for use in manufacturing statistical quality control techniques may be applicable to other frequently performed, standardized processes. METHODS: We employed statistical quality control charts (X- s, p, and u) to analyze perioperative morbidity and mortality and length of stay in 1,131 nonemergent, isolated, primary coronary bypass operations conducted within a 17-quarter time period. RESULTS: The incidence of the most common adverse outcomes, including death, myocardial infarction, stroke, and atrial fibrillation, appeared to follow the laws of statistical fluctuation and were in statistical control. Postoperative bleeding, leg-wound infection, and the summation of total and major complications were out of statistical control in the early quarters of the study period but showed progressive improvement, as did postoperative length of stay. CONCLUSIONS: The incidence of morbidity and mortality after primary, isolated, nonemergent coronary bypass operations may be described by standard models of statistical fluctuation. Statistical quality control may be a valuable method to analyze the variability of these adverse postoperative events over time, with the ultimate goal of reducing that variability and producing better outcomes.

High hospital admission rates and inappropriate care.

This study tests whether the rate of inappropriate hospital admissions is high in areas with high medical admission rates. Seventy small geographic areas were formed by grouping Massachusetts ZIP codes by similarity of hospital use. Appropriateness of hospital admission was measured both by applying the Appropriateness Evaluation Protocol and by applying physicians' judgment to the medical records of patients age sixty-five and older who were admitted for treatment of a medical condition in 1990-1992. No relationship between hospital admission rate and inappropriate admission rate was found, which calls into question the common assumption that areas with higher hospital use have more inappropriate use of hospital care.

The evolution of hospital utilization review methods in the United States.

The purpose of this paper is to provide a description of utilization review (UR) methods that have been used in the United States and the direction UR is likely to take in the future. In contrast to the European studies reported in this issue, which consist primarily of empirical studies quantifying the magnitude of inappropriate hospital utilization, the focus of this paper is on the methods used to perform UR operationally. Particular attention will be paid to the predominant form of UR employed in the United States, concurrent review of the need for a hospital level of care, and on the criteria used to assess appropriateness of hospital admissions and days.

An integer programming model to limit hospital selection in studies with repeated sampling.

OBJECTIVE: We describe an integer programming model that, for studies requiring repeated sampling from hospitals, can aid in selecting a limited set of hospitals from which medical records are reviewed. STUDY SETTING: The model is illustrated in the context of two studies: (1) an analysis of the relationship between variations in hospital admission rates across geographic areas and rates of inappropriate admissions; and (2) a validation of computerized algorithms that screen for complications of hospital care. STUDY DESIGN: Common characteristics of the two studies: (1) hospitals are classified into categories, e.g., high, medium, and low; (2) the classification process is repeated several times, e.g., for different medical conditions; (3) medical records are selected separately for each iteration of the classification; and (4) for budgetary and logistical reasons, reviews must be concentrated in a relatively small subset of hospitals. DATA COLLECTION/EXTRACTION METHODS. In each study, hospitals are ranked based on analysis of hospital discharge abstract data. CONCLUSIONS: The model is useful for identifying a subset of hospitals at which more intensive reviews will be conducted.

Small area variations in hospitalization rates: how much you see depends on how you look.

This research investigates the degree that estimates of the magnitude of small area variations in hospitalization rates depend on both the estimation method and the number of years of data used. Hospital discharge abstracts for patients 65 and older from acute care hospitals in Massachusetts from 1982 to 1987 were analyzed. The SCV statistic, the approach used in many current small area variation studies, and empirical Bayes (EB), an approach that adjusts more fully for the effect of random variation, were compared. EB estimates based on 3 years of data were best able to predict future area-specific hospitalization rates. Compared to EB estimates using 3 years of data, the SCV statistic with 1 year of data overestimated the median amount of systematic variation by over 70% for the 68 conditions studied; with 3 years of data, the SCV overestimated the median by 55%. Regardless of method, the same conditions were identified as relatively more variable and the same geographic areas were found to have higher than expected hospitalization rates. The magnitude of differences in hospitalization rates depends on how the data are analyzed and how many years of data are used. Hospitalization rates across small geographic areas may vary substantially less than reported previously.

The utility of severity of illness information in assessing the quality of hospital care. The role of the clinical trajectory.

This research explored whether differentiating patients whose severity of illness worsened, improved, or remained the same over the hospital stay is a good screen for quality of care. The hypothesis was that substandard care is more likely to occur among patients who have worsened. Severity was measured using the Computerized Severity Index (CSI) and MedisGroups in 233 patients who had experienced acute myocardial infarction and 279 who had undergone coronary artery bypass graft who were admitted to four New England hospitals in 1987. Deaths and patients with discharge diagnoses indicating iatrogenic events and complications were oversampled. Potential quality problems were identified through explicit screening criteria applied by nurse researchers and implicit physician reviews. Acute myocardial infarction patients who worsened had higher rates of potential quality problems than other patients (CSI, P = 0.06; MedisGroups, P = 0.01). For the CSI, the 49.4% of patients who worsened captured 70.6% of the potentially substandard care; for MedisGroups, the 35.6% of patients who worsened also encompassed 70.6% of the problematic cases. For coronary artery bypass graft, results varied depending on how severity and quality were defined. The CSI performed better using implicit physician review to identify problematic care (P = 0.00), capturing 76.5% of substandard cases among the 41.6% of patients who worsened. In contrast, MedisGroups did better using explicit quality screens (P = 0.04), grouping 60.5% of the problematic cases among the 47.0% of patients who worsened. After removing in-hospital deaths from consideration, a worsening trajectory was generally associated with a higher fraction of potential quality problems among live discharges. This preliminary study suggests that examining changes in illness severity may be a useful screen for substandard hospital care, but its utility could vary by condition and by how quality problems are defined.

The role of severity information in health policy debates: a survey of state and regional concerns.

Severity of illness measurement has recently dominated many regional health policy debates, and some states now require severity ratings for inpatients. We summarize results of a telephone survey of regional activities involving severity data. Parties use severity information either to evaluate hospital resource use or to assist in comparing quality of hospital care. For quality assessment, various constituencies frequently specify different goals for the severity information. Business representatives commonly believe that it can quantify hospital performance and help them target cost-effective providers; in contrast, providers view severity information only as a screen for substandard care, suggesting areas requiring detailed examination.

Using utilization review information to improve hospital efficiency.

Hospitals are currently under great pressure to improve the efficiency of internal operations without sacrificing quality of care. In the rush to do this, they often overlook an extremely useful source of information that already exists--data routinely collected as part of the utilization review (UR) process. This article describes a system using UR data for management purposes that was developed in a large urban teaching hospital. The components described are: (1) data collected systematically by trained reviewers applying the Appropriateness Evaluation Protocol; (2) software for data collection using inexpensive, highly portable computers; and (3) formats for reporting UR findings to hospital administrators and physicians. Information derived from UR in the study hospital is discussed, as well as factors to be considered in adapting some or all of the system's components in other hospitals.

The Self-Adapting Focused Review System. Probability sampling of medical records to monitor utilization and quality of care.

Medical record review is increasing in importance as the need to identify and monitor utilization and quality of care problems grow. To conserve resources, reviews are usually performed on a subset of cases. If judgment is used to identify subgroups for review, this raises the following questions: How should subgroups be determined, particularly since the locus of problems can change over time? What standard of comparison should be used in interpreting rates of problems found in subgroups? How can population problem rates be estimated from observed subgroup rates? How can the bias be avoided that arises because reviewers know that selected cases are suspected of having problems? How can changes in problem rates over time be interpreted when evaluating intervention programs? Simple random sampling, an alternative to subgroup review, overcomes the problems implied by these questions but is inefficient. The Self-Adapting Focused Review System (SAFRS), introduced and described here, provides an adaptive approach to record selection that is based upon model-weighted probability sampling. It retains the desirable inferential properties of random sampling while allowing reviews to be concentrated on cases currently thought most likely to be problematic. Model development and evaluation are illustrated using hospital data to predict inappropriate admissions.

Assessing the need to hospitalize children: pediatric appropriateness evaluation protocol.

Rapidly increasing hospital costs have necessitated use review of hospitalized patients to improve the appropriateness (medical necessity) of hospital use. The development and testing of the Pediatric Appropriateness Evaluation Protocol, an objective, criteria-based instrument intended to assist physicians and use reviewers in making decisions regarding appropriateness of pediatric hospital admissions and days of care, are described.

Assessing the appropriateness of hospital utilization to improve efficiency and competitive position.

A hospital, induced by a certificate of need process and a newly competitive health care environment, made the transition from passive response to outside pressures to active utilization control.

Policy issues related to prospective payment for pediatric hospitalization.

Children's hospitals have been excluded from the Medicare prospective payment system (PPS) because of concerns over the applicability of the DRG case-mix system and PPS payment weights to pediatric hospitalization. Nevertheless, DRG-based payment systems are being adopted by State Medicaid agencies and private third-party payers, and the Health Care Financing Administration has been mandated to report to Congress on the feasibility of including children's hospitals in the Federal PPS. This article summarizes policy research on this issue and discusses options in the design of prospective payment systems for pediatric hospitalization.