Quality collaboratives: lessons from research.

Quality improvement collaboratives are increasingly being used in many countries to achieve rapid improvements in health care. However, there is little independent evidence that they are more cost effective than other methods, and little knowledge about how they could be made more effective. A number of systematic evaluations are being performed by researchers in North America, the UK, and Sweden. This paper presents the shared ideas from two meetings of these researchers. The evidence to date is that some collaboratives have stimulated improvements in patient care and organisational performance, but there are significant differences between collaboratives and teams. The researchers agreed on the possible reasons why some were less successful than others, and identified 10 challenges which organisers and teams need to address to achieve improvement. In the absence of more conclusive evidence, these guidelines are likely to be useful for collaborative organisers, teams and their managers and may also contribute to further research into collaboratives and the spread of innovations in health care.

Evaluating an integrated approach to clinical quality improvement: clinical guidelines, quality measurement, and supportive system design.

BACKGROUND: Implementing clinical practice guidelines to change patient outcomes presents a challenge. Studies of single interventions focused on changing provider behavior demonstrate modest effects, suggesting that effective guideline implementation requires a multifaceted approach. Traditional biomedical research designs are not well suited to evaluating systems interventions. OBJECTIVES: RAND and the Army Medical Department collaborated to develop and evaluate a system for implementing guidelines and documenting their effects on patient care. RESEARCH DESIGN: The evaluation design blended quality improvement, case study, and epidemiologic methods. A formative evaluation of implementation process and an outcome evaluation of patient impact were combined. SUBJECTS: Guidelines were implemented in 3 successive demonstrations targeting low back pain, asthma, and diabetes. This paper reports on the first wave of 4 facilities implementing a low back pain guideline. METHODS: Organizational climate and culture, motivation, leadership commitment, and resources were assessed. Selected indicators of processes and outcomes of care were compared before, during, and after guideline implementation at the demonstration facilities and at comparison facilities. Logistic regression analysis was used to test for guideline effects on patient care. RESULTS: Process evaluation documented varied approaches to quality improvement across sites. Outcome evaluation revealed a significant downward trend in the percentage of acute low back pain patients referred to physical therapy or chiropractic care (10.7% to 7.2%) at demonstration sites and no such trend at control sites. CONCLUSIONS: Preliminary results suggest the power of this design to stimulate improvements in guideline implementation while retaining the power to evaluate rigorously effects on patient care.

Optimal methods for guideline implementation: conclusions from Leeds Castle meeting.

BACKGROUND: Quality problems in medical care are not a new finding. Variations in medical practice as well as actual medical errors have been pointed out for many decades. The current movement to write practice guidelines to attempt to correct these deviations from recommended medical practice has not solved the problem. OBJECTIVE: In order to gain greater acceptance of these guidelines and to change the behavior of health care providers, the science of guideline implementation must be understood better. RESEARCH DESIGN: A group of experts who have studied the problem of implementation in Europe and the United States was convened. This meeting summary enumerates the implementation methods studied to date, reviews the theories of behavioral change, and makes recommendation for effecting better implementation guidelines. RESULTS: A research agenda was proposed to further our knowledge of effective evidence-based implementation.

Implementation of clinical guidelines via a computer charting system: effect on the care of febrile children less than three years of age.

OBJECTIVE: The authors have shown that clinical guidelines embedded in an electronic medical record improved the quality, while lowering the cost, of care for health care workers who incurred occupational exposures to body fluid. They seek to determine whether this system has similar effects on the emergency department care of young children with febrile illness. DESIGN: Off-on-off, interrupted time series with intent-to-treat analysis. SETTING: University hospital emergency department. SUBJECTS: 830 febrile children less than 3 years of age and the physicians who treated them. INTERVENTIONS: Implementation of an electronic medical record that provides real-time advice regarding the content of the history and physical examination and recommendations regarding laboratory testing, treatment, diagnosis, and disposition. MEASUREMENTS: Documentation of essential items in the medical record and after-care instructions; compliance with guidelines regarding testing, treatment, and diagnosis; charges. RESULTS: The computer was used in 64 percent of eligible cases. Mean percentage documentation of 21 essential history and physical examination items increased from 80 percent during the baseline period to 92 percent in the intervention phase (13 percent increase; 95 percent CI, 10-15 percent). Mean percentage documentation of ten items in the after-care instructions increased from 48 percent at baseline to 81 percent during the intervention phase (33 percent increase; 95 percent confidence interval, 28-38 percent). All documentation decreased to baseline when the computer system was removed. There were no demonstrable improvements in appropriateness of care, nor was there evidence that appropriateness worsened. Mean charges were not changed by the intervention. CONCLUSION: The intervention markedly improved documentation, had little effect on the appropriateness of the process of care, and had no effect on charges. Results for the febrile child module differ from those for the module for occupational blood and body fluid exposure (a more focused and straightforward medical condition), underscoring the need for implementation methods to be tailored to specific clinical complaints.

Building better team charters: an example from reengineering the preoperative system.

This article illustrates a six-step chartering method using a successful project at the West Los Angeles VA Medical Center. The annual savings generated from this project were estimated at $13 million.

Estimating effectiveness of cardiac arrest interventions: a logistic regression survival model.

BACKGROUND: The study objective was to develop a simple, generalizable predictive model for survival after out-of-hospital cardiac arrest due to ventricular fibrillation. METHODS AND RESULTS: Logistic regression analysis of two retrospective series (n=205 and n=1667, respectively) of out-of-hospital cardiac arrests was performed on data sets from a Southwestern city (population, 415,000; area, 406 km2) and a Northwestern county (population, 1,038,000; area, 1399 km2). Both are served by similar two-tiered emergency response systems. All arrests were witnessed and occurred before the arrival of emergency responders, and the initial cardiac rhythm observed was ventricular fibrillation. The main outcome measure was survival to hospital discharge. Patient age, initiation of CPR by bystanders, interval from collapse to CPR, interval from collapse to defibrillation, bystander CPR/collapse-to-CPR interval interaction, and collapse-to-CPR/collapse-to-defibrillation interval interaction were significantly associated with survival. There was not a significant difference between observed survival rates at the two sites after control for significant predictors. A simplified predictive model retaining only collapse to CPR and collapse to defibrillation intervals performed comparably to the more complicated explanatory model. CONCLUSIONS: The effectiveness of prehospital interventions for out-of-hospital cardiac arrest may be estimated from their influence on collapse to CPR and collapse to defibrillation intervals. A model derived from combined data from two geographically distinct populations did not identify site as a predictor of survival if clinically relevant predictor variables were controlled for. This model can be generalized to other US populations and used to project the local effectiveness of interventions to improve cardiac arrest survival.

Implementation of clinical guidelines using a computer charting system. Effect on the initial care of health care workers exposed to body fluids.

CONTEXT: While clinical guidelines are considered an important mechanism to improve the quality of medical care, problems with implementation may limit their effectiveness. Few empirical data exist about the effect of computer-based systems for application of clinical guidelines on quality of care. OBJECTIVE: To determine whether real-time presentation of clinical guidelines using an electronic medical record can increase compliance with guidelines. DESIGN: Prospective off-on-off, interrupted time series with intent-to-treat analysis. SETTING: University hospital emergency department. SUBJECTS: Patients were 280 health care workers (50 in the baseline control phase, 156 in the intervention phase, and 74 in the postintervention control phase) who presented for initial treatment of occupational body fluid exposures, including 89% (248/280) who sustained punctures and 81% (208/257) who were exposed to blood. Physicians included resident physicians and attending physicians working in the emergency department during the study. INTERVENTIONS: Implementation of a computer charting system that provides real-time information regarding history and recommendations for laboratory testing, treatment, and disposition based on rules derived from clinical guidelines. MAIN OUTCOME MEASURES: Quality of care as determined by essential items documented in the medical record and in aftercare instructions, compliance with testing and treatment guidelines, and total charges and percentage of charges attributable to guideline-endorsed activities. RESULTS: Mean percent documentation of 7 essential items regarding patient history in the medical record increased from 57% during the baseline period to 98% in the intervention phase (42% increase; 95% confidence interval [CI], 34%-49%) and 11 items in aftercare instruction increased from 31 % at baseline to 93% during the intervention phase (62% increase; 95% CI, 51%-74%), but both decreased to baseline when the computer system was removed. Percent compliance with 4 laboratory testing guidelines increased from 63% at baseline to 83% during the intervention phase (20% increase; 95% CI, 9%-31 %) but decreased to 52% when the computer system was removed. Compliance with 5 treatment guidelines increased from 83% at baseline to 96% during the intervention phase (13% increase; 95% CI, 9%-17%) and decreased to 84% following the intervention. Percentage of charges incurred for indicated laboratory tests and treatment increased from 44% at baseline to 81% during the intervention phase (37% increase; 95% CI, 22%-52%) and decreased to 36% following the intervention. Average total per-patient charges were $460, $384, and $373 in each phase, respectively. CONCLUSIONS: Use of a computer-based system for clinical guidelines for management of patients with occupational exposure to body fluids improved documentation, compliance with guidelines, and percentage of charges spent on indicated activities, while decreasing overall charges. The parameters returned to baseline when the computer system was removed.

Total quality management in hospitals: the contributions of commitment, quality councils, teams, budgets, and training to perceived improvement at Veterans Health Administration hospitals.

Studies of total quality management as a means of improving health care quality to date have relied on case studies of individual teams or hospitals. The Total Quality Improvement Registry Project surveyed quality coordinators (n = 36) and quality improvement team leaders (n = 228) to collect both site-level and team-level data on quality improvement in Veterans Health Administration hospitals. Usable responses were received from 100% of quality coordinators and 73.7% (168) of team leaders. Site-level data include hospital structural characteristics and measures of training and commitment, as well as features and activities of the hospital quality councils. Team-level data include size, membership, task, age, activities, and a proxy measure of quality improvement. The authors report on the relations between levels of commitment to total quality management principles, training levels, activities of quality councils, and team formation and success. These data provide support for a model of commitment to quality improvement that involves four realms of influence within the medical centers: (1) management, (2) physician leadership, (3) physician staff and middle management, and (4) nurses and employees. The authors also report on the activities of quality councils and the relation of their activities to commitment and perceived improvement. Using bivariate correlation and multiple regression, the authors found that the age of the quality council, overall facility commitment to total quality management philosophy, and physician commitment are the most critical variables in explaining numbers of teams, training intensity, and total perceived improvement at this sample of medical centers. Specifically, we find that commitment to total quality management philosophy and the number of active teams explains 41% of the observed variation in quality improvement. In future articles, the authors will report details of team activities and the development of teams over time.

Presentation of clinical guidelines via a rule-based expert charting system.

This paper discusses the theoretical basis and cumulative experience with EDECS, the Emergency Department Expert Charting System. This rule-based expert-system introduces clinical guidelines into the flow of patient care while creating the medical record and patient aftercare instructions.

Development of an instrument to measure job satisfaction among dentists.

Because of the rapid changes that have occurred in the dental profession, the Dentist Satisfaction Survey (DSS) was developed to assess both facet and overall job satisfaction among dentists. The DSS was administered to a sample of 558 California general dentists. The results suggest that the DSS is a practical, reliable, and valid measure of dentists' job satisfaction. Internal consistency reliability coefficients for all facet subscales and the overall job satisfaction scale ranged from 0.60-0.92. The DSS discriminated between groups of dentists known to have different levels of job satisfaction; dentists identified by dental society presidents as being very dissatisfied scored significantly differently in the hypothesized direction on the DSS. A high correlation (r = 0.68) was found between judges' independent ratings of dentists' satisfaction based on their open-ended comments and DSS scores. While the majority of respondents were satisfied with most facets of their jobs, substantial variation was found among dentists in their levels of satisfaction.

Professional satisfaction among California general dentists.

Dentists' assessments of their jobs provide insights about issues in the dental profession needing attention. This study assessed professional satisfaction among 558 California general dentists using the 54-item, multi-faceted Dentist Satisfaction Survey (DSS). Although dentists tended to be satisfied with various facets of their jobs and careers, substantial variation in the levels of satisfaction was noted. The most satisfied dentists were older, reported higher incomes, attended more continuing education, and employed more dental auxiliaries than dentists who were the most dissatisfied. Fifty-eight percent of the variation in overall job satisfaction was explained by quality of nonwork life and satisfaction with five facets of the profession: respect received from being a dentist, the actual process of delivering care, income derived from dentistry, relationships with patients, and reduced levels of job-related stress. Dentists were most dissatisfied with the threat of malpractice, level of income, demands of managing the practice, and amount of personal time. Professional organizations should direct programs to address these issues. Dental educators should use these findings to counsel predoctoral students about the realities of dental practice and as an outcome measure of their programs. Moreover, these findings document the perceptions of the practicing profession and support several components of the SELECT recruitment strategy.

Modeling the effect of insurance on health expenditures in the People's Republic of China.

A pilot study was undertaken in two rural counties in Sichuan Province to determine the feasibility of offering health insurance to peasant families. Models of per capital inpatient and outpatient health care expenditures were developed using self-reported utilization from a survey of 880 households, supplemented by cost and utilization data from the providers in the counties. Expenditures at a facility were modeled as a function of level of insurance in three parts: (1) as the product of the probability of any use, (2) the expected number of visits given any use, and (3) the cost per visit at the facility. Output from the model for representative insurance plans is presented.

Predicting hospital accounting costs.

Two alternative methods to Medicare Cost Reports that provide information about hospital costs more promptly but less accurately are investigated. Both employ utilization data from current-year bills. The first attaches costs to utilization data using cost-charge ratios from the previous year's cost report; the second uses charges from current year's bills. The first method is the more accurate of the two, but even using it, only 40% of hospitals had predicted costs within plus or minus 5% of actual costs. The feasibility and cost of obtaining cost reports from a small, fast-track sample of hospitals should be investigated.

Use of meta-analysis to evaluate tolonium chloride in oral cancer screening.

This study used meta-analysis, an analytical technique that uses raw data from previous researchers, to estimate the efficacy of screening for oral cancer with tolonium chloride (TCl). TCl, a vital blue dye, selectively stains acid tissue components such as DNA and RNA and has been used for more than 20 years by surgeons to identify suspected cancerous lesions. It has not been routinely used by dentists to screen either general or high-risk populations. Sensitivity and specificity, as well as positive and negative predictive values for the TCl test, were calculated. Sensitivity of TCl ranged from 97.8% to 93.5%, and specificity ranged from 92.9% to 73.3%. It was determined that if TCl is used to screen high-risk populations, the likelihood of a false negative finding is extremely low, whereas false positive results will be relatively numerous. However, given the high sensitivity of the test, the absolute number of false positive tests will be small. Further analysis is needed to evaluate the economic costs of false positives and false negatives, versus the value of identifying true positives at an early stage.

Predicting hospital accounting costs.

Two alternative methods to Medicare Cost Reports that provide information about hospital costs more promptly but less accurately are investigated. Both employ utilization data from current-year bills. The first attaches costs to utilization data using cost-charge ratios from the previous year's cost report; the second uses charges from current year's bills. The first method is the more accurate of the two, but even using it, only 40 percent of hospitals had predicted costs within plus or minus 5 percent of actual costs. The feasibility and cost of obtaining cost reports from a small, fast-track sample of hospitals should be investigated.

Are the diagnosis-related group case weights compressed?

One problem noted recently with the diagnosis-related group payment system is that the distribution of Medicare case weights and case-mix indexes are compressed; that is, the payment rates for high-cost procedures are too low and those for low-cost procedures are too high. Despite the attention compression has received, there are no direct estimates of its magnitude or importance. Presented in this article are an empirical test for compression and a suggestion for a simple correction to decompress the relative prices.

Estimating treatment and treatment times for special and nonspecial patients in hospital ambulatory dental clinics.

Increased time to treat the special patient is often cited as a barrier to dental care. The purpose of this study was to analyze the separate and combined effects of differences in dental services planned, services actually performed, and differences in treatment time requirements between special and nonspecial patients in a hospital ambulatory clinical setting. Data for this study were obtained from the UCLA evaluators of the RWJ-funded Hospital-Sponsored Ambulatory Dental Services Program (HSADSP). The results show that special patients require more dental treatment than nonspecial patients for advanced dental disease (i.e., periodontics, surgery, and removable prosthetics) and that they receive more of such services. The study also found that special patients should not be viewed as a homogeneous group when evaluating dental needs and required time resources for treatment. Three subgroups of special patients were identified: developmentally disabled, severely compromised, and moderately compromised. The developmentally disabled as compared to the nonspecial patients required significantly more (20 percent) provider time in completing a "representative" treatment plan.