Tutorial on risk-adjusted P-charts.

Several books are available that describe how to construct control charts. Unfortunately none of these books show how to adjust control charts for the severity of the patients' illness. Without such adjustments control charts not only loose face validity but could lead to wrong conclusions. In this article we provide a step-by-step guide regarding how to control for changes in patients' severity of illness across time periods. We illustrate the use of the approach in examining falls in nursing homes. A control chart was constructed to compare observed falls to the residents' risk of fall measured on admission. Analysis led to conclusions radically different from an unadjusted chart.

A survey of 92 quality improvement projects.

BACKGROUND: Studies focusing on the impact of improvement efforts on the organization have yielded mixed results, which has increased interest in comparing the processes of improvement used. Data for a convenience sample of 92 quality improvement (QI) projects in 32 organizations were gathered from interviews and self-reported surveys from 1998 to 2000. A self-administered questionnaire was developed to measure 70 characteristics of improvement projects. RESULTS: Most (80%) of the improvement projects were conducted by hospitals or clinics affiliated with hospitals. The projects took an average of 13 months from the team's first meeting to the end of the pilot study. Project teams met 14 times (approximately once a month) and spent 1.5 hours per meeting. Some projects did not measure the impact, others did not intend to have a specific impact, and still others measured but did not achieve the planned impact. DISCUSSION: Patients and employees may be benefitting from improvement projects, but organizations may not be leveraging these improvements to reduce cost of delivery or increase market share. Considerable variation in the projects' impact raises the question of the need to improve the improvement methods. Generalization from this study should be made with caution, as data were based on a self-selected convenience sample of organizations. Furthermore, respondents did not complete all items, and missing information may affect the conclusions. The data on current improvement practices that are provided in this study can serve as baseline data against which rapid improvement efforts can be judged.

Continuous improvement evaluation: a framework for multisite evaluation studies.

This article proposes the use of the continuous improvement evaluation (CIE), a framework for multisite demonstration or evaluation studies. This framework is designed for studying intervention programs that change during the evaluation. The development of family drug courts is provided as an example. CIE relies on outcome data collected over time and benchmarked across similar cases in comparison sites; thus, this study was designed to collect data on effectiveness of intervention programs at multiple sites and over time. A weight is calculated for similarity of any two cases based on features they share. In statistical process control charts, these weights are used to compare outcomes at the site against the average of similar cases in comparison groups. Once data are benchmarked, program staff meet to discuss process changes that have led to improvements in outcomes. To ensure that intervention programs have access to evaluation reports on demand, information technology is used to collect, clean, and pool data. Computers generate study reports, and evaluators review reports after release to clients. Statistical tools can be used to evaluate changing programs. Traditional evaluators may be concerned about some threats to validity associated with CIE. The article concludes with a discussion of typical threats to validity and how these threats are addressed in the CIE framework.

Tutorial on risk adjusted X-bar charts: applications to measurement of diabetes control.

Two data elements are needed for constructing a risk-adjusted X-bar chart. They are a continuous observed outcome collected over time across a sample of patients, and an expected outcome for each patient. The authors selected blood glucose levels to reflect diabetes control. They then reviewed data showing blood glucose levels of 60 Type 2 diabetes patients in a family practice clinic of five providers. The data covered a period of 21 consecutive months. They present the data relating to two of the providers using this set of data to demonstrate how to create a risk-adjusted X-bar control chart.

Continuous self-improvement: systems thinking in a personal context.

BACKGROUND: Continuous quality improvement (CQI) thinking and tools have broad applicability to improving people's lives--in continuous self-improvement (CSI). Examples include weight loss, weight gain, increasing exercise time, and improving relationship with spouse. In addition, change agents, who support and facilitate organizational efforts, can use CSI to help employees understand steps in CQI. A STEP-BY-STEP APPROACH: Team members should be involved in both the definition of the problem and the search for the solution. How do everyday processes and routines affect the habit that needs to change? What are the precursors of the event? Clients list possible solutions, prioritize them, and pilot test the items selected. One needs to change the daily routines until the desired behavior is accomplished habitually and with little external decision. DISCUSSION: CSI is successful because of its emphasis on habits embedded in personal processes. CSI organizes support from process owners, buddies, and coaches, and encourages regular measurement, multiple small improvement cycles, and public reporting.

Management matters: technology succeeds when management innovates.

It is widely believed that better technology means better business: now that consumers and patients have direct access to computers and information services, the cost of care should decrease and services improve. Yet even with the advent of computer technology and the phenomenal growth of the Internet, costs have increased and quality problems have persisted. Far more important than the technology, then, is how business is conducted and systems are organized. Despite overwhelming evidence that computer services can significantly reduce the costs of care, healthcare organizations have not adopted the changes, or have tried and failed. This article explores what it will take to succeed. We propose a list of necessary nontechnical changes. Patient expectations will change the nature of care; clinicians' roles and training, the gatekeeper profession, healthcare financing and bundling of services, and capital costs will all change. In the end, management innovations make the difference between the success and failure of new technology. Technology is important, but it is not enough. Without new practices, we can buy the technology but will fail to effectively use it. Unless management modifies the very nature of its business, technology's promise to the healthcare industry will go unfulfilled.

Validity of three measures of severity of AIDS for use in health services research studies.

Policy makers and hospital managers often use severity adjustments as a control for patient mix differences when evaluating outcomes of care. Unfortunately, few indices are carefully examined and therefore the evaluations based on these methods of severity adjustment are suspect. This paper examines the accuracy of three indices for measuring the severity of illness of AIDS patients. We examine the Diagnosis-Based Severity Index (DBSI), a modified version of DBSI referred to as MDBSI and the Composite Laboratory Index (CLI) in predicting survival of AIDS patients at one medical centre. We analysed the correlation between indices and months of survival. We also examined the percentage of variance in survival months explained by each index separately and together. Finally, we used survival analysis to examine whether DBSI classifies patients in groups with distinct patterns of survival. Only patients who had died were included in the analysis so that information on the patients' full course of illness was available. Of the 91 patients abstracted, 81 cases had date of AIDS, date of death, and the CLI. These 81 cases were the focus of the analysis. Both CLI and DBSI were predictive of months of survival but were not correlated to each other. Predictions of months of survival were improved if both indices were used together rather than separately. Survival analysis confirmed that patients classified by DBSI had distinctly different survival patterns. Each index measures different aspects of the severity of the patient's condition and when possible both indices should be used together. When laboratory data are not available, e.g. in Medicaid administrative files, the use of DBSI may be reasonable.

Policy issues relevant to evaluation of interactive health communication applications. The Science Panel on Interactive Communication and Health.

This article provides an analysis of policy-related issues associated with the evaluation of interactive health communication (IHC) applications. These include an assessment of the current health and technology policy environment pertinent to public (government, education, public health) and private (medical care providers, purchasers, consumers, IHC developers) IHC stakeholders and discussion of issues likely to merit additional consideration by these stakeholders in the future.

Integrating the Internet into health administration education: a report from AUPHA's Faculty Internet integration task force.

HRSA funded a survey to determine what Internet resources would be most useful to AUPHA membership. This manuscript describes the Internet-intensive survey methodology, reports the survey results, and lists the task force recommendations. The task force used sequential questionnaires posted on the Web to gather both potentially useful Internet resource ideas and membership perceptions of the importance of each idea. Resources recommended by survey participants and the Task Force members emphasized potential improvements to the AUPHA and AUPHA-member Web pages.

Using PDSA (Plan-Do-Study-Act) to establish academic-community partnerships: the Cleveland experience.

The Schools of Medicine and Nursing at Case Western Reserve University and the Program in Health Administration at Cleveland State University have created an interdisciplinary course in continuous improvement that emphasizes learning through experience, accommodates a large number of students, and has created new partnerships with Cleveland area health care organizations. An approach that respects these partners as customers and refines the relationships with serial tests of change (e.g., PDSA) has contributed significantly to this program's success.

Rapid improvement teams.

BACKGROUND: Suggestions, most of which are supported by empirical studies, are provided on how total quality management (TQM) teams can be used to bring about faster organizationwide improvements. SUGGESTIONS: Ideas are offered on how to identify the right problem, have rapid meetings, plan rapidly, collect data rapidly, and make rapid whole-system changes. Suggestions for identifying the right problem include (1) postpone benchmarking when problems are obvious, (2) define the problem in terms of customer experience so as not to blame employees nor embed a solution in the problem statement, (3) communicate with the rest of the organization from the start, (4) state the problem from different perspectives, and (5) break large problems into smaller units. Suggestions for having rapid meetings include (1) choose a nonparticipating facilitator to expedite meetings, (2) meet with each team member before the team meeting, (3) postpone evaluation of ideas, and (4) rethink conclusions of a meeting before acting on them. Suggestions for rapid planning include reducing time spent on flowcharting by focusing on the future, not the present. Suggestions for rapid data collection include (1) sample patients for surveys, (2) rely on numerical estimates by process owners, and (3) plan for rapid data collection. Suggestions for rapid organizationwide implementation include (1) change membership on cross-functional teams, (2) get outside perspectives, (3) use unfolding storyboards, and (4) go beyond self-interest to motivate lasting change in the organization. CONCLUSIONS: Additional empirical investigations of time saved as a consequence of the strategies provided are needed. If organizations solve their problems rapidly, fewer unresolved problems may remain.

Internet course on total quality management.

The Internet provides new opportunities to information-based industries such as universities. If a vast majority of potential customers of our programs have access to the Internet, then it is conceivable that many universities will begin teaching Internet courses and that such courses could, in many cases, replace or at least augment existing face-to-face classroom activities. This paper is an account of one such course at Cleveland State University and the type of policy issues raised by the course at this institution.

Participation in teleconference support groups: application to drug-using pregnant patients.

This paper describes our experience using telephone conference calls to conduct support groups for chemically-dependent women. Forty-seven women agreed to participate in regular, weekly support groups that were conducted by two chemical dependency counselors. Counselors attempted to conduct 59 support groups via a telephone conferencing system. Our data indicated that attendance at these sessions was poor. Seventeen sessions had no participants. No group sessions were attended by 4 or more women, and only 3 sessions (7.1%) included 3 participants. Very few clients made regular use of the support groups. Only 4 clients (8.5%) participated in more than 3 group sessions. Our findings suggest that teleconferencing may not be the most effective method for providing support services to chemically-dependent women.

Computer services for patients. Description of systems and summary of findings.

This article (1) describes a number of computer services provided to patients' homes, (2) summarizes four evaluation studies reported in this supplement, and (3) suggests direction for future research. The studies reviewed in this article suggest that computer services have little impact on patients' health status but a significant impact on use of services. Some computer services (eg, reminders) increase client visits and other services (eg, voice bulletin board) reduce clinic visits. The article suggests that a paradigm for conducting research on impact of home computer services on care of patients is missing and should be developed.

A randomized trial of a telecommunications network for pregnant women who use cocaine.

OBJECTIVES: The authors describe computer services designed to make patients better informed and more motivated to participate in treatment. Patients use these services through their touch-tone telephone; access to a home computer or a modem is not needed. METHODS: The authors tested the impact of these computer services on the management of 179 pregnant women who had used cocaine during pregnancy or 1 month before pregnancy (as reported by the woman). Patients were randomly assigned to control and experimental groups; only the experimental group had access to the computer services. Patients were enrolled during the prenatal period and followed for 6 months after the birth of a live child. Self-reported data on the subjects' participation in drug treatment programs, health status (using the SF-36), and addiction severity (using the addiction severity index) were collected. The computer collected data on the frequency of the use of the service. The dependent variables were participation in formal treatment during the course of the project, and drug and alcohol use at exit interview. Multiple and logistic regressions were used to identify the effects of the intervention after controlling for demographic and baseline variables. RESULTS: Data showed that poor, pregnant, undereducated clients who use drugs and lived in multiple residences could use the services; about one-third of clients used the services more than three times a week. Access to the service did not lead to significant change in patients' health status, drug use, or utilization of services. Use of the services did lead to significant changes in participation in drug treatment: subjects who used the service more than three times a week were 1.5 times more likely to participate in formal drug treatment than subjects who used the service less often. Participation in formal drug treatment, however, was not effective in reducing the drug or the alcohol use. CONCLUSIONS: Almost all patients used the computer services to some extent, but there seems to be a threshold after which the use of the services had a more positive impact.

Educating patients at home. Community Health Rap.

OBJECTIVES: The authors analyzed the impact of home health education by studying the impact of a computer service called Community Health Rap. When patients call this service, the computer records their questions and alerts an expert who records a response. Subsequently, the computer alerts the patient that the question has been addressed. METHODS: Subjects included a group of 82 pregnant women who had used cocaine during or 1 month before pregnancy (as reported by the woman) and a group of residents of zip code areas with the lowest income in Cleveland. From the drug-using pregnant women, we collected data regarding satisfaction with Community Health Rap, usage of Community Health Rap per month, self-reported health status (using the General Health Survey), and the extent of drug use (using the Addiction Severity Index). Trained coders also classified the nature of questions posed to the Community Health Rap by either the pregnant women who abuse drugs or the members of target households. Among the pregnant women who abuse drugs, we compared the differences between those who used the service and those who did not. To control for baseline differences between the two groups, analysis of co-variance was used with exit values as the dependent variables, the baseline values as the co-variates, and participation in the Community Health Rap as the independent variable. RESULTS: Almost half (45%) of poor, undereducated subjects who lived in inner urban areas used the computer service. Content analysis of Community Health Rap messages revealed that subjects had many questions that were of a social nature (regarding sex, relationships, etc), in addition to medical questions. Analysis showed that poor health status, more frequent drug use, lower education, and age did not affect regular use of Community Health Rap service. No health outcomes or utilization of treatment were associated with regular use of Community Health Rap. One exception, however, was that regular users of Community Health Rap reported slower improvement of their pain than those who did not use Community Health Rap. CONCLUSIONS: These data suggest that expansion of information services to households will not leave the poor and the undereducated population groups "behind." They will use computer services, though such services may not have an impact on their health status or cost of care.

Electronic self-help and support groups.

OBJECTIVES: The authors examined the impact of a voice bulletin board on the following: (1) participation in self-help efforts, (2) expression of emotional support, (3) development of solidarity within the group, (4) utilization of health-care services, and (5) health status of subjects. METHODS: Subjects were 53 pregnant women who abused drugs. A quasi-experimental design with matched control group and observations before and after intervention were carried out. Clients in the control group were asked to participate in biweekly face-to-face meetings. Clients in the experimental group participated in the voice bulletin board. Experimental subjects had previous experience with computer services. Subjects reported their level of drug use, health status, and utilization of health services. They also reported on their symptoms, attitudes toward use of physician services, loneliness, willingness to disclose information in groups, and sense of solidarity with their group. The content of the communication among the experimental group was recorded and the utterances were classified as to the type of communication. Exit interviews were done 4 months after baseline interviews were conducted. Clients were paid to complete the baseline and the exit questionnaires, and 94% completed the exit questionnaires. The dependent variables were utilization or health status at exit; the co-variate was utilization or health status at baseline; and the independent variable was the group in which the subject participated. RESULTS: Clients were eight times more likely to participate in the voice bulletin board than in the face-to-face meeting (alpha < 0.01). The majority of the comments left on the bulletin board (54.6%) were for emotional support of each other; no "flaming" or overt disagreements occurred. The more clients participated in the voice bulletin board, the more they felt a sense of solidarity with each other (alpha < 0.001). Members of the experimental group reported significantly lower rates of visiting outpatient clinics than members of the control group (alpha < 0.05). Lower utilization did not lead to poor health status or more drug use: There were no statistically significant differences in the health status and drug use between the experimental and the control groups. CONCLUSIONS: Voice bulletin boards may be an effective method of providing support to mothers who have a history of drug use. Use of these services may lead to lower cost without worsening patients' health.