Waiting for scheduled services in Canada: development of priority-setting scoring systems.

RATIONALE, AIMS AND OBJECTIVES: An Achilles' heel of Canadian Medicare is long waits for elective services. The Western Canada Waiting List (WCWL) project is a collaboration of 19 partner organizations committed to addressing this issue and influencing the way waiting lists are structured and managed. The focus of the WCWL project has been to develop and refine practical tools for prioritizing patients on scheduled waiting lists. METHODS: Scoring tools for priority setting were developed through extensive clinical input and highly iterative exchange by clinical panels constituted in five clinical areas: cataract surgery; general surgery procedures; hip and knee replacement; magnetic resonance imaging (MRI) scanning, and children's mental health. Several stages of empirical work were conducted to formulate and refine criteria and to assess and improve their reliability and validity. To assess the acceptability and usability of the priority-setting tools and to identify issues pertaining to implementation, key personnel in the seven regional health authorities (RHAs) participated in structured interviews. Public opinion focus groups were conducted in the seven western cities. RESULTS: Point-count scoring systems were constructed in each of the clinical areas. Participating clinicians confirmed that the tools offered face validity and that the scoring systems appeared practical for implementation and use in clinical settings. Reliability was strongest for the general surgery and hip and knee criteria, and weakest for the diagnostic MRI criteria. Public opinion focus groups endorsed wholeheartedly the application of point-count priority measures. Regional health authorities were generally supportive, though cautiously optimistic towards implementation. CONCLUSIONS: While the WCWL project has not 'solved' the problem of waiting lists and times, having a standardized, reliable means of assigning priority for services is an important step towards improved management in Canada and elsewhere.

Setting priorities for waiting lists: defining our terms. Steering Committee of the Western Canada Waiting List Project.

In order to develop standardized measures to assess patients' relative priority for services for which there are waiting lists, it is essential that key terms be clearly defined. We propose that severity be defined as the degree or extent of suffering, limits to activities or risk of death; that urgency be defined as severity combined with considerations of the expected benefit and the natural history of the condition; that need be considered equivalent to urgency; and that priority be defined in terms of urgency (or need) with or without consideration of social factors.

Anticoagulation in chronic nonvalvular atrial fibrillation: a critical appraisal and meta-analysis.

OBJECTIVE: To assess the outcomes associated with warfarin treatment of patients with chronic nonvalvular atrial fibrillation (CNVAF) for prevention of primary stroke. DATA SOURCES: MEDLINE was searched for literature published from 1987 to August 1996. Search terms used were 'atrial fibrillation' and 'anticoagulants'. STUDY SELECTION: Five published randomized controlled trials concerning primary stroke prevention. DATA EXTRACTION: Data were pooled across trials to estimate the magnitude of the effect for each of nine reported end-points. The annual probability of occurrence of each outcome was calculated, including standard errors and Mantel-Haenszel significance tests with 95% CIs. DATA SYNTHESIS: In view of the lack of blinded assessment and documented low inter-rater reliability of soft neurological end-points, the analysis was limited to the relatively objective end-points of major strokes, fatal strokes, major bleeding and fatal bleeding. Warfarin did not reduce the incidence of fatal strokes to a statistically significant extent, nor was incidence of fatal bleeding increased significantly. Warfarin reduced the absolute annual incidence of major strokes in patients with CNVAF by 0.89%, while at the same time it increased the absolute annual risk of major bleeding incidents by 1.8%. Though small, these differences were statistically significant. CONCLUSIONS: On balance, the margin between expected benefit and harm for warfarin prophylaxis in patients with CNVAF is uncomfortably thin. These results and conclusions differ from those of a previously published meta-analysis of these same studies.

The New Zealand priority criteria project. Part 1: Overview.

New Zealand restructured its health system in 1992 with the aim of achieving greater levels of assessment and accountability in the publicly funded health sector. A committee was established specifically to advise the minister of health on the kinds, and relative priorities, of health services that should be publicly funded. One of its projects has been to develop standardised sets of criteria to assess the extent of benefit expected from elective surgical procedures. These have been developed with the help of professional advisory groups using a modified Delphi technique to reach consensus. So far the committee has developed criteria for cataract surgery, coronary artery bypass grafting, hip and knee replacement, cholecystectomy, and tympanostomy tubes for otitis media with effusion. These criteria incorporate both clinical and social factors. Use of priority criteria to ensure consistency and transparency regarding patients' priority for surgery is required for access to a dedicated NZ$130m (pounds 57m; US$90m) pool of money, created to help eliminate surgical waiting lists and move to booking systems. The criteria will also be used in surgical outcome studies, currently in the planning phase.

The New Zealand priority criteria project. Part 2: Coronary artery bypass graft surgery.

Priority criteria developed during a national project were used to conduct an audit of all 662 patients on waiting lists for coronary artery bypass surgery in New Zealand during spring 1996. Based on the observed distribution of priority scores, the cost of providing surgery to all patients down to various levels of priority was estimated. Descriptions incorporating life expectancy and quality of life implications of surgery were developed of the kinds of patients who would or would not receive surgery at each of several possible funding levels. Cardiologists and cardiac surgeons agreed that a threshold of 25 points was a reasonable clinical goal but to work with a threshold of 35, which can be sustained with current levels of funding. All agree that the gap between these clinically preferred and currently afforded thresholds is a subject for wider societal dialogue and decision. The ability to measure the size of the gap between clinical desirability and financial sustainability provides a new transparency to the problem of healthcare resource allocation.

Kinds of patients.

The basic goal of health outcomes research is to identify the kinds of patients who do (or do not) benefit substantially from specified medical or surgical treatments and procedures. Similarly, clinicians must determine whether particular patients are the kinds of patients who do (or do not) benefit from specified interventions. Such a kinds-based approach to clinical practice is often resisted, however, when physicians are asked to standardize their practices based on the results of health outcome data. In such settings, clinicians often assert that "every patient is unique". The present paper explores the coherence of this claim. In particular, I examine the applicability of the philosophical notion of natural kinds to a kinds-based approach to clinical research and practice. I conclude that the claim of patient uniqueness is misguided. Two key difficulties with a kinds-based approach are examined: the problems associated with (1) assigning single-case probabilities and (2) stereotyping and discrimination.

Rating the quality of evidence for clinical practice guidelines.

This article describes the system for rating the quality of medical evidence developed and used during creation of the Agency for Health Care Policy and Research-sponsored heart failure guideline. Previous approaches to rating evidence were not designed for use in the setting of clinical practice guidelines. The present system is based on the tenet that flaws in research design are serious to the extent they threaten the validity of the results of studies. A taxonomy of major and minor flaws based on that tenet was developed for randomized controlled trials and for cohort and medical registry studies. The use of the system is described in the context of two difficult clinical issues considered by the Panel: the role of coronary artery revascularization and the use of metoprolol.

Phase II of the AHCPR-sponsored heart failure guideline: translating practice recommendations into review criteria.

BACKGROUND: In 1992, under the sponsorship of the U.S. Agency for Health Care Policy and Research, RAND assembled an expert panel to develop the Heart Failure Clinical Practice Guideline. Phase II of the effort was intended to identify which of the guideline's recommendations the panel felt were suitable for use in retrospective utilization review and quality assessment programs and to develop review criteria, performance measures, and standards of quality for use in monitoring compliance with those recommendations. SELECTION OF RECOMMENDATIONS: Selecting guideline recommendations for translation into review criteria and ultimately into standards of care was a multistep process comprising (1) identification of 34 recommendations from the guideline, (2) rating them on the basis of importance to quality of care and feasibility of monitoring, (3) review by a subcommittee and the full guideline panel, (4) translation into review criteria, and (5) further review and input by panelists and peer and pilot reviewers. Finally, standards of care (the minimum proportion of cases expected to be in accordance with guideline recommendations) were determined to be 90%-95% for six of the final criteria and 75%-80% for the other two. CONCLUSION: Despite some reservations, physicians and other health care professionals agreed to be held accountable for following a core set of guideline recommendations for the treatment of heart failure. Substantial progress was made in identifying recommendations that panelists and reviewers were willing to endorse in utilization review activities, including adoption of improved documentation standards. The review criteria's major impact may be the knowledge that the criteria are in place and that care is being monitored based on those standards.

Development of the AHCPR-sponsored heart failure guideline: methodologic and procedural issues.

BACKGROUND: RAND, a nonprofit research and policy organization, served as contractor for the Agency for Health Care Policy and Research (AHCPR)-sponsored guideline on the management of patients with heart failure due to left-ventricular systolic dysfunction. PANEL: At meetings of the 16-member panel, discussions concerning practice recommendations were held until a consensus was reached. A draft algorithm was the key tool, serving as a starting point for panel discussion. The algorithm was revised after almost every panel meeting. KEY METHODS AND PROCEDURAL ISSUES: Early decisions included how best to (1) focus the guideline and literature review, (2) rate the strength of evidence underlying practice recommendations, and (3) determine the relationship between strength of evidence and strength of recommendation. In the absence of data, or when panelists did not share a common opinion, the panel attempted to achieve a consensus. An example of this process--when and how patients should be evaluated for possible coronary artery revascularization--is discussed at length. Unlike previous AHCPR guideline panels, this panel solicited opinions of national experts early and often during guideline development. This process helped the panelists arrive at conclusions on certain controversial issues. CONCLUSIONS: The guideline development process was complex and painstaking. The panelists and project staff believe that frequent peer review helped produce a guideline that can be widely accepted across clinical and geographic lines.

The problem of discrimination in health care priority setting.

Increasingly stringent fiscal restrictions on the scope of medical services available to patients have resulted in calls for explicit health care priority setting. Several commentators have called for the application of decision-analytic principles to such efforts, which would assign services priority based on the extent to which they produce preferred health outcomes. The Oregon Medicaid exercise is an example of such a process. An important challenge to these utilitarian efforts is the need to avoid discrimination against people with medical disabilities. Both of the key elements entailed by decision-analytic approaches to priority setting--estimation of outcomes and assignment of values to those outcomes--are vulnerable to charges of discrimination, primarily because both the medical outcomes expected in disabled individuals and the values they place on those outcomes may differ from the general public. Priority-setting efforts must proceed carefully to avoid the appearance (and reality) of discrimination.

An annotated algorithm approach to clinical guideline development.

The Urinary Incontinence in Adults Guideline Panel facilitated the ready elucidation of its guideline's management recommendations through the use of an annotated algorithm approach. The algorithms created as part of this guideline differ from previous algorithms in two ways: (1) they employ systematic annotation to link explicitly the algorithms' recommendations to the literature, and (2) they contain patient counseling and decision nodes to depict the major preference-dependent decision or branch points in the algorithm. We believe that these two innovations can help ensure the clinical validity of guidelines' algorithms while preserving appropriate clinical flexibility and respecting patient preferences.

Improving task comprehension in the measurement of health state preferences. A trial of informational cartoon figures and a paired-comparison task.

Information concerning people's relative preferences for health care outcomes is usually obtained using questionnaires which ask subjects to imagine health states of various kinds. When using illness- or treatment-specific states, elaborate descriptions of patients' quality of life may adequately convey to the rater a sense of the real situation. Such descriptions are not possible with generic questionnaires, which frame outcomes in general terms, such as pain, limits on activities, etc. This study evaluated two methods for facilitating task comprehension with generic preference-assessment instruments: (1) use of informational figures that provide visual representations of the described health state, and (2) measurement of preferences by means of a paired-comparison task. The use of figures did not change rating variance or the number of counter-intuitive ratings, but did improve one-week test-retest reliability. Paired comparisons had reliability comparable to the direct rating tasks and reduced the number of counter-intuitive ratings, although not to a statistically significantly extent.

Cross-validation performance of mortality prediction models.

Mortality prediction models hold substantial promise as tools for patient management, quality assessment, and, perhaps, health care resource allocation planning. Yet relatively little is known about the predictive validity of these models. We report here a comparison of the cross-validation performance of seven statistical models of patient mortality: (1) ordinary-least-squares (OLS) regression predicting 0/1 death status six months after admission; (2) logistic regression; (3) Cox regression; (4-6) three unit-weight models derived from the logistic regression, and (7) a recursive partitioning classification technique (CART). We calculated the following performance statistics for each model in both a learning and test sample of patients, all of whom were drawn from a nationally representative sample of 2558 Medicare patients with acute myocardial infarction: overall accuracy in predicting six-month mortality, sensitivity and specificity rates, positive and negative predictive values, and per cent improvement in accuracy rates and error rates over model-free predictions (i.e., predictions that make no use of available independent variables). We developed ROC curves based on logistic regression, the best unit-weight model, the single best predictor variable, and a series of CART models generated by varying the misclassification cost specifications. In our sample, the models reduced model-free error rates at the patient level by 8-22 per cent in the test sample. We found that the performance of the logistic regression models was marginally superior to that of other models. The areas under the ROC curves for the best models ranged from 0.61 to 0.63. Overall predictive accuracy for the best models may be adequate to support activities such as quality assessment that involve aggregating over large groups of patients, but the extent to which these models may be appropriately applied to patient-level resource allocation planning is less clear.

Emerging parallels in the American health care and legal-judicial systems.

The structure and principal decision-making processes of the American health care system have, in recent years, evolved to closely resemble those of the legal-judicial system. This transformation reflects important common values that underlie both systems, including the values of life and liberty. This Article analyzes quasi-legal features of the health care system and draws conclusions about how those features might be used to address the problem of health care rationing. It concludes that coverage rules, if properly developed, can provide the sort of objective framework necessary to evaluate claims of health care needs. This Article also demonstrates that by defining legitimate health care needs, society can thereby potentially eliminate or forestall the need to ration necessary care. This can be achieved by using carefully developed coverage rules, rather than the informal rules currently in place, in conjunction with already existing due process methods for interpreting and implementing those rules.

The health care resource allocation debate. Defining our terms.

The problem of health care distribution in the United States demands immediate action. Many different solutions have been proposed to slow rising health care costs and to improve access to care for the poor and uninsured. Debate among proponents of these various proposals might be advanced if a common language were adopted with regard to certain key terms instead of the various meanings currently assigned to these terms. For this reason, we propose and defend the following three definitions: (1) rationing is the societal toleration of inequitable access to health services acknowledged to be necessary by reference to necessary-care guidelines; (2) health care needs are desires for services that have been reasonably well demonstrated to provide significant net benefit for patients with specified clinical conditions; and (3) basic benefit plans are insurance packages that provide for all and only acknowledged health care needs, again by reference to appropriate clinical guidelines.

Multitrait-multimethod analysis of health-related quality-of-life measures.

Interest in health-related quality of life (HRQOL) is burgeoning and there has been a proliferation of self-report measures of HRQOL. However, only two instruments available for measuring HRQOL have been calibrated using empirically derived preferences; both are long and complex. This study tested a brief survey designed to concurrently assess HRQOL and preferences for different HRQOL states. Multitrait-multimethod (MTMM) analysis was used to evaluate the construct validity of the survey in a convenience sample of 116 persons. Two methods were used to assess six aspects of HRQOL: general health perceptions, meaningful activities, outlook on life, physical suffering, self-care activities, and social relationships. HRQOL preferences were assessed using two methods similar to those used for self-reports, as well as one additional method. The construct validity of self-reported HRQOL was supported. On the other hand, substantial method variance and little valid trait variance was observed for the HRQOL preferences. Results are discussed in terms of their implications for evaluating and measuring HRQOL and related preferences.