Publishing web-based guidelines using interactive decision models.

Commonly used methods for guideline development and dissemination do not enable developers to tailor guidelines systematically to specific patient populations and update guidelines easily. We developed a web-based system, ALCHEMIST, that uses decision models and automatically creates evidence-based guidelines that can be disseminated, tailored and updated over the web. Our objective was to demonstrate the use of this system with clinical scenarios that provide challenges for guideline development. We used the ALCHEMIST system to develop guidelines for three clinical scenarios: (1) Chlamydia screening for adolescent women, (2) antiarrhythmic therapy for the prevention of sudden cardiac death; and (3) genetic testing for the BRCA breast-cancer mutation. ALCHEMIST uses information extracted directly from the decision model, combined with the additional information from the author of the decision model, to generate global guidelines. ALCHEMIST generated electronic web-based guidelines for each of the three scenarios. Using ALCHEMIST, we demonstrate that tailoring a guideline for a population at high-risk for Chlamydia changes the recommended policy for control of Chlamydia from contact tracing of reported cases to a population-based screening programme. We used ALCHEMIST to incorporate new evidence about the effectiveness of implantable cardioverter defibrillators (ICD) and demonstrate that the cost-effectiveness of use of ICDs improves from $74 400 per quality-adjusted life year (QALY) gained to $34 500 per QALY gained. Finally, we demonstrate how a clinician could use ALCHEMIST to incorporate a woman's utilities for relevant health states and thereby develop patient-specific recommendations for BRCA testing; the patient-specific recommendation improved quality-adjusted life expectancy by 37 days. The ALCHEMIST system enables guideline developers to publish both a guideline and an interactive decision model on the web. This web-based tool enables guideline developers to tailor guidelines systematically, to update guidelines easily, and to make the underlying evidence and analysis transparent for users.

Choice-matching preference reversals in health outcome assessments.

BACKGROUND: Health outcome utility assessments generally assume procedural invariance. Preference reversals violating procedural invariance occur in economic scenarios when the assessment process shifts from a choice to a fill-in-the-blank task. PURPOSE: To determine if similar reversals occur in utility assessments. METHODS: One hundred thirty-six volunteer subjects completed 6 preference assessments of 4 personal health scenarios. Patients responded to otherwise identical tasks using either choice or fill-in-the-blank processes in a randomized crossover design. The authors determined the percentage of subjects preferring, or inferred to prefer, a given choice. RESULTS: Preference reversals occurred in all assessment scenarios. CONCLUSIONS: These preference reversals are a potential source of confusion for utility assessment and informed consent. They could be manipulated to achieve ends other than the best interest of patients. Anchoring or the prominence hypothesis may explain these findings.

Patient preferences for adjuvant interferon alfa-2b treatment.

PURPOSE: Although trials of adjuvant interferon alfa-2b (IFN alpha-2b) in high-risk melanoma patients suggest improvement in disease-free survival, it is unclear whether treatment offers improvement in overall survival. Widespread use of adjuvant IFN alpha-2b has been tempered by its significant toxicity. To quantify the trade-offs between IFN alpha-2b toxicity and survival, we assessed patient utilities for health states associated with IFN therapy. Utilities are measures of preference for a particular health state on a scale of 0 (death) to 1 (perfect health). PATIENTS AND METHODS: We assessed utilities for health states associated with adjuvant IFN among 107 low-risk melanoma patients using the standard gamble technique. Health states described four IFN alpha-2b toxicity scenarios and the following three posttreatment outcomes: disease-free health and melanoma recurrence (with or without IFN alpha-2b) leading to cancer death. We also asked patients the improvement in 5-year disease-free survival they would require to tolerate IFN. RESULTS: Utilities for melanoma recurrence with or without IFN alpha-2b were significantly lower than utilities for all IFN alpha-2b toxicities but were not significantly different from each other. At least half of the patients were willing to tolerate mild-moderate and severe IFN alpha-2b toxicity for 4% and 10% improvements, respectively, in 5-year disease-free survival. CONCLUSION: On average, patients rate quality of life with melanoma recurrence much lower than even severe IFN alpha-2b toxicity. These results suggest that recurrence-free survival is highly valued by patients. The utilities measured in our study can be applied directly to quality-of-life determinations in clinical trials of adjuvant IFN alpha-2b to measure the net benefit of therapy.

Utility-based estimates of the relative morbidity of visual impairment and angina.

PURPOSE: To quantify and compare the reduction in quality of life due to visual impairment and angina using patient preferences (utilities). METHODS: Using a standard time tradeoff method, we obtained utilities for current vision, monocular and binocular blindness, current angina, and moderate angina in 60 patients with both vision problems and angina pectoris who sought care at the National Eye Institute (NEI), National Naval Medical Center, or Barnes-Jewish Hospital. Patients were characterized clinically based on visual acuity and the Duke Activity Status Index (DASI). Patients also completed a seven-item version of the NEI Visual Functioning Questionnaire and the SF-36 Health Survey Questionnaire. RESULTS: Patients had a median visual acuity of 20/100 in the worst eye, 20/40 in the better eye, and a median DASI of 24.2 (0 = severe functional limitations due to anginal symptoms, 58.2 = no limitations). There was substantial variation in utilities among patients. The average utility for current vision (relative to ideal vision [= 1.0] and death [= 0.0]) was 0.82; the average utility for current angina (relative to no angina symptoms [= 1.0] and death [ = 0.0]) was 0.89. Among 26 patients with both visual impairment and recent anginal symptoms, the decrement in utility (on a scale ranging from ideal health [= 1.0] to death [= 0.0]) imposed by current visual impairment was greater than that imposed by current angina symptoms (0.146 versus 0.072, p=0.08, Wilcoxon signed rank test). The decrement in utility associated with binocular blindness was greater than the decrement associated with the symptoms of moderate angina (0.477 versus 0.039, p<0.0001). CONCLUSIONS: Clinical status is not a surrogate for patient preferences regarding vision impairment or angina. There is substantial variation in utilities within the study population for both experienced and theoretical impairment states which is not explained by variations in clinical status. Some states of visual impairment may pose a greater quality of life burden than anginal symptoms. Because patient preferences for vision vary greatly, individual assessment is warranted for consideration in therapeutic decision making.

Procedure-related miscarriages and Down syndrome-affected births: implications for prenatal testing based on women's preferences.

OBJECTIVE: To determine how pregnant women of varying ages, races, ethnicities, and socioeconomic backgrounds value procedure-related miscarriage and Down-syndrome-affected birth. METHODS: We studied cross-sectionally 534 sociodemographically diverse pregnant women who sought care at obstetric clinics and practices throughout the San Francisco Bay area. Preferences for procedure-related miscarriage and the birth of an infant affected by Down syndrome were assessed using the time trade-off and standard gamble metrics. Because current guidelines assume that procedure-related miscarriage and Down syndrome-affected birth are valued equally, we calculated the difference in preference scores for those two outcomes. We also collected detailed information on demographics, attitudes, and beliefs. RESULTS: On average, procedure-related miscarriage was preferable to Down syndrome-affected birth, as evidenced by positive differences in preference scores for them (time trade-off difference: mean = 0.09, median = 0.06; standard gamble difference: mean = 0.11, median = 0.02; P <.001 for both, one-sample sign test). There was substantial subject-to-subject variation in preferences that correlated strongly with attitudes about miscarriage, Down syndrome, and diagnostic testing. CONCLUSION: Pregnant women tend to find the prospect of a Down syndrome-affected birth more burdensome than a procedure-related miscarriage, calling into question the equal risk threshold for prenatal diagnosis. Individual preferences for those outcomes varied profoundly. Current guidelines do not appropriately consider individual preferences in lower-risk women, and the process for developing prenatal testing guidelines should be reconsidered to better reflect individual values.

Design and pilot evaluation of a system to develop computer-based site-specific practice guidelines from decision models.

BACKGROUND: Local tailoring of clinical practice guidelines (CPGs) requires experts in medicine and evidence synthesis unavailable in many practice settings. The authors' computer-based system enables developers and users to create, disseminate, and tailor CPGs, using normative decision models (DMs). METHODS: ALCHEMIST, a web-based system, analyzes a DM, creates a CPG in the form of an annotated algorithm, and displays for the guideline user the optimal strategy. ALCHEMIST'S interface enables remote users to tailor the guideline by changing underlying input variables and observing the new annotated algorithm that is developed automatically. In a pilot evaluation of the system, a DM was used to evaluate strategies for staging non-small-cell lung cancer. Subjects (n = 15) compared the automatically created CPG with published guidelines for this staging and critiqued both using a previously developed instrument to rate the CPGs' usability, accountability, and accuracy on a scale of 0 (worst) to 2 (best), with higher scores reflecting higher quality. RESULTS: The mean overall score for the ALCHEMIST CPG was 1.502, compared with the published-CPG score of 0.987 (p = 0.002). The ALCHEMIST CPG scores for usability, accountability, and accuracy were 1.683, 1.393, and 1.430, respectively; the published CPG scores were 1.192, 0.941, and 0.830 (each comparison p < 0.05). On a scale of 1 (worst) to 5 (best), users' mean ratings of ALCHEMIST'S ease of use, usefulness of content, and presentation format were 4.76, 3.98, and 4.64, respectively. CONCLUSIONS: The results demonstrate the feasibility of a web-based system that automatically analyzes a DM and creates a CPG as an annotated algorithm, enabling remote users to develop site-specific CPGs. In the pilot evaluation, the ALCHEMIST guidelines met established criteria for quality and compared favorably with national CPGs. The high usability and usefulness ratings suggest that such systems can be a good tool for guideline development.

Parents' preferences for outcomes associated with childhood vaccinations.

BACKGROUND: The number of shots in the childhood immunization schedule has been increasing and is likely to continue to increase in the coming years. Consideration of the psychologic costs of multiple injections, adverse events and vaccine-preventable disease is therefore growing in importance. METHODS: We assessed parent preferences, using both the time tradeoff (i.e. amount of parent time willing to trade) and willingness-to-pay (i.e. dollars willing to pay) metrics, for possible outcomes of vaccination among 206 parents of infants receiving care at Kaiser, Northern California Region. We also explored the relationship between preferences and subject characteristics. RESULTS: In general the amount of time subjects were willing to give up and the quantity of money they were willing to spend to avoid an outcome increased with the severity of the outcome. Preferences for our six main outcomes of interest all differed from one another (P < 0.0001, Tukey's multiple comparisons procedure). Rank correlation coefficients between time tradeoff and willingness-to-pay values for the six main outcomes ranged from 0.42 to 0.52 (all P < 0.004). Subject characteristics, including education, income, race/ethnicity and the child's birth order, did not explain the variation in parent preferences. CONCLUSIONS: In general subjects were willing to give up more money or time to avoid less desired outcomes. They were willing to give up only very small amounts of their own life expectancy or money to avoid minor, temporary outcomes (e.g. moderate fussiness, fever and pain) whereas they were willing to forego substantial lengths of their life or amounts of money to avoid a major, permanent outcome (i.e. permanent disability). Nonetheless much variation surfaced in the amount of time (or money) subjects were willing to trade to avoid outcomes. If this variation represents true differences in preferences, guideline developers must consider the role of individual parent preferences in decisions concerning vaccination.

Cost-effectiveness analysis in clinical practice: the case of heart failure.

Heart failure is the leading cause of hospitalization in adults older than 65 years, and it is currently the most costly cardiovascular disorder in the United States, with estimated annual expenditures in excess of $20 billion. Recent studies have shown that selected pharmacological agents, behavioral interventions, and surgical therapies are associated with improved clinical outcomes in patients with heart failure, but the cost implications of these diverse treatment modalities are not widely appreciated. In this review, a brief outline of cost-effectiveness analysis is provided, and current data on the cost-effectiveness of specific approaches to managing heart failure are discussed. Available evidence indicates that angiotensin converting enzyme inhibitors, other vasodilators, digoxin, carvedilol, multidisciplinary heart failure management teams, and heart transplantation are all cost-effective approaches to treating heart failure; moreover, some of these interventions may result in net cost savings.

Who should be offered prenatal diagnosis? The 35-year-old question.

Prenatal diagnosis of chromosomal disorders is generally offered to women who will be 35 years or older at the time of delivery or who have been determined via serum screening to be at risk similar to that of a woman older than 35 years. This age threshold was based on 4 major rationales that reflect considerations of resources and effectiveness. In this paper, we explore the current screening recommendations and consider new information that calls the 35-years threshold into question. We conclude that guidelines regarding use of prenatal diagnosis account for the preferences of the individual patient as well as for individual risk.

Patient preferences for location of care: implications for regionalization.

BACKGROUND: Regionalization of high-risk surgical procedures to selected high-volume centers has been proposed as a way to reduce operative mortality. For patients, however, travel to regional centers may be undesirable despite the expected mortality benefit. OBJECTIVE: To determine the strength of patient preferences for local care. DESIGN: Using a scenario of potentially resectable pancreatic cancer and a modification of the standard gamble utility assessment technique, we determined the level of additional operative mortality risk patients would accept to undergo surgery at a local rather than at a distant regional hospital in which operative mortality was assumed to be 3%. We used multiple logistic regression to identify predictors of willingness to accept additional risk. SUBJECTS: One hundred consecutive patients (95% male, median age 65) awaiting elective surgery at the Veterans Affairs Medical Center in White River Jct., VT. MAIN OUTCOME MEASURE: Additional operative mortality risk patients would accept to keep care local. RESULTS: All patients preferred local surgery if the operative mortality risk at the local hospital were the same as the regional hospital (3%). If local operative mortality risk were 6%, which is twice the regional risk, 45 of 100 patients would still prefer local surgery. If local risk were 12%, 23 of 100 patients would prefer local surgery. If local risk were 18%, 18 of 100 patients would prefer local surgery. Further increases in local risk did not result in large changes in the proportion of patients preferring local care. CONCLUSIONS: Many patients prefer to undergo surgery locally even when travel to a regional center would result in lower operative mortality risk. Therefore, policy makers should consider patient preferences when assessing the expected value of regionalizing major surgery.

Use of the polymerase chain reaction in the diagnosis of herpes simplex encephalitis: a decision analysis model.

PURPOSE: To evaluate the utility of an assay based on a polymerase chain reaction (PCR) of cerebrospinal fluid in the management of patients with suspected herpes simplex encephalitis. METHODS: A decision model was constructed and used to compare a PCR-based approach with empiric therapy. Inputs required by the model included the sensitivity (96%) and specificity (99%) of PCR (derived from review of the literature), the prevalence of herpes simplex encephalitis (5%, based on the actual prevalence at Barnes Hospital among patients treated empirically with acyclovir), the outcomes for patients with and without herpes simplex encephalitis (derived from clinical studies of the Collaborative Antiviral Study Group and the actual experience at Barnes Hospital), and the average duration of empiric acyclovir therapy for patients with possible herpes simplex encephalitis (5.3 days based on actual experience at Barnes Hospital). RESULTS: Using these input values, the decision model predicted better outcomes with empiric therapy. However, low rates of inappropriate discontinuation of empiric therapy in patients with herpes simplex encephalitis or improved diagnosis and outcome resulting from a negative PCR assay result in patients without herpes simplex encephalitis led to better outcomes with the PCR-based approach. The PCR-based approach was associated with 9.2 fewer doses of acyclovir per patient. CONCLUSION: Based on the decision model using conservative assumptions, a PCR-based approach can yield better outcomes and reduced acyclovir use compared with empiric therapy.

Assessment of patient preferences among men with prostate cancer.

PURPOSE: We developed a self-administered paper based instrument to assess patient preferences quantified as utilities for common outcomes associated with the management of prostate cancer. MATERIALS AND METHODS: A total of 50 patients was invited to test a self-administered paper based instrument designed to assess preferences for health outcomes associated with the management of localized prostate cancer. The 50 patients were selected from a group of 625 randomly identified men with prostate cancer who responded to a survey instrument designed to assess health related quality of life. The 50 patients selected for this pilot project were chosen because of the wide range of responses to the quality of life survey. Patient utilities were assessed for the 5 health states of overall quality of life, problems related to prostate cancer, and problems related to urinary, bowel and sexual dysfunction. RESULTS: Patients were able to complete the assigned tasks. The self-administered instrument had high test-retest reliability. In addition results obtained from this instrument showed a correlation with results obtained from assessments using other instruments, including an analog scale, a computer based system known as U-Titer, a quality of life survey and the Health Utility Index:3. CONCLUSIONS: A self-administered paper based instrument can be used to assess patient utilities for health states associated with prostate cancer management. Results from the instrument tested appear to be reliable and valid, and are comparable to those obtained from other assessment techniques. A self-administered paper based instrument has distinct advantages when conducting large survey studies because it can be incorporated at relatively low cost.

A normative analytic framework for development of practice guidelines for specific clinical populations.

BACKGROUND: A central problem in practice guideline development is how to develop guidelines that appropriately account for variations in clinical populations and practice settings. Despite recognition of this problem, there is no formal mechanism for assessing what the need is for flexibility in guidelines, or for deciding how to incorporate such flexibility into recommendations. OBJECTIVE: This research sought to provide a formal basis to determine when clinical circumstances vary sufficiently that guideline recommendations should differ, how recommendations should be tailored for a specific clinical setting, and whether the benefit associated with such site-specific guidelines justifies the expense of their development. RESULTS: The authors describe an approach for estimating the maximum health benefit that developers can obtain by eliminating uncertainty about differences in the patient populations and practice settings in which a guideline will be used. This estimate, the expected value of customization, provides a mechanism to evaluate the cost-effectiveness of the development of site-specific guidelines that account explicitly for variation in clinical circumstances. Application of this method to the development of screening guidelines for human immunodeficiency virus (HIV) infection indicates that the development of site-specific guidelines potentially is cost-effective. Site-specific guidelines either improve, or leave unchanged, the efficiency of HIV screening; whether they increase or decrease total expenditures and health benefits depends on the choice of a cost-effectiveness threshold, and the clinical problem. CONCLUSIONS: Development of guideline recommendations based on decision models provides a normative approach for evaluating the need for and the cost-effectiveness of site-specific guidelines that have been tailored to specific practice settings. Such site-specific guidelines can improve substantially the expected health benefit and the economic efficiency of practice guidelines.

Representation and analysis of medical decision problems with influence diagrams.

Influence diagrams are a powerful graphic representation for decision models, complementary to decision trees. Influence diagrams and decision trees are different graphic representations for the same underlying mathematical model and operations. This article describes the elements of an influence diagram, and shows several familiar decision problems represented as decision trees and as influence diagrams. The authors also contrast the information highlighted in each graphic representation, demonstrate how to calculate the expected utilities of decision alternatives modeled with an influence diagram, provide an overview of the conceptual basis of the solution algorithms that have been developed for influence diagrams, discuss the strengths and limitations of influence diagrams relative to decision trees, and describe the mathematical operations that are used to evaluate both decision trees and influence diagrams. They use clinical examples to illustrate the mathematical operations of the influence-diagram-evaluation algorithm; these operations are arc reversal, chance node removal by averaging, and decision node removal by policy determination. Influence diagrams may be helpful when problems have a high degree of conditional independence, when large models are needed, when communication of the probabilistic relationships is important, or when the analysis requires extensive Bayesian updating. The choice of graphic representation should be governed by convenience, and will depend on the problem being analyzed, on the experience of the analyst, and on the background of the consumers of the analysis.

Use of influence diagrams to structure medical decisions.

Influence diagrams are compact representations of decision problems that are mathematically equivalent to decision trees. The authors present five important principles for structuring a decision as an influence diagram: 1) start at the value node and work back to the decision nodes; 2) draw the arcs in the direction that makes the probabilities easiest to assess; 3) use informational arcs to specify which events will have been observed at the time each decision is made; 4) ensure that missing arcs reflect intentional assertions about conditional independence and the timing of observations; and 5) ensure that there are no cycles in the influence diagram. They then build an influence diagram for the problem of staging non-small-cell lung cancer as an illustration. Influence diagrams offer several strengths for structuring medical decisions. They represent graphically and compactly the probabilistic relationships between parameters in the model. Influence diagrams also allow the model to be structured in a fashion that eases the necessary probability assessments, regardless of whether the assessments are based on available evidence or on expert judgment. Influence diagrams provide an important complement to decision trees, especially for representing probabilistic relationships among variables in a decision model.

Determining transition probabilities from mortality rates and autopsy findings.

The Markov process is a useful tool for modeling the natural history of disease, which is becoming increasingly important as new diagnostic tests increase the detectability of early-stage disease. The accuracy of a Markov model, however, depends on the accuracy of the estimates for the transition probabilities between different stages of disease. Because these estimates are usually based on "expert opinion" or small cohort studies, they are subject to imprecision and bias. The authors describe an alternative method of estimating transition probabilities from the stage distribution of disease observed at the time of death and age-specific mortality rates from other causes. In addition, they prove that the transition probabilities are unique given certain assumptions about how they change with age. Finally, they illustrate the method using population-based data for prostate cancer.