A psychometric analysis of the measurement level of the rating scale, time trade-off, and standard gamble.

A fundamental assumption of utility-based analyses is that patient utilities for health states can be measured on an equal-interval scale. This assumption, however, has not been widely examined. The objective of this study was to assess whether the rating scale (RS), standard gamble (SG), and time trade-off (TTO) utility elicitation methods function as equal-interval level scales. We wrote descriptions of eight prostate-cancer-related health states. In interviews with patients who had newly diagnosed, advanced prostate cancer, utilities for the health states were elicited using the RS, SG, and TTO methods. At the time of the study, 77 initial and 73 follow-up interviews had. been conducted with a consecutive sample of 77 participants. Using a Rasch model, the boundaries (Thurstone Thresholds) between four equal score sub-ranges of the raw utilities were mapped onto an equal-interval logit scale. The distance between adjacent thresholds in logit units was calculated to determine whether the raw utilities were equal-interval. None of the utility scales functioned as interval-level scales in our sample. Therefore, since interval-level estimates are assumed in utility-based analyses, doubt is raised regarding the validity of findings from previous analyses based on these scales. Our findings need to be replicated in other contexts, and the practical impact of non-interval measurement on utility-based analyses should be explored. If cost-effectiveness analyses are not found to be robust to violations of the assumption that utilities are interval, serious doubt will be cast upon findings from utility-based analyses and upon the wisdom of expending millions in research dollars on utility-based studies.

A trial for comparing methods for eliciting treatment preferences from men with advanced prostate cancer: results from the initial visit.

OBJECTIVE: The objective of this study was to evaluate the convergent validity of 3 types of utility measures: standard gamble, time tradeoff, and rating scale. RESEARCH DESIGN: A prospective cohort of 120 men with advanced prostate cancer were first asked to rank order 8 health states, and then utility values were obtained from each participant for each of the 8 health states through 2 of the 3 techniques evaluated (standard gamble, time tradeoff and rating scale). Participants were randomly assigned to 1 of 3 possible pairs of techniques. The validity of the 3 methods, as measured by the convergence and raw score differences of the techniques, was assessed with ANOVA. The ability of the techniques to differentiate health states was determined. The inconsistencies between rankings and utility values were also measured. Proportions of illogical utility responses were assessed as the percent of times when states with more symptoms were given higher or equal utility values than states with fewer symptoms. RESULTS: There were significant differences in raw scores between techniques, but the values were correlated across health states. Utility values were often inconsistent with the rank order of health states. In addition, utility assessment did not differentiate the health states as well as the rank order. Furthermore, utility values were often illogical in that states with more symptoms received equal or higher utility values than states with fewer symptoms. CONCLUSIONS: Use of the utility techniques in cost-effectiveness analysis and decision making has been widely recommended. The results of this study raise serious questions as to the validity and usefulness of the measures.

Assessing quality of life in men with clinically localized prostate cancer: development of a new instrument for use in multiple settings.

BACKGROUND: Quality of life in prostate cancer patients with clinically localized disease has become the focus of increasing attention over the past decade. However, few instruments have been developed and validated to assess quality of life specifically in this patient population. OBJECTIVE: The purpose of this investigation was to create a comprehensive, multi-scale quality of life instrument that can be tailored to the needs of the clinician/investigator in multiple settings. DESIGN, SUBJECTS, AND MEASURES: Patients diagnosed with clinically localized prostate cancer were mailed a questionnaire consisting of new and previously validated quality of life items and ancillary scales. Data from returned questionnaires were analyzed and used to create a multiscale instrument that assesses the effects of treatment and disease on urinary, sexual, and bowel domains, supplemented by a scale assessing anxiety over disease course/effectiveness of treatment. The instrument was then mailed to a second sample of prostate cancer patients once and then again two weeks later to assess test retest reliability. To assess feasibility in clinical settings, the instrument was self-administered to a third patient sample during a urology clinic visit. RESULTS: All scales exhibited good internal consistency and test retest reliability, convergent and discriminant validity, and significant correlations with disease specific, generic health-related, and global measures of quality of life. Men with greater physiologic impairment reported more limitations in role activities and more bother. Scales were also able to differentiate patients undergoing different therapies. All scales exhibited negligible correlations with a measure of socially desirable responding. Additionally, the instrument proved feasible when used as a self-administered questionnaire in a clinical setting. CONCLUSIONS: The current instrument possesses brief multi-item scales that can be successfully self-administered in multiple settings. The instrument is flexible, relatively quick, psychometrically reliable and valid, and permits a more comprehensive assessment of patients' quality of life.

Telephone interviews vs. workstation sessions for acquiring quality of life data.

Patient quality of life data can be acquired in a variety of ways, including over the telephone and through computerized questionnaires. However, if the method of collection produces different results, medical decisions regarding appropriate and cost-effective care may be influenced by collection method. We conducted an experiment where subjects had two quality of life measures, the time trade-off and rating scale utilities, assessed both in telephone interivews and via computer touchscreens. The order of telephone and touchscreen was randomized. We found that rating scale utilities were similar whether obtained via the telephone or via touchscreen regardless of which was done first. However, patients who had their time trade-off utilities assessed over the telephone first did not provide as consistent responses as those elicited first via touchscreen (p = 0.01). Caution is suggested when considering eliciting time trade-off over the telephone with subjects who have not had time trade-off elicited previously.

Assessing the performance of utility techniques in the absence of a gold standard.

BACKGROUND: Utility techniques are the most commonly used means to assess patient preferences for health outcomes. However, whether utility techniques produce valid measures of preference has been difficult to determine in the absence of a gold standard. OBJECTIVE: To introduce and demonstrate two methods that can be used to evaluate how well utility techniques measure patients' preferences. SUBJECTS AND DESIGN: Patients treated for advanced prostate cancer (n = 57) first ranked eight health states in order of preference. Four utility techniques were then used to elicit patients' utilities for each health state. MEASURES: The rating scale, standard gamble, time trade-off, and a modified version of willingness-to-pay techniques were used to elicit patients' utilities. Technique performance was assessed by computing a differentiation and inconsistency score for each technique. RESULTS: Differentiation scores indicated the rating scale permitted respondents to assign unique utility values to about 70% of the health states that should have received unique values. When the other techniques were used, about 40% or less of the health states that should have received unique utility scores actually did receive unique utility scores. Inconsistency scores, which indicate how often participants assign utility scores that contradict how they value health states, indicated that the willingness-to-pay technique produced the lowest rate of inconsistency (10%). However, this technique did not differ significantly from the rating scale or standard gamble on this dimension. CONCLUSIONS: Differentiation and inconsistency offer a means to evaluate the performance of utility techniques, thereby allowing investigators to determine the extent to which utilities they have elicited for a given decision problem are valid. In the current investigation, the differentiation and inconsistency methods indicated that all four techniques performed at sub-optimal levels, though the rating scale out-performed the standard gamble, time trade-off, and willingness-to-pay techniques.

The danger of applying group-level utilities in decision analyses of the treatment of localized prostate cancer in individual patients.

The optimal management strategy for men who have localized prostate cancer remains controversial. This study examines the extent to which suggested treatment based on the perspective of a group or society agrees with that derived from individual patients' preferences. A previously published decision analysis for localized prostate cancer was used to suggest the treatment that maximized quality-adjusted life expectancy. Two treatment recommendations were obtained for each patient: the first (group-level) was derived using the mean utilities of the cohort; the second (individual-level) used his own set of utilities. Group-level utilities misrepresented 25-48% of individuals' preferences depending on the grade of tumor modeled. The best kappa measure achieved between group and individual preferences was 0.11. The average quality-adjusted life years lost due to misrepresentation of preference was as high as 1.7 quality-adjusted life years. Use of aggregated utilities in a group-level decision analysis can ignore the substantial variability at the individual level. Caution is needed when applying a group-level recommendation to the treatment of localized prostate cancer in an individual patient.

Self-verification in clinical depression: the desire for negative evaluation.

Do clinically depressed individuals seek favorable or unfavorable information about the self? Self-verification theory makes the counterintuitive prediction that depressed individuals solicit feedback that confirms their negative self-views. To test this prediction, participants were classified on the basis of a structured clinical interview and self-report measures into high-esteem, low self-esteem, and depressed groups. All participants were offered a choice between receiving favorable or unfavorable feedback; 82% of the depressed participants chose the unfavorable feedback, compared to 64% of the low self-esteem participants and 25% of the high self-esteem participants. Additional evidence indicated that depressed individuals also failed to exploit fully an opportunity to acquire favorable evaluations that were self-verifying. The authors discuss how seeking negative evaluations and failing to seek favorable evaluations may help maintain depression.

When comparisons arise.

People acquire information about their abilities by comparison, and research suggests that people restrict such comparisons to those whom they consider sources of diagnostic information. We suggest that diagnosticity is often considered only after comparisons are made and that people do not fail to make nondiagnostic comparisons so much as they mentally undo them. In 2 studies, participants made nondiagnostic comparisons even when they knew they should not, and quickly unmade them when they were able. These results suggest that social comparisons may be relatively spontaneous, effortless, and unintentional reactions to the performances of others and that they may occur even when people consider such reactions logically inappropriate.

Why people self-verify.

Why do people choose interaction partners who see them as they see themselves? Self-verification theorists propose that a desire to bolster perceptions of predictability and control underlies such activities. In contrast, advocates of positivity strivings argue that people choose such interaction partners in the hope of making themselves feel good. Two studies tested these competing explanations by examining the spontaneous verbalizations of participants as they chose interaction partners. The results suggested that positivity as well as self-verification strivings caused participants with positive self-views to choose partners who appraised them favorably. The epistemic considerations underlying self-verification processes, however, best explained why people with negative self-views chose partners who appraised them unfavorably.