Statistical issues in interpreting clinical trials.

Randomized clinical trial is an important research tool in evaluating new therapeutic agents, devices and procedures. In order to obtain reliable and unbiased results, careful consideration must be given in the design and conduct of the trial. However, bias can be introduced in the analysis of the final data if certain principles are not followed. Several issues are described that make interpretation of analyses challenging. These include the intent-to-treat principle, the use of surrogate outcome measures, subgroup analyses, missing data and noninferiority trials.

Considerations in the evaluation of surrogate endpoints in clinical trials. summary of a National Institutes of Health workshop.

We report on recommendations from a National Institutes of Health Workshop on methods for evaluating the use of surrogate endpoints in clinical trials, which was attended by experts in biostatistics and clinical trials from a broad array of disease areas. Recent advances in biosciences and technology have increased the ability to understand, measure, and model biological mechanisms; appropriate application of these advances in clinical research settings requires collaboration of quantitative and laboratory scientists. Biomarkers, new examples of which arise rapidly from new technologies, are used frequently in such areas as early detection of disease and identification of patients most likely to benefit from new therapies. There is also scientific interest in exploring whether, and under what conditions, biomarkers may substitute for clinical endpoints of phase III trials, although workshop participants agreed that these considerations apply primarily to situations where trials using clinical endpoints are not feasible. Evaluating candidate biomarkers in the exploratory phases of drug development and investigating surrogate endpoints in confirmatory trials require the establishment of a statistical and inferential framework. As a first step, participants reviewed methods for investigating the degree to which biomarkers can explain or predict the effect of treatments on clinical endpoints measured in clinical trials. They also suggested new approaches appropriate in settings where biomarkers reflect only indirectly the important processes on the causal path to clinical disease and where biomarker measurement errors are of concern. Participants emphasized the need for further research on development of such models, whether they are empirical in nature or attempt to describe mechanisms in mathematical terms. Of special interest were meta-analytic models for combining information from multiple studies involving interventions for the same condition. Recommendations also included considerations for design and conduct of trials and for assemblage of databases needed for such research. Finally, there was a strong recommendation for increased training of quantitative scientists in biologic research as well as in statistical methods and modeling to ensure that there will be an adequate workforce to meet future research needs.

Challenges of subgroup analyses in multinational clinical trials: experiences from the MERIT-HF trial.

BACKGROUND: International placebo-controlled survival trials (Metoprolol Controlled-Release Randomised Intervention Trial in Heart Failure [MERIT-HF], Cardiac Insufficiency Bisoprolol Study [CIBIS-II], and Carvedilol Prospective Randomized Cumulative Survival trial [COPERNICUS]) evaluating the effects of b-blockade in patients with heart failure have all demonstrated highly significant positive effects on total mortality as well as total mortality plus all-cause hospitalization. Also, the analysis of the US Carvedilol Program indicated an effect on these end points. Although none of these trials are large enough to provide definitive results in any particular subgroup, it is natural for physicians to examine the consistency of results across various subgroups or risk groups. Our purpose was to examine both predefined and post hoc subgroups in the MERIT-HF trial to provide guidance as to whether any subgroup is at increased risk, despite an overall strongly positive effect, and to discuss the difficulties and limitations in conducting such subgroup analyses. METHODS: The study was conducted at 313 clinical sites in 16 randomization regions across 14 countries, with a total of 3991 patients. Total mortality (first primary end point) and total mortality plus all-cause hospitalization (second primary end point) were analyzed on a time to first event. The first secondary end point was total mortality plus hospitalization for heart failure. RESULTS: Overall, MERIT-HF demonstrated a hazard ratio of 0.66 for total mortality and 0.81 for mortality plus all-cause hospitalization. The hazard ratio of the first secondary end point of mortality plus hospitalization for heart failure was 0.69. The results were remarkably consistent for both primary outcomes and the first secondary outcome across all predefined subgroups as well as for nearly all post hoc subgroups. The results of the post hoc US subgroup showed a mortality hazard ratio of 1.05. However, the US results regarding both the second primary combined outcome of total mortality plus all-cause hospitalization and of the first secondary combined outcome of total mortality plus heart failure hospitalization were in concordance with the overall results of MERIT-HF. Tests of country by treatment interaction (14 countries) revealed a nonsignificant P value of.22 for total mortality. The mortality hazard ratio for US patients in New York Heart Association (NYHA) class III/IV was 0.80, and it was 2.24 for patients in NYHA class II, which is not consistent with causality by biologic gradient. We have not been able to identify any confounding factor in baseline characteristics, baseline treatment, or treatment during follow-up that could account for any treatment by country interaction. Thus we attribute the US subgroup mortality hazard ratio to be due to chance. CONCLUSIONS: Just as we must be extremely cautious in overinterpreting positive effects in subgroups, even those that are predefined, we must also be cautious in focusing on subgroups with an apparent neutral or negative trend. We should examine subgroups to obtain a general sense of consistency, which is clearly the case in MERIT-HF. We should expect some variation of the treatment effect around the overall estimate as we examine a large number of subgroups because of small sample size in subgroups and chance. Thus the best estimate of the treatment effect on total mortality for any subgroup is the estimate of the hazard ratio for the overall trial.

Effect of carvedilol on survival in severe chronic heart failure.

BACKGROUND: Beta-blocking agents reduce the risk of hospitalization and death in patients with mild-to-moderate heart failure, but little is known about their effects in severe heart failure. METHODS: We evaluated 2289 patients who had symptoms of heart failure at rest or on minimal exertion, who were clinically euvolemic, and who had an ejection fraction of less than 25 percent. In a double-blind fashion, we randomly assigned 1133 patients to placebo and 1156 patients to treatment with carvedilol for a mean period of 10.4 months, during which standard therapy for heart failure was continued. Patients who required intensive care, had marked fluid retention, or were receiving intravenous vasodilators or positive inotropic drugs were excluded. RESULTS: There were 190 deaths in the placebo group and 130 deaths in the carvedilol group. This difference reflected a 35 percent decrease in the risk of death with carvedilol (95 percent confidence interval, 19 to 48 percent; P=0.00013, unadjusted; P=0.0014, adjusted for interim analyses). A total of 507 patients died or were hospitalized in the placebo group, as compared with 425 in the carvedilol group. This difference reflected a 24 percent decrease in the combined risk of death or hospitalization with carvedilol (95 percent confidence interval, 13 to 33 percent; P<0.001). The favorable effects on both end points were seen consistently in all the subgroups we examined, including patients with a history of recent or recurrent cardiac decompensation. Fewer patients in the carvedilol group than in the placebo group withdrew because of adverse effects or for other reasons (P=0.02). CONCLUSIONS: The previously reported benefits of carvedilol with regard to morbidity and mortality in patients with mild-to-moderate heart failure were also apparent in the patients with severe heart failure who were evaluated in this trial.

University of Wisconsin IAIMS planning: organizational challenges within a faculty governance model.

The University of Wisconsin-Madison Health Sciences Schools are currently in the planning stage of developing an Integrated Advanced Information Management System (IAIMS). The planning phase of this project attends to the unique opportunities that are found at the flagship campus of a large state university system. Statewide teaching and research initiatives and accelerated campus-level capital development challenge the planners to create an IAIMS plan that anticipates an emerging health science environment. Additionally, UW-Madison has an organizational culture with a strong tradition of faculty governance, which provides a very desirable and flexible decision-making environment for a cross-discipline collaborative information management initiative. Development of a shared IAIMS vision conflicts with a governance model that most directly supports intradepartmental decision-making. The challenge presented here for an IAIMS initiative has less to do with hard wiring a technical infrastructure and more to do with increased stakeholder cooperation in a highly decentralized organization with autonomous information systems.

Estimating significance level and power comparisons for testing multiple endpoints in clinical trials.

Clinical trials generally include several outcome measures of interest for assessing treatment efficacy and harm. Traditionally a single measure, the primary outcome, is selected and used as the basis for the design, including sample size and power. Secondary outcomes are then generally ordered with respect to their clinical relevance and importance. While this has become the traditional paradigm, recent trials have suggested the need for additional approaches. In this setting, two outcomes are viewed as key, either one being sufficient for proof of efficacy, but with an ordering of preference. The basic question, in such cases, is how to control the overall significance level for the trial. We describe and compare two methods for testing primary and secondary endpoints, accounting for their hierarchical nature-the ordering preference. Both methods are sequential, in the sense that the secondary endpoint is only tested when the primary outcome fails to reach significance. The first method uses a global test for the combination of the primary and secondary endpoints, while the second uses a partial Bonferroni correction. Simulation results indicate that the Bonferroni adjustment method performs as well as the global test method in most cases, and even better in some cases.

Computations for group sequential boundaries using the Lan-DeMets spending function method.

We describe an interactive Fortran program which performs computations related to the design and analysis of group sequential clinical trials using Lan-DeMets spending functions. Many clinical trials include interim analyses of accumulating data and rely on group sequential methods to avoid consequent inflation of the type I error rate. The computations are appropriate for interim test statistics whose distribution or limiting distribution is multivariate normal with independent increments. Recent theoretical results indicate that virtually any design likely to be used in a clinical trial will fall into this category. Interim analyses need not be equally spaced, and their number need not be specified in advance. In addition to determining sequential boundaries using an alpha spending function, the program can perform power computations, compute probabilities associated with a given set of boundaries, and generate confidence intervals.

An overview of group sequential methods in longitudinal clinical trials.

During the last decade, several papers have been published on group sequential methods in general and on sequential longitudinal clinical trials in particular. This paper gives an overview of the proposed methods, emphasizing longitudinal clinical trials. Furthermore, it tries to answer some practical questions that may arise during the conduct of interim analyses in longitudinal trials. Simulations have been carried out to obtain insight in these practical considerations.

The agonising negative trend in monitoring of clinical trials.

Randomised clinical trials are undertaken in the hope of showing positive benefits of a new treatment, but on occasion quite the opposite trend can occur, If the interim data suggest possible negative (harmful) effects of a new treatment. The handling of such emerging negative trends is among the most complicated and ethically challenging scenarios in monitoring clinical trials through repeated interim analyses. Statistical methods are helpful to detect the point of no likely beneficial effect, and the point that separates neutral results from harmful results. However, in practice the decision whether (and exactly when) to stop such a trial involves a complex of other issues that depends on the context of the disease, the treatment being assessed, and the current practice of medicine. Owing to this complexity, an Independent Data and Safety Monitoring Board (DSMB) is best suited to deal with such a situation. Prediction of whether a negative trend will emerge in any trial is not possible. Negative trends were not anticipated in the cardiovascular trials and the trials of lung-cancer prevention described here. In the light of these experiences, all trials and their DSMBs should consider ahead of time the possibility of unexpectedly harmful results, and should document appropriately the statistical guidelines and the decision-making process required to cope with such undesirable events.

A dose-dependent increase in mortality with vesnarinone among patients with severe heart failure. Vesnarinone Trial Investigators.

BACKGROUND: Vesnarinone, an inotropic drug, was shown in a short-term placebo-controlled trial to improve survival markedly in patients with severe heart failure when given at a dose of 60 mg per day, but there was a trend toward an adverse effect on survival when the dose was 120 mg per day. In a longer-term study, we evaluated the effects of daily doses of 60 mg or 30 mg of vesnarinone, as compared with placebo, on mortality and morbidity. METHODS: We enrolled 3833 patients who had symptoms of New York Heart Association class III or IV heart failure and a left ventricular ejection fraction of 30 percent or less despite optimal treatment. The mean follow-up was 286 days. RESULTS: There were significantly fewer deaths in the placebo group (242 deaths, or 18.9 percent) than in the 60-mg vesnarinone group (292 deaths, or 22.9 percent) and longer survival (P=0.02). The increase in mortality with vesnarinone was attributed to an increase in sudden death, presumed to be due to arrhythmia. The quality of life had improved significantly more in the 60-mg vesnarinone group than in the placebo group at 8 weeks (P<0.001) and 16 weeks (P=0.003) after randomization. Trends in mortality and in measures of the quality of life in the 30-mg vesnarinone group were similar to those in the 60-mg group but not significantly different from those in the placebo group. Agranulocytosis occurred in 1.2 percent of the patients given 60 mg of vesnarinone per day and 0.2 percent of those given 30 mg of vesnarinone. CONCLUSIONS: Vesnarinone is associated with a dose-dependent increase in mortality among patients with severe heart failure, an increase that is probably related to an increase in deaths due to arrhythmia. A short-term benefit in terms of the quality of life raises issues about the appropriate therapeutic goal in treating heart failure.

Effect of amlodipine on mode of death among patients with advanced heart failure in the PRAISE trial. Prospective Randomized Amlodipine Survival Evaluation.

Investigations of calcium antagonists in patients with advanced heart failure have raised concern over an increased risk of worsening heart failure and heart failure deaths. We assessed the effect of amlodipine on cause-specific mortality in such patients enrolled in a randomized, double-blind, placebo-controlled trial. In total, 1,153 patients in New York Heart Association class IIIb or IV heart failure were randomized to receive amlodipine or placebo, along with angiotensin-converting enzyme inhibitors, diuretics, and digitalis. Over a median 14.5 months of follow-up, 413 patients died. Cardiovascular deaths accounted for 89% of fatalities, 50% of which were sudden deaths and 45% of which were due to pump failure, with fewer attributed to myocardial infarction (3.3%) or other cardiovascular causes (1.6%). Amlodipine treatment resulted in a greater relative reduction in sudden deaths (21%) than in pump failure deaths (6.6%) overall. When patients were classified by etiology of heart failure (ischemic or nonischemic), cause-specific mortality did not differ significantly between treatment groups in the ischemic stratum. In the nonischemic stratum, however, sudden deaths and pump failure deaths were reduced by 38% and 45%, respectively, with amlodipine. Thus, when added to digitalis, diuretics, and angiotensin-converting enzyme inhibitors in patients with advanced heart failure, amlodipine appears to have no effect on cause-specific mortality in ischemic cardiomyopathy, but both pump failure and sudden deaths appear to be decreased in nonischemic heart failure patients treated with amlodipine.

Distinctions between fraud, bias, errors, misunderstanding, and incompetence.

Randomized clinical trials are challenging not only in their design and analysis, but in their conduct as well. Despite the best intentions and efforts, problems often arise in the conduct of trials, including errors, misunderstandings, and bias. In some instances, key players in a trial may discover that they are not able or competent to meet requirements of the study. In a few cases, fraudulent activity occurs. While none of these problems is desirable, randomized clinical trials are usually found sufficiently robust by many key individuals to produce valid results. Other problems are not tolerable. Confusion may arise among scientists, scientific and lay press, and the public about the distinctions between these areas and their implications. We shall try to define these problems and illustrate their impact through a series of examples.

Approached to monitoring the results of long-term disease prevention trials: examples from the Women's Health Initiative.

We contrast monitoring therapeutic trials with monitoring prevention trails. We argue that in monitoring prevention trials one should place more emphasis on formally defined global measures of health, not simply on a single targeted disease, particularly when an intervention may reduce the incidence of some diseases but increase the incidence of others. We describe one approach, illustrated by the Women's Health Initiative. For each of several sets of hypothetical interim results ("scenarios"), members of the Data and Safety Monitoring Committee (DSMC) were asked whether they would continue or stop the trial. In parallel with this exercise, various statistical methods of monitoring that are based on (1) the primary targeted disease, (2) a combination of various disease outcomes, or (3) a mixture of both were applied to these scenarios. One objective was to find a statistical approach that mirrors the majority view of the DSMC. A second objective was to stimulate discussion among DSMC members in preparation for their task of monitoring the trial as the real data become available. We found that no single method fully matched the majority vote of the DSMC. However, a mixed approach requiring the primary outcome to be significant and the global index to be "supportive," with separate monitoring of adverse effects, corresponded with the majority vote quite well. This approach maintains the emphasis on the primary hypothesis while assuring that broader safety and ethical issues of multiple diseases are incorporated.

Effect of amlodipine on morbidity and mortality in severe chronic heart failure. Prospective Randomized Amlodipine Survival Evaluation Study Group.

BACKGROUND: Previous studies have shown that calcium-channel blockers increase morbidity and mortality in patients with chronic heart failure. We studied the effect of a new calcium-channel blocker, amlodipine, in patients with severe chronic heart failure. METHODS: We randomly assigned 1153 patients with severe chronic heart failure and ejection fractions of less than 30 percent to double-blind treatment with either placebo (582 patients) or amlodipine (571 patients) for 6 to 33 months, while their usual therapy was continued. The randomization was stratified on the basis of whether patients had ischemic or nonischemic causes of heart failure. The primary end point of the study was death from any cause and hospitalization for major cardiovascular events. RESULTS: Primary end points were reached in 42 percent of the placebo group and 39 percent of the amlodipine group, representing a 9 percent reduction in the combined risk of fatal and nonfatal events with amlodipine (95 percent confidence interval, 24 percent reduction to 10 percent increase; P=0.31). A total of 38 percent of the patients in the placebo group died, as compared with 33 percent of those in the amlodipine group, representing a 16 percent reduction in the risk of death with amlodipine (95 percent confidence interval, 31 percent reduction to 2 percent increase; P=0.07). Among patients with ischemic heart disease, there was no difference between the amlodipine and placebo groups in the occurrence of either end point. In contrast, among patients with nonischemic cardiomyopathy, amlodipine reduced the combined risk of fatal and nonfatal events by 31 percent (P=0.04) and decreased the risk of death by 46 percent (P<0.001). CONCLUSIONS: Amlodipine did not increase cardiovascular morbidity or mortality in patients with severe heart failure. The possibility that amlodipine prolongs survival in patients with nonischemic dilated cardiomyopathy requires further study.