Statistical issues in interpreting clinical trials.

Randomized clinical trial is an important research tool in evaluating new therapeutic agents, devices and procedures. In order to obtain reliable and unbiased results, careful consideration must be given in the design and conduct of the trial. However, bias can be introduced in the analysis of the final data if certain principles are not followed. Several issues are described that make interpretation of analyses challenging. These include the intent-to-treat principle, the use of surrogate outcome measures, subgroup analyses, missing data and noninferiority trials.

Considerations in the evaluation of surrogate endpoints in clinical trials. summary of a National Institutes of Health workshop.

We report on recommendations from a National Institutes of Health Workshop on methods for evaluating the use of surrogate endpoints in clinical trials, which was attended by experts in biostatistics and clinical trials from a broad array of disease areas. Recent advances in biosciences and technology have increased the ability to understand, measure, and model biological mechanisms; appropriate application of these advances in clinical research settings requires collaboration of quantitative and laboratory scientists. Biomarkers, new examples of which arise rapidly from new technologies, are used frequently in such areas as early detection of disease and identification of patients most likely to benefit from new therapies. There is also scientific interest in exploring whether, and under what conditions, biomarkers may substitute for clinical endpoints of phase III trials, although workshop participants agreed that these considerations apply primarily to situations where trials using clinical endpoints are not feasible. Evaluating candidate biomarkers in the exploratory phases of drug development and investigating surrogate endpoints in confirmatory trials require the establishment of a statistical and inferential framework. As a first step, participants reviewed methods for investigating the degree to which biomarkers can explain or predict the effect of treatments on clinical endpoints measured in clinical trials. They also suggested new approaches appropriate in settings where biomarkers reflect only indirectly the important processes on the causal path to clinical disease and where biomarker measurement errors are of concern. Participants emphasized the need for further research on development of such models, whether they are empirical in nature or attempt to describe mechanisms in mathematical terms. Of special interest were meta-analytic models for combining information from multiple studies involving interventions for the same condition. Recommendations also included considerations for design and conduct of trials and for assemblage of databases needed for such research. Finally, there was a strong recommendation for increased training of quantitative scientists in biologic research as well as in statistical methods and modeling to ensure that there will be an adequate workforce to meet future research needs.

Effect of carvedilol on survival in severe chronic heart failure.

BACKGROUND: Beta-blocking agents reduce the risk of hospitalization and death in patients with mild-to-moderate heart failure, but little is known about their effects in severe heart failure. METHODS: We evaluated 2289 patients who had symptoms of heart failure at rest or on minimal exertion, who were clinically euvolemic, and who had an ejection fraction of less than 25 percent. In a double-blind fashion, we randomly assigned 1133 patients to placebo and 1156 patients to treatment with carvedilol for a mean period of 10.4 months, during which standard therapy for heart failure was continued. Patients who required intensive care, had marked fluid retention, or were receiving intravenous vasodilators or positive inotropic drugs were excluded. RESULTS: There were 190 deaths in the placebo group and 130 deaths in the carvedilol group. This difference reflected a 35 percent decrease in the risk of death with carvedilol (95 percent confidence interval, 19 to 48 percent; P=0.00013, unadjusted; P=0.0014, adjusted for interim analyses). A total of 507 patients died or were hospitalized in the placebo group, as compared with 425 in the carvedilol group. This difference reflected a 24 percent decrease in the combined risk of death or hospitalization with carvedilol (95 percent confidence interval, 13 to 33 percent; P<0.001). The favorable effects on both end points were seen consistently in all the subgroups we examined, including patients with a history of recent or recurrent cardiac decompensation. Fewer patients in the carvedilol group than in the placebo group withdrew because of adverse effects or for other reasons (P=0.02). CONCLUSIONS: The previously reported benefits of carvedilol with regard to morbidity and mortality in patients with mild-to-moderate heart failure were also apparent in the patients with severe heart failure who were evaluated in this trial.

Estimating significance level and power comparisons for testing multiple endpoints in clinical trials.

Clinical trials generally include several outcome measures of interest for assessing treatment efficacy and harm. Traditionally a single measure, the primary outcome, is selected and used as the basis for the design, including sample size and power. Secondary outcomes are then generally ordered with respect to their clinical relevance and importance. While this has become the traditional paradigm, recent trials have suggested the need for additional approaches. In this setting, two outcomes are viewed as key, either one being sufficient for proof of efficacy, but with an ordering of preference. The basic question, in such cases, is how to control the overall significance level for the trial. We describe and compare two methods for testing primary and secondary endpoints, accounting for their hierarchical nature-the ordering preference. Both methods are sequential, in the sense that the secondary endpoint is only tested when the primary outcome fails to reach significance. The first method uses a global test for the combination of the primary and secondary endpoints, while the second uses a partial Bonferroni correction. Simulation results indicate that the Bonferroni adjustment method performs as well as the global test method in most cases, and even better in some cases.

Computations for group sequential boundaries using the Lan-DeMets spending function method.

We describe an interactive Fortran program which performs computations related to the design and analysis of group sequential clinical trials using Lan-DeMets spending functions. Many clinical trials include interim analyses of accumulating data and rely on group sequential methods to avoid consequent inflation of the type I error rate. The computations are appropriate for interim test statistics whose distribution or limiting distribution is multivariate normal with independent increments. Recent theoretical results indicate that virtually any design likely to be used in a clinical trial will fall into this category. Interim analyses need not be equally spaced, and their number need not be specified in advance. In addition to determining sequential boundaries using an alpha spending function, the program can perform power computations, compute probabilities associated with a given set of boundaries, and generate confidence intervals.

An overview of group sequential methods in longitudinal clinical trials.

During the last decade, several papers have been published on group sequential methods in general and on sequential longitudinal clinical trials in particular. This paper gives an overview of the proposed methods, emphasizing longitudinal clinical trials. Furthermore, it tries to answer some practical questions that may arise during the conduct of interim analyses in longitudinal trials. Simulations have been carried out to obtain insight in these practical considerations.

The agonising negative trend in monitoring of clinical trials.

Randomised clinical trials are undertaken in the hope of showing positive benefits of a new treatment, but on occasion quite the opposite trend can occur, If the interim data suggest possible negative (harmful) effects of a new treatment. The handling of such emerging negative trends is among the most complicated and ethically challenging scenarios in monitoring clinical trials through repeated interim analyses. Statistical methods are helpful to detect the point of no likely beneficial effect, and the point that separates neutral results from harmful results. However, in practice the decision whether (and exactly when) to stop such a trial involves a complex of other issues that depends on the context of the disease, the treatment being assessed, and the current practice of medicine. Owing to this complexity, an Independent Data and Safety Monitoring Board (DSMB) is best suited to deal with such a situation. Prediction of whether a negative trend will emerge in any trial is not possible. Negative trends were not anticipated in the cardiovascular trials and the trials of lung-cancer prevention described here. In the light of these experiences, all trials and their DSMBs should consider ahead of time the possibility of unexpectedly harmful results, and should document appropriately the statistical guidelines and the decision-making process required to cope with such undesirable events.

A dose-dependent increase in mortality with vesnarinone among patients with severe heart failure. Vesnarinone Trial Investigators.

BACKGROUND: Vesnarinone, an inotropic drug, was shown in a short-term placebo-controlled trial to improve survival markedly in patients with severe heart failure when given at a dose of 60 mg per day, but there was a trend toward an adverse effect on survival when the dose was 120 mg per day. In a longer-term study, we evaluated the effects of daily doses of 60 mg or 30 mg of vesnarinone, as compared with placebo, on mortality and morbidity. METHODS: We enrolled 3833 patients who had symptoms of New York Heart Association class III or IV heart failure and a left ventricular ejection fraction of 30 percent or less despite optimal treatment. The mean follow-up was 286 days. RESULTS: There were significantly fewer deaths in the placebo group (242 deaths, or 18.9 percent) than in the 60-mg vesnarinone group (292 deaths, or 22.9 percent) and longer survival (P=0.02). The increase in mortality with vesnarinone was attributed to an increase in sudden death, presumed to be due to arrhythmia. The quality of life had improved significantly more in the 60-mg vesnarinone group than in the placebo group at 8 weeks (P<0.001) and 16 weeks (P=0.003) after randomization. Trends in mortality and in measures of the quality of life in the 30-mg vesnarinone group were similar to those in the 60-mg group but not significantly different from those in the placebo group. Agranulocytosis occurred in 1.2 percent of the patients given 60 mg of vesnarinone per day and 0.2 percent of those given 30 mg of vesnarinone. CONCLUSIONS: Vesnarinone is associated with a dose-dependent increase in mortality among patients with severe heart failure, an increase that is probably related to an increase in deaths due to arrhythmia. A short-term benefit in terms of the quality of life raises issues about the appropriate therapeutic goal in treating heart failure.

Effect of amlodipine on mode of death among patients with advanced heart failure in the PRAISE trial. Prospective Randomized Amlodipine Survival Evaluation.

Investigations of calcium antagonists in patients with advanced heart failure have raised concern over an increased risk of worsening heart failure and heart failure deaths. We assessed the effect of amlodipine on cause-specific mortality in such patients enrolled in a randomized, double-blind, placebo-controlled trial. In total, 1,153 patients in New York Heart Association class IIIb or IV heart failure were randomized to receive amlodipine or placebo, along with angiotensin-converting enzyme inhibitors, diuretics, and digitalis. Over a median 14.5 months of follow-up, 413 patients died. Cardiovascular deaths accounted for 89% of fatalities, 50% of which were sudden deaths and 45% of which were due to pump failure, with fewer attributed to myocardial infarction (3.3%) or other cardiovascular causes (1.6%). Amlodipine treatment resulted in a greater relative reduction in sudden deaths (21%) than in pump failure deaths (6.6%) overall. When patients were classified by etiology of heart failure (ischemic or nonischemic), cause-specific mortality did not differ significantly between treatment groups in the ischemic stratum. In the nonischemic stratum, however, sudden deaths and pump failure deaths were reduced by 38% and 45%, respectively, with amlodipine. Thus, when added to digitalis, diuretics, and angiotensin-converting enzyme inhibitors in patients with advanced heart failure, amlodipine appears to have no effect on cause-specific mortality in ischemic cardiomyopathy, but both pump failure and sudden deaths appear to be decreased in nonischemic heart failure patients treated with amlodipine.

Distinctions between fraud, bias, errors, misunderstanding, and incompetence.

Randomized clinical trials are challenging not only in their design and analysis, but in their conduct as well. Despite the best intentions and efforts, problems often arise in the conduct of trials, including errors, misunderstandings, and bias. In some instances, key players in a trial may discover that they are not able or competent to meet requirements of the study. In a few cases, fraudulent activity occurs. While none of these problems is desirable, randomized clinical trials are usually found sufficiently robust by many key individuals to produce valid results. Other problems are not tolerable. Confusion may arise among scientists, scientific and lay press, and the public about the distinctions between these areas and their implications. We shall try to define these problems and illustrate their impact through a series of examples.

Effect of amlodipine on morbidity and mortality in severe chronic heart failure. Prospective Randomized Amlodipine Survival Evaluation Study Group.

BACKGROUND: Previous studies have shown that calcium-channel blockers increase morbidity and mortality in patients with chronic heart failure. We studied the effect of a new calcium-channel blocker, amlodipine, in patients with severe chronic heart failure. METHODS: We randomly assigned 1153 patients with severe chronic heart failure and ejection fractions of less than 30 percent to double-blind treatment with either placebo (582 patients) or amlodipine (571 patients) for 6 to 33 months, while their usual therapy was continued. The randomization was stratified on the basis of whether patients had ischemic or nonischemic causes of heart failure. The primary end point of the study was death from any cause and hospitalization for major cardiovascular events. RESULTS: Primary end points were reached in 42 percent of the placebo group and 39 percent of the amlodipine group, representing a 9 percent reduction in the combined risk of fatal and nonfatal events with amlodipine (95 percent confidence interval, 24 percent reduction to 10 percent increase; P=0.31). A total of 38 percent of the patients in the placebo group died, as compared with 33 percent of those in the amlodipine group, representing a 16 percent reduction in the risk of death with amlodipine (95 percent confidence interval, 31 percent reduction to 2 percent increase; P=0.07). Among patients with ischemic heart disease, there was no difference between the amlodipine and placebo groups in the occurrence of either end point. In contrast, among patients with nonischemic cardiomyopathy, amlodipine reduced the combined risk of fatal and nonfatal events by 31 percent (P=0.04) and decreased the risk of death by 46 percent (P<0.001). CONCLUSIONS: Amlodipine did not increase cardiovascular morbidity or mortality in patients with severe heart failure. The possibility that amlodipine prolongs survival in patients with nonischemic dilated cardiomyopathy requires further study.

The data and safety monitoring board and acquired immune deficiency syndrome (AIDS) clinical trials.

The urgency of the Acquired immune deficiency syndrome (AIDS) epidemic has mandated that multiple therapeutic approaches be developed and that these approaches be evaluated through clinical trials. To oversee these trials, the National Institute of Allergy and Infectious Diseases (NIAID) has created three large clinical trial programs monitored by a Data and Safety Monitoring Board (DSMB). For each clinical trial, this Board uses a standardized approach employing contemporary biostatistical, medical, and ethical principles. The DSMB is responsible for reviewing interim data on clinical trial performance, treatment safety and efficacy, and overall study progress. If interim results provide convincing evidence of either excessive adverse effects or significant treatment benefit, the DSMB may recommend early termination of the trial to the NIAID and the study investigators. The responsibility, organization, and operating procedures of this DSMB are presented and illustrated through three clinical trials sponsored by NIAID and monitored by the Board. The rationale and operational model for the DSMB may be a useful example for the development of similar review processes in other HIV clinical trial settings.

A comparison of methods for correlated ordinal measures with ophthalmic applications.

For many clinical trials and epidemiologic investigations in the field of ophthalmology, paired ordinal data are often collected through the detailed grading of retinal photographs. One method for analysis of these data is the extension of the generalized estimating equation (GEE) methodology to multinomial data with cumulative link functions. Prior to the development of this advanced technique, however, ophthalmologists developed a method of combining the ordinal responses of both eyes of a patient into a single person-level response on a new ordinal scale. A relationship between the regression coefficients of these two methods is derived as a function of the correlation between eyes. We investigate the applicability of this result and the relationship of the standard errors in simulation experiments and in an example from the Wisconsin Epidemiologic Study of Diabetic Retinopathy.

Group sequential comparison of changes: ad-hoc versus more exact method.

There are many clinical trials in which we are interested in comparing changes in responses between two treatment groups sequentially. Investigators might assume a simple linear model, take the average of the ordinary least square estimators of slope within each treatment group, and use the standardized difference between these two averages as the test statistic at each interim analysis. Ad-hoc construction of the boundary values for interim analysis might be based on group sequential methods which assume that the interim test statistics have independent increments even though it may not be true when a response variable is measured repeatedly over time for each subject. This ad-hoc method is very simple and appealing, and thus has been used in a clinical trial setting. Lee and DeMets (1991, Journal of the American Statistical Association 86, 757-762) have proposed a more exact group sequential method for comparing rates of change. Under the assumption that the response follows the linear mixed effects model, they have derived the asymptotic joint distribution of the sequentially computed statistics. Construction of group sequential boundaries is based on this distribution theory. By simulation studies, we first study the robustness of the more exact method to violations of the typical assumptions. In addition, we compare the ad-hoc method with the more exact method and examine how well these two methods work for various situations. The relationship between two different information times, Fisher information and a surrogate information, are also discussed from the simulation studies.