Some of David Byar's work: in appreciation.

David Byar wrote and thought with great clarity about randomized clinical trials (RCTs), pointing to their strengths--and to the limitations of alternative investigative approaches. An issue that he treated quite briefly in the seven publications that I review here is the possible gains to be had from post hoc covariate adjustment of RCT results. Calculations that I offer suggest that for realistic sample sizes and covariates of modest strength, those gains cannot be large, on average. The price to be paid for them is to replace the clear, cogent RCT paradigm with a more complex and diffuse one; this seems just cause for unease.

Measuring effects without randomized trials? Options, problems, challenges.

The idea of using information routinely generated in the course of health care delivery for assessing the relative efficacy of alternative therapies has an undeniable attraction. If feasible, it would enable the difficulties of running a randomized controlled trial (RCT) to be bypassed. But grave obstacles, both practical and theoretical, beset the effort. If, nonetheless, the data base approach is chosen, certain suggestions may be helpful. First, make sure that necessity actually prevents using an RCT. Reconsider that question. Special attention is due to two variants of the usual RCT: the "firms" approach and the large simple trial. If, after reconsideration, a data base approach still is chosen, then let the undertaking be carefully planned, with participation from those who will produce the data and also from future users of the results. Let the planning result in a protocol, and in provisions for a quality assurance program. Finally, it is time to respond to a challenge left us by David Byar: arrange to record why the patient is being given the therapy selected. This information should be a powerful adjustment variable; arranging to collect it will call for imaginative thinking, experimentation, and patience, but it is an idea deserving much effort.

Thoracic CT images: effect of lossy image compression on diagnostic accuracy.

PURPOSE: To evaluate the effects of lossy image (noninvertible) compression on diagnostic accuracy of thoracic computed tomographic images. MATERIALS AND METHODS: Sixty images from patients with mediastinal adenopathy and pulmonary nodules were compressed to six different levels with tree-structured vector quantization. Three radiologists then used the original and compressed images for diagnosis. Unlike many previous receiver operating characteristic-based studies that used confidence rankings and binary detection tasks, this study examined the sensitivity and predictive value positive scores from nonbinary detection tasks. RESULTS: At the 5% significance level, there was no statistically significant difference in diagnostic accuracy of image assessment at compression rates of up to 9:1. CONCLUSION: The techniques presented for evaluation of image quality do not depend on the specific compression algorithm and provide a useful approach to evaluation of the benefits of any lossy image processing technique.

Estimating diagnostic accuracy from multiple conflicting reports: a new meta-analytic method.

Reports of diagnostic accuracy often differ. The authors present a method to summarize disparate reports that uses a logistic transformation and linear regression to produce a summary receiver operating characteristic curve. The curve is useful for summarizing a body of diagnostic accuracy literature, comparing technologies, detecting outliers, and finding the optimum operating point of the test. Examples from clinical chemistry and diagnostic radiology are provided. By extending the logic of meta-analysis to diagnostic testing, the method provides a new tool for technology assessment.

Combining independent studies of a diagnostic test into a summary ROC curve: data-analytic approaches and some additional considerations.

We consider how to combine several independent studies of the same diagnostic test, where each study reports an estimated false positive rate (FPR) and an estimated true positive rate (TPR). We propose constructing a summary receiver operating characteristic (ROC) curve by the following steps. (i) Convert each FPR to its logistic transform U and each TPR to its logistic transform V after increasing each observed frequency by adding 1/2. (ii) For each study calculate D = V - U, which is the log odds ratio of TPR and FPR, and S = V + U, an implied function of test threshold; then plot each study's point (Si, Di). (iii) Fit a robust-resistant regression line to these points (or an equally weighted least-squares regression line), with V - U as the dependent variable. (iv) Back-transform the line to ROC space. To avoid model-dependent extrapolation from irrelevant regions of ROC space we propose defining a priori a value of FPR so large that the test simply would not be used at that FPR, and a value of TPR so low that the test would not be used at that TPR. Then (a) only data points lying in the thus defined north-west rectangle of the unit square are used in the data analysis, and (b) the estimated summary ROC is depicted only within that subregion of the unit square. We illustrate the methods using simulated and real data sets, and we point to ways of comparing different tests and of taking into account the effects of covariates.

Tree-structured vector quantization of CT chest scans: image quality and diagnostic accuracy.

The authors apply a lossy compression algorithm to medical images, and quantify the quality of the images by the diagnostic performance of radiologists, as well as by traditional signal-to-noise ratios and subjective ratings. The authors' study is unlike previous studies of the effects of lossy compression in that they consider nonbinary detection tasks, simulate actual diagnostic practice instead of using paired tests or confidence rankings, use statistical methods that are more appropriate for nonbinary clinical data than are the popular receiver operating characteristic curves, and use low-complexity predictive tree-structured vector quantization for compression rather than DCT-based transform codes combined with entropy coding. The authors' diagnostic tasks are the identification of nodules (tumors) in the lungs and lymphadenopathy in the mediastinum from computerized tomography (CT) chest scans. Radiologists read both uncompressed and lossy compressed versions of images. For the image modality, compression algorithm, and diagnostic tasks the authors consider, the original 12 bit per pixel (bpp) CT image can be compressed to between 1 bpp and 2 bpp with no significant changes in diagnostic accuracy. The techniques presented here for evaluating image quality do not depend on the specific compression algorithm and are useful new methods for evaluating the benefits of any lossy image processing technique.

Enhancement of human immunodeficiency virus (HIV)-specific CD4+ and CD8+ cytotoxic T-lymphocyte activities in HIV-infected asymptomatic patients given recombinant gp160 vaccine.

Twenty-six human immunodeficiency virus (HIV)-infected asymptomatic patients with CD4+ lymphocytes > 400 per mm3 were randomly allocated to a range of doses of recombinant gp160 or a control (recombinant hepatitis B vaccine) on a double-blind basis. Each patient received an injection at 0, 4, 12, 24, 36, and 48 weeks. Treatment assignments were decoded when all patients reached 28 weeks of the study period. HIV-1-specific CD4+ and CD8+ cytotoxic T lymphocyte (CTL) activities were assessed in vitro before vaccination and 2 weeks after each injection. There were significant increases in major histocompatibility complex-restricted HIV-1 Env-specific CD4+ and CD8+ CTL activities in 18 of 21 gp160 vaccinees. No control-injected patients showed a significant change. Neither gp160 nor control recipients showed significant changes in HIV-1 Gag- and Pol-specific CTL activities. HIV-1 Env-specific CD4+ and CD8+ CTL precursor frequencies were also measured in three vaccinees before and at 24 weeks after vaccine was started. CTL precursor frequencies also increased in both CD4+ and CD8+ populations. This study shows that this gp160 vaccine is immunogenic in enhancing HIV-1 Env-specific cytotoxic T-cell-mediated immunity in HIV-seropositive individuals.

Framework for considering the role of data bases in technology assessment.

The use of large data bases in technology assessment offers a number of attractive possibilities for the future. This article outlines the major approaches to technology assessment and how data bases can be used to enhance each approach.

A note on the Wilcoxon-Mann-Whitney test for 2 X kappa ordered tables.

Biological and medical investigations often use ordered categorical data. When two groups are to be compared and the data for the groups fall in three or more ordered categories, the Wilcoxon-Mann-Whitney (WMW) test uses information in the ordering to give a test that is usually powerful against shift alternatives. However, such applications of WMW often involve distributions for which extensive ties play an important role. Newly available computer programs for performing exact tests give deeper insights into the characteristics of the exact WMW distributions and the suitability of normal approximations. We offer practical advice, based on experience with published biomedical data sets and on numerical studies of hypothetical ordered tables, for the use of WMW and its normal approximations.

Analyzing data from ordered categories.

Clinical investigations often involve data in the form of ordered categories--e.g., "worse," "unchanged," "improved," "much improved." Comparison of two groups when the data are of this kind should not be done by the chi-square test, which wastes information and is insensitive in this context. The Wilcoxon-Mann-Whitney test provides a proper analysis. Alternatively, scores may be assigned to the categories in order, and the t-test applied. We demonstrate both approaches here. Sometimes data in ordered categories are reduced to a two-by-two table by the collapsing of the high categories into one category and the low categories into another. This practice is inefficient; moreover, it entails avoidable subjectivity in the choice of the cutting point that defines the two super-categories. The Wilcoxon-Mann-Whitney procedure (or the t-test with use of ordered scores) is preferable. A survey of research articles in Volume 306 of the New England Journal of Medicine shows many instances of ordered-category data (about 20 per cent of the articles had such data) and no instance of analysis by the preferred methods presented here. We suggest that investigators who are unfamiliar with these methods should seek the assistance of a professional statistician when they must deal with such data.

Limits of detection and quantification, as applied to an assay for digoxin.

The low-range sensitivity of a drug assay in routine use is an important aspect of that assay's clinical usefulness. We define three concepts for determining low range sensitivity: LC, the detection threshold; LD, the lower limit of "virtually ensured" detection; and LQ, the limit of quantitative determination. To illustrate these concepts, we apply them to a radioimmunoassay for digoxin, and show that for standards ranging from 0.3 to 5.0 micrograms/L, the LQ is 0.35 micrograms/L (+/- 10%).