Prevalence-value-accuracy plots: a new method for comparing diagnostic tests based on misclassification costs.

The clinical accuracy of diagnostic tests commonly is assessed by ROC analysis. ROC plots, however, do not directly incorporate the effect of prevalence or the value of the possible test outcomes on test performance, which are two important factors in the practical utility of a diagnostic test. We describe a new graphical method, referred to as a prevalence-value-accuracy (PVA) plot analysis, which includes, in addition to accuracy, the effect of prevalence and the cost of misclassifications (false positives and false negatives) in the comparison of diagnostic test performance. PVA plots are contour plots that display the minimum cost attributable to misclassifications (z-axis) at various optimum decision thresholds over a range of possible values for prevalence (x-axis) and the unit cost ratio (UCR; y-axis), which is an index of the cost of a false-positive vs a false-negative test result. Another index based on the cost of misclassifications can be derived from PVA plots for the quantitative comparison of test performance. Depending on the region of the PVA plot that is used to calculate the misclassification cost index, it can potentially lead to a different interpretation than the ROC area index on the relative value of different tests. A PVA-threshold plot, which is a variation of a PVA plot, is also described for readily identifying the optimum decision threshold at any given prevalence and UCR. In summary, the advantages of PVA plot analysis are the following: (a) it directly incorporates the effect of prevalence and misclassification costs in the analysis of test performance; (b) it yields a quantitative index based on the costs of misclassifications for comparing diagnostic tests; (c) it provides a way to restrict the comparison of diagnostic test performance to a clinically relevant range of prevalence and UCR; and (d) it can be used to directly identify an optimum decision threshold based on prevalence and misclassification costs.

Adsorption to aluminum hydroxide promotes the activity of IL-12 as an adjuvant for antibody as well as type 1 cytokine responses to HIV-1 gp120.

A series of protocols were tested to examine the adjuvant effects of IL-12 on humoral and type 1 cytokine responses elicited in mice by recombinant gp120 envelope protein from HIV-1. This Ag fails to induce detectable Ab responses when administered s.c. alone, but stimulates low Ab levels when combined with aluminum hydroxide (alum). Moreover, when i.p. injected rIL-12 was included in the immunization, no increase in Ab production was observed. Importantly, optimal gp120 Ab responses were achieved by immunizing mice s.c. with gp120 and rIL-12 simultaneously coadsorbed to alum. These animals displayed a highly polarized, type 1 cytokine profile, with the emergence of anti-gp120 Ig belonging to the IgG2 and IgG3 isotypes. In addition, a major increase occurred in Ab of the IgG1 subclass. The superior adjuvant activity of alum-adsorbed IL-12 compared with that of the free cytokine correlated with the prolonged detection of IFN-gamma in the sera of animals immunized using the former procedure. In related experiments, in vitro neutralization of IL-12 was shown to inhibit IFN-gamma production by spleen cells from mice immunized with gp120 plus alum, but not by splenocytes from mice primed in the presence of IL-12, suggesting that the latter protocol induces a stable type 1 phenotype. These studies demonstrate that presentation of IL-12 on alum enhances its immunomodulatory effects and establish a protocol for the use of the cytokine as an adjuvant for simultaneously promoting both humoral Ab and type 1 cytokine responses.

Linear regression estimation of minimal detectable concentration. Thyrotropin as an example.

BACKGROUND: Minimal detectable concentration is an important analytic feature of certain clinical immunoassays. We believe that accuracy is an important component of the minimal detectable concentration; for a given observed concentration to be meaningful, it should reflect a consistent linear relationship with the amount of analyte actually present. METHODS: To evaluate the minimal detectable concentration, we developed a linearity regression protocol based on accuracy and also accounting for between-run variability. Using serial twofold dilutions of serum samples, we regressed the log of concentration (x) and of dilution (y) with linear, second-, and third-order polynomials. Initially, we evaluated two elements to find the linear region of the dataset, establishing the statistical significance of the beta coefficients with a t test and the reduction of the sum of square of the residuals between the linear regression and the higher-order regressions by means of an F test. As needed, we successively eliminated the lowest point until the linear regression was the best fit. Once we found the best fit, we added the most recently removed point back and calculated the difference between the value predicted by the first-order regression and the observed value. If the difference was not analytically significant, then we considered the point to be part of the linear set; otherwise, it was not included. In either case, the lowest included point was considered to be the minimal detectable concentration. RESULTS: We applied the technique in evaluating two automated systems for serum thyrotropin. One system appeared linear and accurate down to 0.02 mU/L, or better, approximately 77% of the time, and to 0.01 mU/L 68% of the time. The second system was linear infrequently and appeared to be useful down to 0.02 mU/L, or better, only about 20% of the time. CONCLUSIONS: This accuracy-based approach to determining the minimal detectable concentration is an attractive alternative to current empiric approaches, which are based only on interassay variability.

Apolipoproteins and lipids in coronary artery disease. Analysis of diagnostic accuracy using receiver operating characteristic plots and areas.

Identification of clinically important coronary artery disease is a current goal in patient treatment. Serum lipid and apolipoprotein concentrations are commonly used to identify individuals who may have significant disease. With data published earlier from 304 men classified by coronary angiography, this study used receiver operating characteristic plots and analysis to examine the ability of 10 lipid or lipoprotein measures to discriminate between subjects with and without coronary artery disease. The analysis illustrated the simple elegance of receiver operating characteristic plots and demonstrated that all 10 measures are, at best, only moderately accurate.

Purification of an Escherichia coli-expressed Nef protein from the human immunodeficiency virus-type 2.

The entire nef gene sequence of HIV-2, NIH-Z strain, has been cloned into the pJL6 expression vector and used for the synthesis of a 23-kDa protein in E. coli. The expressed protein is a fusion between the N-terminal 13 amino acids of the cII gene, 8 amino acids resulting from the ligation procedure, and the 180 amino acids that comprise the HIV-2 Nef sequence from the NIH-Z strain. The bacterially expressed Nef protein has been purified to apparent homogeneity on analytical scale (10-20 micrograms) by a combination of sequential detergent extraction, gel filtration, and reversed-phase high-performance chromatography. The expressed Nef protein is highly susceptible to proteolysis (chymotryptic-like activity) and this property accounts for the low yield obtained by gel filtration and RP-HPLC. Larger amounts (> 100 micrograms) of the purified Nef protein have been produced by a purification procedure that employs sequential detergent extraction, chromatography on Q-Sepharose in the presence of 7 M urea, and chromatography on hydroxylapatite, also in 7 M urea. The purified HIV-2 Nef protein has been used for the production of polyclonal and monoclonal antibodies. The milder method of purification should facilitate structure-function studies of the Nef protein and its role in the life cycle of HIV.

Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine.

The clinical performance of a laboratory test can be described in terms of diagnostic accuracy, or the ability to correctly classify subjects into clinically relevant subgroups. Diagnostic accuracy refers to the quality of the information provided by the classification device and should be distinguished from the usefulness, or actual practical value, of the information. Receiver-operating characteristic (ROC) plots provide a pure index of accuracy by demonstrating the limits of a test's ability to discriminate between alternative states of health over the complete spectrum of operating conditions. Furthermore, ROC plots occupy a central or unifying position in the process of assessing and using diagnostic tools. Once the plot is generated, a user can readily go on to many other activities such as performing quantitative ROC analysis and comparisons of tests, using likelihood ratio to revise the probability of disease in individual subjects, selecting decision thresholds, using logistic-regression analysis, using discriminant-function analysis, or incorporating the tool into a clinical strategy by using decision analysis.

New automated nonisotopic immunoassays for free thyroxin: effect of albumin and thyroxin-binding globulin concentrations.

Recently, nonisotopic (often automated) immunoassays for measuring serum free thyroxin (FT4) have become available. Though more costly than radioimmunoassays, they are considerably more convenient. We studied the influence of endogenous albumin and thyroxin-binding globulin concentration on five automated, nonisotopic methods of measuring FT4 [Enzymun on ES300 (one-step), Stratus I and II (essentially two-step), Delfia (two-step), and IMx (two-step)] in a mixed patient population. We observed that they (a) are influenced very little by endogenous serum binding proteins and (b) seem to have sufficient within-run precision to justify performing single measurements on patients' specimens.

ROC curve analysis: an example showing the relationships among serum lipid and apolipoprotein concentrations in identifying patients with coronary artery disease.

Clinical accuracy, defined as the ability to discriminate between states of health, is the fundamental property of any diagnostic test or system. It is readily expressed as clinical sensitivity and specificity, and elegantly represented by the receiver operating characteristic (ROC) curve. To demonstrate the use of ROC curves, we reexamine a study of the ability of serum lipid and apolipoprotein measures to discriminate among degrees of coronary artery disease in patients undergoing coronary angiography. ROC curve analysis reveals that none of these indexes is highly accurate, but demonstrates a modest increase in the accuracy of apolipoprotein over lipid indexes.

Urine free cortisol in the high-dose dexamethasone suppression test for the differential diagnosis of the Cushing syndrome.

OBJECTIVE: To develop criteria for interpreting the high-dose dexamethasone suppression test using urine free cortisol as an end point for the differential diagnosis of the Cushing syndrome. DESIGN: Retrospective review. SETTING: Inpatient research ward. PATIENTS: Patients (118) with surgically confirmed causes of the Cushing syndrome: 94 with pituitary disease, 14 with primary adrenal disease, and 10 with ectopic adrenocorticotropic hormone (ACTH) secretion. MAIN OUTCOME MEASURES: The sensitivity, specificity, and diagnostic accuracy were determined for the high-dose dexamethasone suppression test using urine free cortisol and using 17-hydroxysteroid excretion. For each analysis, patients with pituitary disease were considered to be "diseased" and patients with nonpituitary disease were considered to be "non-diseased". The level of suppression that gave 100% specificity was determined for each steroid. RESULTS: The accuracy of urine free cortisol when used as an end point in the high-dose dexamethasone suppression test was equivalent to that of 17-hydroxysteroid excretion. At all levels of sensitivity and specificity, however, the degree of suppression of urine free cortisol used for the diagnosis of pituitary disease was greater than that of 17-hydroxysteroid excretion. The likelihood ratios for pituitary disease based on urine free cortisol suppression of greater than 50%, of greater than 80%, and of greater than 90% were 4.2, 10.1, and "infinite," respectively. Suppression of urine free cortisol greater than 90% or suppression of 17-hydroxysteroid excretion greater than 64% was associated with 100% specificity. When these criteria were combined, the percentage of correct predictions (102 of 118 [86%; 95% CI, 78% to 92%]) was higher than that obtained using either steroid alone (89 of 118 [75%; CI, 65% to 83%]) (P = 0.009) and higher than that obtained using the traditional criterion of 50% suppression for 17-hydroxysteroid excretion (95 of 118 [80%; CI, 71% to 87%]) (P = 0.016). CONCLUSIONS: In the high-dose dexamethasone suppression test, the degree of suppression of urine free cortisol used for the diagnosis of pituitary disease is greater than that traditionally used for 17-hydroxysteroid excretion. The diagnostic performance of the test is improved by measuring both urine free cortisol and 17-hydroxysteroid excretion and by requiring greater suppression of both steroids.

Thyroid dysfunction associated with immunotherapy for patients with cancer.

The authors performed a prospective study to evaluate thyroid dysfunction in 130 patients with cancer who were receiving interleukin-2 (IL-2)-based immunotherapy. Primary hypothyroidism was the most common abnormality, occurring in 12% of patients before, 38% during, and 23% after immunotherapy. Hyperthyroidism occurred in 1%, 4%, and 7% of patients at those time intervals. Among patients initially euthyroid (n = 111), primary hypothyroidism developed in 32% during and 14% after immunotherapy, persisting a median of 54 days. Three patients required levothyroxine. Hyperthyroidism developed in 2% of patients during immunotherapy and 6% after. Thyroid dysfunction was not a function of sex, diagnosis, type of treatment, or response to immunotherapy. Elevated titers of antithyroglobulin and antithyroid microsomal antibodies were detected after treatment in 9% and 7%, respectively, of all patients without prior antibody abnormalities and did not correlate with response to therapy. The high incidence of therapy-induced thyroid dysfunction suggests that thyroid function should be carefully monitored in all patients receiving IL-2-based immunotherapy.

Priming with T helper cell epitope peptides enhances the antibody response to the envelope glycoprotein of HIV-1 in primates.

The induction of a memory immune response to HIV, mediated by any kind of effector mechanism, requires the induction of T cell help. In previous studies performed in different murine MHC haplotypes, three immunodominant T cell epitopes (T1, T2, and TH4.1) had been identified in the HIV envelope glycoprotein. Moreover, these peptides were proliferative T cell epitopes in humans. In this study, rhesus monkeys, Macaca mulatta, were primed with these three peptides either in combination or given separately. Half of the monkeys had a proliferative response to one or more of the priming peptide(s). Those monkeys who had a T cell proliferative response also had a high antibody response after one boost with a suboptimal dose of the native protein gp 160, whereas three of four control monkeys who had received only the native protein immunization gave no detectable antibody response, and one displayed a very weak response. For reasons that are unclear, antibodies only to the gp41 portion of gp 160 could be detected in the sera. Thus, the peptides can prime Th cells in primates for an enhanced antibody response on first exposure to the whole protein. The three peptides belong to highly conserved and nonglycosylated regions of the envelope protein. The fact that the peptides acted as immunogenic T cell proliferative and helper epitopes in nonhuman primates is very encouraging for including them in future vaccine studies in humans.

Interference by iatrogenically induced anti-mouse IgG antibodies in a two-site immunometric assay for thyrotropin.

Two-site immunometric assays using mouse monoclonal antibodies are gaining increasingly widespread popularity and use. Patients with circulating antimurine immunoglobulin antibodies capable of interfering in these assays have been encountered and described sporadically. Parenteral administration of murine monoclonal antibodies for imaging and therapeutic purposes is increasing and is known to induce human anti-murine antibodies frequently. We examined 60 serum samples from 48 individuals who received such murine immunoglobulin to determine whether iatrogenically induced anti-murine antibodies could interfere in a two-site (sandwich) immunoradiometric assay for serum thyrotropin. We found that these circulating antibodies can indeed interfere in an "unblocked" assay, but that the interference appears to be suppressed by including nonspecific IgG in the commercial version of the assay kit.

Expression of HIV-1 integrase in E. coli: immunological analysis of the recombinant protein.

Sequences encoding the human immunodeficiency virus type 1 (HIV-1) integrase gene have been cloned and expressed in Escherichia coli. The expressed protein is a lambda cII fusion protein of 37 kD containing the carboxyl-terminal 23 [corrected] amino acids of reverse transcriptase fused to the entire integrase sequence and is insoluble, a feature which allows partial purification away from soluble bacterial proteins. As judged by its reactivity with HIV positive sera in Western blot and in enzyme-linked immunosorbent assay (ELISA), the recombinant integrase retains antigenicity similar to native protein. Additionally, ELISA data obtained with the cloned protein indicate that patients infected with HIV-1 who are at different stages of progression to AIDS have antibodies reactive with the cloned integrase. HIV-2 positive human sera are also reactive with the cloned integrase. Rabbit antibodies produced against the recombinant protein react both by ELISA and Western blot with the homologous bacterially expressed protein, recognize both virion HIV-1 integrase and reverse transcriptase in Western blots, and immunoprecipitate an HIV-1 virion protein of 34 kD. Unlike human antisera from patients infected with HIV-1 or HIV-2 which are frequently reactive with both HIV-1 and HIV-2 integrase, the rabbit antibodies are type specific, reacting with HIV-1, but not with HIV-2 integrase by Western blot.

Heterogeneity of Nef proteins in cells infected with human immunodeficiency virus type 1.

Human T-lymphocytic cell line H9 infected with the HTLV-IIIB isolate of human immunodeficiency virus type 1 (HIV-1) synthesizes two forms of the Nef protein (p25 and p27) that differ both in molecular weight and charge. Different subpopulations of viruses were isolated from the HTLV-IIIB stock which induce expression of only p25 or p27. Cells infected with HIV-1 derived from the HXB3 clone of the HTLV-IIIB isolate made only the p25 species, whereas the 8E5/LAV cell line which harbors a single defective LAV provirus produces only the p27 species. These findings are consistent with the notion that the HTLV-IIIB isolate consists of at least two distinct variants with different nef genes, one specifying p25 and the other encoding p27. After a considerable number of passages in culture, H9 cells chronically infected with the HTLV-IIIB isolate produced high levels of p25 and lower levels of p27. Passages in culture appear to select for a subpopulation of virus variants that specify high levels of p25 Nef expression.

Direct and indirect techniques for free thyroxin compared in patients with nonthyroidal illness. III. Analysis of interference variables by stepwise regression.

We applied stepwise regression for multivariate analysis of data for free thyroxin (FT4) in serum and for other laboratory tests of thyroid function in patients with nonthyroidal illness. Using the maximum R2 improvement and backward elimination methods to test five variables [prealbumin, albumin, T4-binding globulin (TBG), free fatty acids (FFA), and FFA/albumin molar ratio], we found that the variables with the greatest predictive power clustered according to the methodology of FT4 measurement. Thus, we best predicted the FT4 results obtained by 16 techniques as follows: FT4 measured by one-step (analog) RIAs, with albumin; FT4 determined by two-step (sequential) RIAs, with FFA or FFA/albumin molar ratio; FT4 estimated by a binding-rate-based RIA or conceptually related FT4 indices [based on triiodothyronine (T3) uptake], with TBG; FT4 measured by equilibrium dialysis, with TBG and FFA/albumin molar ratio; and T4/TBG ratios, with either none or prealbumin and albumin. We could very highly (P less than 0.001) predict total T4 and T3 by considering TBG, and total T3 also by considering prealbumin and albumin, whereas reverse T3 was predictable with prealbumin only (negative relationship). We found comparatively weak associations between thyrotropin (TSH) and albumin or TBG. In clinical practice, abnormalities in key variables should call attention to possible effects of these variables on FT4 and other thyroid-test results and thus to the need for appropriate correction or alternative testing.