An evaluation of analytic goals for assays of drugs: a College of American Pathologists Therapeutic Drug Monitoring Survey Study.

OBJECTIVE: To determine if the levels of imprecision of the commonly used analytic methods for drug measurements are suitable for long-term therapeutic drug monitoring. DESIGN: In 1996, 4 identical lyophilized samples (2 in the first mailing and 2 in the second mailing 4 months later) were sent to laboratories participating in a nationwide proficiency testing program. Similarly, in 1999, replicates from a liquid pool of spiked sera were mailed 3 times, 4 months apart, to participating laboratories. For each of 11 drugs regulated under the Clinical Laboratory Improvement Amendments of 1988 and 1 metabolite, the total variance for each method was partitioned into within- and between-laboratory components. The total within-laboratory and the total survey coefficients of variation (CVs) for each method were then compared with the "acceptable" precision criteria of Glick, Burnett, and Fraser for each drug. SETTING: The first 2 mailings of the College of American Pathologists Therapeutic Drug Monitoring surveys for 1996, sets Z and ZM, and the 3 mailings of 1999, sets ZM, Z, and Z2. MAIN OUTCOME MEASURES: For each drug studied, the CV of each method was compared with the various imprecision criteria, and if greater than any of the criteria, the method was then tabulated as not meeting that specific criterion.Participants.-The approximately 5000 participants of the survey. RESULTS: The number of methods deemed as not having acceptable total long-term within-laboratory precision by the various criteria ranged from 35% to 88% in 1996 and from 22% to 77% in 1999. CONCLUSION: The number of failures possibly indicates that many of the reagent assays being utilized are not precise enough for long-term therapeutic drug monitoring of chronically administered drugs or that the published criteria used to evaluate the data in this study are too stringent.

Sources of variability: a College of American Pathologists Therapeutic Drug Monitoring study.

OBJECTIVE: To determine the magnitudes and sources of analytic variation in testing for therapeutic drugs. Specifically, among laboratories using the same analytic method, to compare the within-laboratory variation (including both short- and long-term variation) with the between-laboratory variation. DESIGN: Four identical challenges were prepared from a lyophilized pool of spiked sera and were sent in pairs 4 months apart to laboratories participating in a nationwide proficiency-testing program. For each of 25 drugs, the variability in reported results from laboratories using the same method was investigated using nested analysis of variance. SETTING: The first 2 mailings of the College of American Pathologists Therapeutic Drug Monitoring Survey, 1996, sets Z and ZM. MAIN OUTCOME MEASURES: For each drug, total variance was partitioned into within- and between-laboratory components for common methods. The within-laboratory component was further partitioned into short- and long-term components. PARTICIPANTS: The approximately 5000 laboratories enrolled in the survey. RESULTS: For the 25 drugs, the average percentages of the total variance due to short-term, within-laboratory variance; long-term, within-laboratory variance; between-laboratory variance; and total laboratory variance were 25.0% (range, 8.8--50.6%), 57.8% (35.3--73.7%), 17.3% (5.0--35.4%), and 82.7% (64.6--95.0%), respectively. CONCLUSION: For all drugs tested, the within-laboratory component of variance was greater than the between-laboratory component of variance. Within laboratories, the magnitude of the long-term component was generally greater than the magnitude of the short-term component. This information will be helpful in determining the clinical utility of various drug assays and in evaluating the appropriateness of regulations involving therapeutic drug testing.

Redesigned proficiency testing materials improve survey outcomes for prostate-specific antigen. A College of American Pathologists Ligand Assay Survey tool.

CONTEXT: Large disparities in prostate-specific antigen (PSA) results from different assays have been observed in the College of American Pathologists (CAP) Ligand Assay Survey, with interassay results varying severalfold. Survey specimens are predominately composed of free PSA and do not reflect the composition of typical patient specimens. OBJECTIVES: To characterize a pilot material developed for CAP in which pooled sera samples were spiked with purified PSA and alpha(1)-antichymotrypsin-bound PSA at targeted concentrations and to compare it to CAP survey and reference materials. DESIGN: CAP survey, reference, and pilot materials were analyzed using 10 total PSA and 7 free PSA assays. These assays included Food and Drug Administration-approved assays and assays for research use only. RESULTS: Variability among the 10 total PSA methods was greatest for the 1997 ligand survey material (CV range, 56%-65%) followed by the pilot material (CV range, 10%-29%) and the reference material (CV range, 6%-13%). In contrast, interassay variability for the 7 free PSA methods was similar for the 3 preparations, with the exception of one specimen close to the limit of detection of the assays. As determined with the Hybritech Tandem-R method, the ligand survey specimens were essentially composed of all free PSA, whereas the reference and pilot materials were composed of approximately 10% and 35% free PSA, respectively. CONCLUSIONS: The newly formulated pilot material prepared using a human base that contained defined concentrations of free PSA and alpha(1)-antichymotrypsin-bound PSA more closely resembled patient specimens and minimized differences among methods compared with the semen-supplemented original survey material.

Strategies to increase detection of hemochromatosis.

As part of the Iron Overload, Public Health and Genetics conference, sponsored by the Centers for Disease Control and Prevention in March 1997, a working group was convened to consider strategies to increase early case detection of hemochromatosis. This group emphasized that the primary public health goal should be to diagnose hemochromatosis before symptoms appear. To reach this goal, education and action need to be targeted to physicians and other health care workers, laboratorians, administrators, payers, and the public. Strategies to disseminate updated information and increase early case detection were prioritized according to expected effectiveness. Strategies targeting physicians are 1) to identify national and local physician-leaders and 2) to educate physicians about hemochromatosis in basic, graduate specialty, and continuing medical education. Strategies aimed at the health system are 1) to encourage laboratories to provide the transferrin saturation test as part of routine laboratory panels and 2) to work with policymakers and payers to allow reimbursement for case detection. Finally, public education is recommended to increase lay support for the early diagnosis of hemochromatosis. Attempts to educate the public should be aimed first at persons who receive diagnoses of hemochromatosis in order to ensure that they are properly treated and then at asymptomatic persons who could be screened as part of health appraisals. Although identifying physician-leaders and educating physicians are the highest priorities, physicians should not be targeted at the exclusion of payers and the public. Simultaneous efforts to reach all groups in appropriate ways should be initiated to provide the interest and infrastructure necessary to decrease morbidity and mortality from hemochromatosis.

College of American Pathologists Conference XXXI on laboratory monitoring of anticoagulant therapy: laboratory monitoring of unfractionated heparin therapy.

OBJECTIVE: To review the state of the art as reflected in the medical literature and the consensus opinion of recognized experts in the field regarding the laboratory monitoring of unfractionated heparin therapy. DATA SOURCES, EXTRACTION AND SYNTHESIS: The authors made an extensive review of the literature. The draft manuscript was circulated to every participant in the consensus conference prior to the convening of the conference. Extensive discussion concerning all of the issues addressed in the manuscript as well as the resulting recommendations occurred. This information was then used to revise the manuscript into its final form. CONCLUSIONS: The resulting manuscript has 23 specific recommendations regarding preanalytic, analytic, and postanalytic phases of monitoring and testing for complications related to unfractionated heparin therapy. This report contains detailed discussion of these recommendations and includes literature citations that support them. A number of issues for which consensus could not be reached are also discussed. A method is provided to assist laboratories, particularly small laboratories, in providing clinicians with an appropriate therapeutic range for the activated partial thromboplastin time, the most commonly used test in monitoring heparin therapy.

Errors, mistakes, blunders, outliers, or unacceptable results: how many?

We have studied 219353 individual clinical chemistry results obtained in methods comparison studies. Each result was prospectively compared with its replicate, comparative, or repeat value to identify differences from expected values. Unacceptable results were defined as differing from the expected values by < or = 7 SDs or CVs. We believe these differences represent special-cause variation and should be expressed as unacceptable rates per million results (ppm). We observed 447 ppm unacceptables: 196 ppm in control samples and 251 ppm in patients' samples. Results judged likely to alter patient care occurred at a rate of 41 ppm. To better understand the magnitude of these rates, we compared these results with reports of error rates in HIV testing and the airline industry. The measurements reported were made for the purpose of quality improvement, not judgment or discovery. The significance of these findings for laboratorians, manufacturers, and regulators is discussed.

Mild liver enzyme abnormalities: eliminating hemochromatosis as cause.

Chronic mild liver enzyme abnormalities are attributable to hereditary hemochromatosis in at least 3% of cases. Hemochromatosis formerly was diagnosed late with diabetes and hepatic and cardiac failure. Only recently have the autosomal recessive inheritance and subtle early presentations been understood. However, patients still wait many years and see many physicians before receiving a correct diagnosis. Increased serum transferrin saturation is currently the best test for detection of those likely to accumulate iron. Serum ferritin identifies those requiring treatment. When liver biopsy (controversial in asymptomatic individuals) is indicated, chemical measurement of liver iron content is helpful and therapeutic phlebotomy is the only effective treatment. Caucasian-type hemochromatosis (prevalence of 0.005) is associated with genetic abnormalities in HLA-H but also occurs in other ethnic groups. Those of African descent may have a different but also heritable iron-loading disease. Caucasian-type and to a lesser extent African iron loading are detectable early by laboratory testing. Early treatment restores normal expectations of length and quality of life in the Caucasian disease. Long-term treatment data are not yet available in African iron loading. Laboratory-initiated screening programs using unsaturated iron-binding capacity can eliminate symptomatic hemochromatosis.

The complex connections between test properties and relevant outcomes: widening the perspective.

The formulation of analytical goals for the clinical laboratory should be based upon a wide perspective embracing multiple viewpoints. Mathematical consideration of analytical error is necessary but insufficient to address societal forces demanding increased effectiveness while also reducing costs. Appropriate goals also require the study of cognitive science, health policy research, and the measurement of subjective preferences and of predictive probability. Overall goals must focus on health outcomes that emphasize prevention or postponement of morbidity and the need for acute care. Care process variables that influence health outcomes must be identified. Process improvement through reduction in process variation can then improve health outcomes. Important process variables will be identified in the cognitive process as well as in the pre- and post-analytical phases of laboratory care. The impact of test characteristics upon the laboratory's contribution to health goals are exemplified by monitoring of oral anticoagulant, thyroxine, aminoglycoside and intensive insulin therapy plus identification of clinically occult diseases such as hypothyroidism and hemochromatosis.

The effects of modifying proficiency testing materials on thyroid function test results. A College of American Pathologists Ligand Assay Survey Study.

OBJECTIVE: To gain insight on the matrix effects, and possible clinical implications, resulting from diluting and concentrating proficiency testing survey material used for the measurement of thyroid function tests. DESIGN: To the standard set of five proficiency survey samples, three supplementary "Wildcard" samples were added. These additional samples were manufactured by overfilling and underfilling vials prior to lyophilization so as to vary the thyroxine-binding protein concentrations. Survey participants measured thyroxine, free thyroxine, and the triiodothyronine uptake and related tests on the Wildcard samples. In addition, free thyroxine indices were calculated. SETTING: The first mailing of the 1995 College of American Pathologists (CAP) Ligand Assay--Series 1 Survey. MAIN OUTCOME MEASURES: Results obtained from the regular set of survey samples and the Wildcard set were compared to values expected by the laws of conservation of matter and mass action. PARTICIPANTS: The approximately 2000 participants of the first mailing of the 1995 CAP Ligand Assay--Series 1 Survey. RESULTS: Numerous assays systems did not give the predicted results, including all of the single-step radioimmunoassays for free thyroxine and over three quarters of free thyroxine index determinations. CONCLUSIONS: Varying the dilution of proficiency survey material produced results that were not predicted by the laws of conservation of matter and of mass action. Although these observations may have been the result of matrix effects, one cannot rule out the possibility that certain thyroid assays may not work in clinical situations having abnormal thyroxine-binding protein concentrations.

Practice guideline development task force of the College of American Pathologists. Hereditary hemochromatosis.

Hereditary hemochromatosis is an autosomal recessive disorder, the gene for which occurs in approximately 10% of Americans, most of whom are unaffected heterozygotes. Approximately 5/1000 white Americans are homozygous and at risk of developing severe and potentially lethal hemochromatosis. The disorder affects numerous organ systems, but the most common symptoms are fatigue, palpitations, joint pains, and impotence; the most common signs are those that relate to hypothalamic, cardiac, hepatic or pancreatic dysfunction, including poor cold tolerance, impotence in males, amenorrhea in females, cardiac arrhythmias, dyspnea, edema, hepatosplenomegaly, spider telangiectases, ascites, deformity, swelling or limitation of motion of joints, weight loss, hyperpigmentation. Characteristic abnormalities of laboratory tests include elevated serum iron concentration, high transferrin saturation, elevated serum ferritin concentration, elevated serum transaminases, hyperglycemia and low values for thyroid-stimulating hormone (TSH) and gonadotropins. Death may be the result of cardiac arrhythmia, congestive heart failure, liver failure or liver cancer. Since many of these complications cannot be reversed once they have developed, early diagnosis and treatment are essential. In view of the high prevalence in the American population (prevalence varies with ethnic background), the low cost of diagnosis and treatment, the efficacy of treatment if begun early, and, on the other hand, high costs and low success rate of late diagnosis and treatment, systematic screening for hemochromatosis is warranted for all persons over the age of 20 years. The initial screening should be by measurement of serum iron concentration and transferrin saturation. The practice guideline provides a diagnostic algorithm for cases in which the serum transferrin saturation is 60% or greater. It also provides guidelines for clinical management.

Measuring outcomes: why now?

Intense scrutiny of the American healthcare paradigm will alter the activities of patients, providers, and payors. Government reform and marketplace-driven managed care programs create uncertainty. Quality, access, and cost concerns also drive change. Quality is conformity to requirements, and specification of requirements creates policy debate. Variability in utilization creates an accountability chasm between payors and providers that cannot be bridged without understanding the uncertainties in "appropriateness" research. Quality and access appear secondary to cost. Cost discussions must differentiate cost and charge. Inappropriate charge benefit analyses may temporarily benefit a specific organization but are unlikely to create long-term societal benefit. Multiple transitions have begun, including a shift from disease care to healthcare, provider mentality to consumer mentality, and provider autonomy to collaboration and accountability. Laboratories will be expected to provide outcomes, not tests; income will be related to covered lives, not volume, and profit will shift from "piecework and efficiency profit" to "prevention profit." Only good "outcomes" measurement can reduce uncertainty. The laboratory contribution to value in care processes remains unclear. What information is added by each result? How can results help prevent the need for future services? These are our challenges.

Medically relevant laboratory-performance goals: a listing of the complexities and a call for action.

This 1992 Clinical Chemistry Forum asks if accuracy and precision goals for the laboratory can be specified by reference to medical requirements, which are a function of the total laboratory testing process as well as the needs of patients, clinicians, and societal institutions. Furthermore, medical decisions and decisions about medical requirements must be made in situations of uncertainty and thus are subject to predictable cognitive errors. The interaction of all these factors must be considered by patients, laboratorians, and clinicians to identify practical and effective performance goals. Pathologists and clinical chemists are uniquely trained to identify thoughtful clinicians who are knowledgeable in quantitative judgment to participate in this goal-setting endeavor. It is time for these parties to accumulate the available data and engage in the synthesis of effective performance goals.

Wellness testing. Design and experience of an established program.

Consumers have responded to available information and generated a large demand for self-initiated testing. Carefully planned testing can provide significant benefits to these self-selected subjects. Most notable examples include using sensitive and specific tests for iron metabolism and thyroid dysfunction. Cholesterol and lipid concerns are also motivators for testing. Nutrition and laboratory professionals must get involved with self-initiated testing to assure proper design, proper methods, proper reporting, and proper follow-up. It is hoped that this article and accompanying references will aid the implementation of effective wellness testing.

Comparison of oral cholecystography and ultrasonography for evaluating chronic cholecystitis using patient outcome as the reference standard.

RATIONALE AND OBJECTIVES: The authors compared oral cholecystography (OCG) with ultrasound (US) in the detection of chronic gallbladder disease using clinical outcome, rather than pathology results, as the reference standard. METHODS: The authors interviewed 269 patients who underwent either OCG, US, or both, for evaluation of chronic right upper quadrant abdominal pain. The authors considered patients who underwent cholecystectomy with improved symptoms 1 to 4 years after surgery to be reference-standard positive for gallbladder disease, and patients with objective evidence of an alternative diagnosis (eg, peptic ulcer disease), which improved with therapy as reference-standard negative. RESULTS: The sensitivity and specificity of OCG were 83% and 97%, respectively, and for US, 86% and 90%, respectively. CONCLUSIONS: OCG is comparable with US in evaluating of chronic gallbladder disease. In institutions where OCG is used for diagnosing chronic cholecystitis, it may be reasonable to continue using OCG.

Matrix effects in therapeutic drug monitoring surveys. Proposed protocol to identify error components and quality improvement opportunities.

Therapeutic drugs are not endogenous compounds. Therefore, differing reactivities between proficiency testing and patient samples have potential mechanisms beyond those attributable to alterations of the serum protein base, stabilizing materials, and other manufacturing components. The therapeutic drug monitoring proficiency sample is the ultimate in polypharmacy. Cross-reactivities may be uncovered that may or may not occur with any frequency in patient samples. Drugs are present in therapeutic drug monitoring proficiency samples in the absence of their metabolites, which may alter interactions with the assay systems. The CAP Therapeutic Drug Monitoring Resource Committee has contacted manufacturers when a specific method yields proficiency test results that differ from all other methods and/or the weighed-in target values. Only a few examples have been formally evaluated with the "matrix evaluation protocol." The protocol is very useful and has identified some matrix effects. A protocol is proposed using multiple samples containing survey base material. Data analysis using reciprocal plots will identify interference and corroborate calibration or recovery errors in the presence of survey base material. Data from 1987 through 1991 surveys for lithium and theophylline illustrate this data analysis and show interference from other drugs and suggest calibration errors. Weighted-in target values are the most rigorous method to provide accurate, transferable patient results and convince ourselves and our evaluators that we know what we are measuring. All of us need to be committed to improvement of laboratory procedures. I believe weighed-in target values are the appropriate and achievable goal for proficiency testing the majority of therapeutic drug monitoring analytes. Implementation of target values requires diligent and intricate collaboration among laboratorians, surveyors, manufacturers, and regulators.