Models for combining random and systematic errors. assumptions and consequences for different models.

A series of models for handling and combining systematic and random variations/errors are investigated in order to characterize the different models according to their purpose, their application, and discuss their flaws with regard to their assumptions. The following models are considered 1. linear model, where the random and systematic elements are combined according to a linear concept (TE = absolute value(bias) + z x sigma), where TE is total error, bias is the systematic error component, sigma is the random error component (standard deviation or coefficient of variation) and z is the probability factor; 2. squared model with two sub-models of which one is the classical statistical variance model and the other is the GUM (Guide to Uncertainty in Measurements) model for estimating uncertainty of a measurement; 3. combined model developed for the estimation of analytical quality specifications according to the clinical consequences (clinical outcome) of errors. The consequences of these models are investigated by calculation of the functions of transformation of bias into imprecision according to the assumptions and model calculations. As expected, the functions turn out to be rather different with considerable consequences for these types of transformations. It is concluded that there are at least three models for combining systematic and random variation/errors, each created for its own specific purpose, with its own assumptions and resulting in considerably different results. These models should be used according to their purposes.

Electronic quality control, the total testing process, and the total quality control system.

Traditional statistical quality control (QC) using matrix controls is often difficult to implement in point-of-care settings. Alternative QC procedures, such as electronic QC, have been developed by many manufacturers and approved for use by regulatory and accreditation organizations. Electronic QC usually involves the substitution of an electrical signal for the signal that would normally be generated by a sensor responding to an analyte in a specimen; sometimes an artificial nonliquid sample is substituted to cause the sensor to generate an electrical signal. The usefulness of electronic QC can be assessed by identifying the steps in the total testing process that are being monitored. An example is provided for blood gas measurements to illustrate the steps that can be monitored by different types of QC procedures and materials. The need to monitor all of the steps generally requires a combination of procedures and materials, or a total QC system that includes electronic QC, matrix controls, and even real patient specimens. Electronic QC is an essential part of the total QC system, but is not by itself sufficient.

The need for a system of quality standards for modern quality management.

The management of analytical quality depends on the careful evaluation of the imprecision (uncertainty) and inaccuracy (trueness) of laboratory methods and the application of statistical quality control procedures to detect medically important analytical errors that may occur during routine analysis. A system of quality standards is recommended to incorporate different types of requirements, such as clinical outcome criteria, analytical outcome criteria, and analytical performance criteria. For practical applications, all need to be translated into operating specifications for the imprecision, inaccuracy, control rules, and number of control measurements that are necessary to assure analytical quality during routine production of test results.

Comparison of NCEP performance specifications for triglycerides, HDL-, and LDL-cholesterol with operating specifications based on NCEP clinical and analytical goals.

The National Cholesterol Education Program (NCEP) performance specifications for methods that measure triglycerides, HDL-cholesterol, and LDL-cholesterol have been evaluated by deriving operating specifications from the NCEP analytical total error requirements and the clinical requirements for interpretation of the tests. We determined the maximum imprecision and inaccuracy that would be allowable to control routine methods with commonly used single and multirule quality-control procedures having 2 and 4 control measurements per run, and then compared these estimates with the NCEP guidelines. The NCEP imprecision specifications meet the operating imprecision necessary to assure meeting the NCEP clinical quality requirements for triglycerides and HDL-cholesterol but not for LDL-cholesterol. More importantly, the NCEP imprecision specifications are not adequate to assure meeting the NCEP analytical total error requirements for any of these three tests. Our findings indicate that the NCEP recommendations fail to adequately consider the quality-control requirements necessary to detect medically important systematic errors.

Quality control for qualitative assays: quantitative QC procedure designed to assure analytical quality required for an ELISA of hepatitis B surface antigen.

An assay for hepatitis B surface antigen (HBsAg) should reliably detect 0.2 microgram/L, the lowest reported concentration in an asymptomatic blood donor. The difference between this concentration and the assay cutoff defines the analytical quality requirement in a total error format. The design of a statistical QC procedure is critically dependent on the precision of the assay. The precision of a developmental ELISA of HBsAg under study ranged from 17.5% to 9.6% for controls containing 0.07 to 1.50 micrograms/L, respectively. Use of one positive control with the 1(3s), QC rule provided an 85% chance of detecting a critical loss of assay sensitivity; use of two positive controls increased the chance of detecting critical loss of assay sensitivity to nearly 100%. These rules are based on the precision of this developmental assay, and must be developed individually for other assays. The development of the proposed QC procedures illustrates how quantitative QC can be provided for qualitative assays.

QC Validator 2.0: a computer program for automatic selection of statistical QC procedures for applications in healthcare laboratories.

A computer program has been developed to help healthcare laboratories select statistical control rules and numbers of control measurements that will assure the quality required by clinical decision interval criteria or analytical total error criteria. The program (QC Validator 2.0 (QC Validator and OPSpecs are registered trademarks of Westgard Quality Corporation, which has applied for a patent for this automatic QC selection process. Windows is a registered trademark of Microsoft Corporation)) runs on IBM compatible personal computers operating under Windows. The user enters information about the method imprecision, inaccuracy, and expected frequency of errors, defines the quality required in terms of a medically important change (clinical decision interval) or an analytical allowable total error, then initiates automatic selection by indicating the number of control materials that are to be analyzed (1, 2, or 3). The program returns with a chart of operating specifications (OPSpecs chart) that displays the selected control rules and numbers of control measurements. The automatic QC selection process is based on user editable criteria for the types of control rules that can be implemented by the laboratory, total numbers of control measurements that are practical, maximum levels of false rejections that can be tolerated and minimum levels of error detection that are acceptable for detection of medically important systematic or random errors.

Design and assessment of average of normals (AON) patient data algorithms to maximize run lengths for automatic process control.

Achieving high quality and high productivity with automated testing processes will require process control systems that are optimized for the necessary error detection, minimum false rejection, and maximum run length. This study investigates whether run length could be monitored by average of normals (AON) algorithms that truncate the patient test distribution and estimate the average of a suitable number of patient results. The design of AON algorithms for individual analytes is facilitated by computer-simulated power curves that consider the ratio of the population biological variation (Spop) to the test method variation (Smeas), represent a range of Spop/Smeas ratios from 2 to 15, and include numbers of patient test results from 10 to 600. The potential applications of AON algorithms are assessed for 38 tests whose quality requirements represent the total error criteria from the Ontario Medical Association Laboratory Proficiency Testing Program, Spop/Smeas ratios from 0 to 32, critical systematic shifts from 0.02 to 10.85 Smeas, and test workloads representative of a regional reference laboratory. Approximately half of these tests provide high potential for applying AON algorithms to monitor run length.

Laboratory precision performance: state of the art versus operating specifications that assure the analytical quality required by clinical laboratory improvement amendments proficiency testing.

OBJECTIVE: To estimate the percent of laboratories with precision performance sufficient to satisfy the operating specifications and guarantee the quality required by the proficiency testing criteria defined by the Clinical Laboratory Improvement Amendments of 1988 (CLIA). DESIGN: Cumulative distributions that describe state-of-the-art laboratory imprecision were obtained for 1500 laboratories participating in the 1990 College of American Pathologists Quality Assurance Service. Allowable imprecision was estimated from the x-intercepts of charts of operating specifications prepared for commonly used single and multirule quality control procedures having two to four control measurements per run. MAIN OUTCOME MEASURE: The derived values for allowable imprecision were imposed on the cumulative distributions to obtain graphical estimates of the percent of laboratories satisfying the operating specifications. RESULTS: Up to 28% of laboratories achieve the imprecision allowable for albumin, up to 64% for total bilirubin, 52% for calcium, 35% for chloride, 48% for cholesterol, 28% for cortisol, 84% for creatinine, 9% for digoxin, 61% for glucose, 64% for high-density lipoprotein cholesterol, 88% for hemoglobin, 95% for potassium, 66% for total protein, 18% for sodium, 29% for thyroxine, 87% for triglycerides, 35% for urea nitrogen, and 81% for uric acid. CONCLUSION: Improvements in precision are still needed for many laboratory tests to assure the analytical quality required by the CLIA proficiency testing total error criteria.

Error budgets for quality management--practical tools for planning and assuring the analytical quality of laboratory testing processes.

Analytical quality is often assumed, rather than being assured or guaranteed. Given that it is still essential that laboratories produce reliable test results, managers must continue to improve their skills in analytical quality management. This paper shows managers how to use error budgets and charts of operating specifications (¿OPSpecs¿ charts) to select appropriate control rules and numbers of control measurements, taking into account the analytical or clinical quality required for a test and the imprecision and inaccuracy observed for a method. With currently available tools and a little practice, quality control (QC) procedures can be selected quickly and easily, in just 1 minute or less. Future technology is expected to automate the QC selection process and provide dynamic quality control.

Matrix effects and the performance and selection of quality-control procedures to monitor PO2 measurements.

We have assessed how variation in the matrix of control materials would affect error detection and false-rejection characteristics of quality-control (QC) procedures used to monitor PO2 in blood gas measurements. To determine the expected QC performance, we generated power curves for S(mat)/S(meas) ratios of 0.0-4.0. These curves were used to estimate the probabilities of rejecting analytical runs having medically important errors, calculated from the quality required by the CLIA '88 proficiency testing criterion and the precision and accuracy expected for a typical analytical system. When S(mat)/S(meas) ratios are low, the effects of matrix on QC performance are not serious, permitting selections of QC procedures based on simple power curves for a single component of variation. As S(mat)/S(meas) ratios increase, single-rule procedures generally show a loss in error detection, whereas multirule procedures, including the 3(1)s control rule, show an increase in false rejections. An optimized QC design is presented.

A method evaluation decision chart (MEDx chart) for judging method performance.

OBJECTIVE: To introduce a new graphical tool that improves the process of making decisions about method performance. DATA SOURCES: Scientific literature, mathematical models, and the author's experience. STUDY SELECTION: Not applicable. DATA EXTRACTION: Not applicable. DATA SYNTHESIS: Relationship to charts of operating specifications (OPSpecs charts) provides guidance for classification of performance as poor, marginal, good, or excellent. CONCLUSION: Objective decisions on the performance of methods can be made quickly and easily with the aid of a Method Evaluation Decision Chart.

Allowable imprecision for laboratory tests based on clinical and analytical test outcome criteria.

The allowable imprecision for laboratory tests has been estimated from criteria based on clinical and analytical test outcome. The analytical outcome criteria studied are the Clinical Laboratory Improvement Amendments (CLIA) criteria for proficiency testing. The clinical outcome criteria are estimates of medically significant changes in test results taken from a study in the literature. The estimates of allowable imprecision were obtained from quality-planning models that relate test outcome criteria to the allowable amount of imprecision and inaccuracy and to the quality control that is necessary to assure achievement of the desired outcome criteria in routine operation. These operating specifications for imprecision are consistently more demanding (require lower CVs) than the medically useful CVs originally recommended in the literature because the latter do not properly consider within-subject biological variation. In comparing estimates of allowable imprecision, the CLIA outcome criteria are more demanding than the clinical outcome criteria for aspartate aminotransferase (asymptomatic patients), cholesterol, creatinine (asymptomatic patients), glucose, thyroxine, total protein, urea nitrogen, hematocrit, and prothrombin time. The clinical outcome criteria are more demanding for bilirubin (acute illness), iron, potassium, urea nitrogen (acute illness), and leukocyte count. The estimates of allowable imprecision from analytical and clinical outcome criteria overlap for aspartate aminotransferase (acute illness), bilirubin (asymptomatic patients), calcium, creatinine (acute illness), sodium, triglyceride, and hemoglobin.

European specifications for imprecision and inaccuracy compared with operating specifications that assure the quality required by US CLIA proficiency-testing criteria.

Proposed and interim European quality specifications for imprecision and inaccuracy have been compared with the US CLIA total error criteria for proficiency testing (PT). To assess the relative demands of separate imprecision and inaccuracy specifications vs total error criteria, we derived the imprecision and inaccuracy that would be allowable if a testing process were to provide 90% assurance of achieving the analytical quality required by CLIA PT criteria. Charts of operating specifications (OPSpecs charts) were prepared for commonly used single-rule and multi-rule quality control procedures with 2 and 4 control measurements per run. Of the 23 tests studied, the proposed European specifications for imprecision and inaccuracy were more demanding than the CLIA requirements for 12 tests (albumin, alkaline phosphatase, amylase, calcium, chloride, creatinine, lactate dehydrogenase, lithium, magnesium, total protein, sodium, and thyroxine). The CLIA total error criteria were more demanding than the proposed European specifications for nine tests (alanine aminotransferase, aspartate aminotransferase, total bilirubin, cholesterol, creatine kinase, iron, triglycerides, uric acid, and urea nitrogen). Two tests (glucose, potassium) showed different requirements at different decision levels. Manufacturers and laboratory analysts need to compare these different quality specifications on a test-by-test basis to guide the development, selection, evaluation, and control of laboratory measurement procedures.

Cholesterol--a model system to relate medical needs with analytical performance.

The evolution of cholesterol testing provides an example of a systematic approach that developed to relate the medical use of a laboratory test with the analytical performance requirements for that test. Laboratories today have the capability to perform cholesterol testing with the accuracy and precision necessary to meet medical needs. This statement can be made because (a) a standard diagnostic process has been established by the National Cholesterol Education Program; (b) an accuracy base is provided through a reference method that is readily available to manufacturers and laboratories; (c) the precision of analytical systems has been improved by manufacturers; (d) operating specifications for all such systems can be established, with statistical quality-control rules to ensure adequate within-run method performance; and (e) analytical performance is monitored by proficiency testing by using national quality requirements defined by CLIA '88 for acceptability. This cholesterol model provides a logical and scientific approach that should be applicable with other analytes to assure that the analytical performance of the laboratory test satisfies medical needs.

Analytical goal-setting for monitoring patients when two analytical methods are used.

Serial results from an individual are often obtained using more than one method. Results should be transferable over time and locale. Every method has inherent analytical error, and goals are required to delineate the maximum allowable random (imprecision) and systematic (inaccuracy, bias) errors to facilitate optimal patient care. Based on Harris's proposal [Am J Clin Pathol 1979;72:374-82] that desirable imprecision should be less than or equal to one-half the within-subject biological variation, if the methods have negligible imprecision, then the maximum allowable bias between two methods used for monitoring is one-third of the within-subject biological variation. A more general model has been developed that relates the analytical imprecisions of two methods, and the bias between them, to biological variation. Applying the general formula derived in specific clinical monitoring situations in which a known change in serial results (occurring at a stated probability) stimulates clinical action allows goals for the imprecisions of the two methods and allows the difference in bias between them to be determined quantitatively.