The value of Sigma-metrics in laboratory medicine.

While Six Sigma is used in different disciplines to improve quality, Tony Badric and Elvar Theodorsson in a recent paper in CCLM have questioned Six Sigma application in medical laboratory concluding Six Sigma has provided no value to medical laboratory. In addition, the authors have expanded their criticism to Total Analytical Error (TAE) model and statistical quality control. To address their arguments, we have explained the basics of TAE model and Six Sigma and have shown the value of Six Sigma to medical laboratory.

Multirule procedures vs moving average algorithms for IQC: An appropriate comparison reveals how best to combine their strengths.

BACKGROUND: Moving Average Algorithms (MAA) have been widely recommended for use in Patient Based Real Time Quality Control applications (PBRTQC) to supplement or replace traditional Internal Quality Control (IQC) techniques. A recent "proof of concept" study recommends applying MAAs to IQC data to replace traditional IQC procedures because they "outperform Westgard Rules," which is a current standard of practice for IQC. METHODS: We generated power curves for multi-rule procedures with 2 and 4 control measurements per QC event, as well as a Simple Moving Average having block sizes of 5, 10, and 20 control measurements. We also assessed time to detection in terms of the Average Number of QC Events required to detect different sizes of systematic errors. RESULTS: As expected, the more control measurements included in the control technique, the better the error detection. However, when QC performance is considered on the Sigma Scale, high Sigma methods require only 1 or 2 control measurements to detect medically important systematic errors. MAAs have very low ability to detect error at the first few QC events following shift, so they suffer a lag phase in detecting medically important errors. MAAs are most useful for methods having 4.0 Sigma performance or less. Even then, large systematic shifts are more quickly detected by simple single and multirule procedures. CONCLUSIONS: Choice of control techniques (rules, means, ranges, etc.) should consider the Sigma-metric of the method. For methods having Sigmas of 4 or greater, traditional single rule and multirule procedures with Ns up to 4 are most effective; below 4 Sigma, a multirule coupled with a Simple Moving Average (SMA) rule with Ns of 4 to 8 can improve error detection.

A multi-test planning model for risk based statistical quality control strategies.

BACKGROUND: Efforts to improve QC for multi-test analytic systems should focus on risk-based bracketed SQC strategies, as recommended in the CLSI C24-Ed4 guidance for QC practices. The objective is to limit patient risk by controlling the expected number of erroneous patient test results that would be reported over the period an error condition goes undetected. METHODS: A planning model is described to provide a structured process for considering critical variables for the development of SQC strategies for continuous production multi-test analytic systems. The model aligns with the principles of the CLSI C24-Ed4 "roadmap" and calculation of QC frequency, or run size, based on Parvin's patient risk model. Calculations are performed using an electronic spreadsheet to facilitate application of the planning model. RESULTS: Three examples of published validation data are examined to demonstrate the application of the planning model for multi-test chemistry and enzyme analyzers. The ability to assess "what if" conditions is key to identifying the changes and improvements that are necessary to simplify the overall system to a manageable number of SQC procedures. CONCLUSIONS: The planning of risk based SQC strategies should align operational requirements for workload and reporting intervals with QC frequency in terms of the run size or the number of patient samples between QC events. Computer tools that support the calculation of run sizes greatly facilitate the planning process and make it practical for medical laboratories to quickly assess the effects of critical variables.

Planning SQC strategies and adapting QC frequency for patient risk.

BACKGROUND: Risk-based Statistical QC strategies are recommended by the CLSI guidance for Statistical Quality Control (C24-Ed4). Using Parvin's patient risk model, QC frequency can be determined in terms of run size, i.e., the number of patient samples between QC events. Run size provides a practical goal for planning SQC strategies to achieve desired test reporting intervals. METHODS: A QC Frequency calculator is utilized to evaluate critical factors (quality required for test, precision and bias observed for method, rejection characteristics of SQC procedure) and also to consider patient risk as a variable for adjusting run size. RESULTS: We illustrate the planning of SQC strategies for a HbA1c test where two levels of controls show different sigma performance, for three different HbA1c analyzers used to achieve a common quality goal in a network of laboratories, and for an 18 test chemistry analyzer where a common run size is achieved by changes in control rules and adjustments for the patient risk of different tests. CONCLUSIONS: Run size provides a practical characteristic for adapting QC frequency to systematize the SQC strategies for multiple levels of controls or multiple tests in a chemistry analyzer. Patient risk can be an important variable for adapting run size to fit the laboratory's desired reporting intervals for high volume continuous production analyzers.

Mistaken assumptions drive new Six Sigma model off the road.

Oosterhuis and Coskun recently proposed a new model for applying the Six Sigma concept to laboratory measurement processes. In criticizing the conventional Six Sigma model, the authors misinterpret the industrial basis for Six Sigma and mixup the Six Sigma "counting methodology" with the "variation methodology", thus many later attributions, conclusions, and recommendations are also mistaken. Although the authors attempt to justify the new model based on industrial principles, they ignore the fundamental relationship between Six Sigma and the process capability indices. The proposed model, the Sigma Metric is calculated as the ratio CVI/CVA, where CVI is individual biological variation and CVA is the observed analytical imprecision. This new metric does not take bias into account, which is a major limitation for application to laboratory testing processes. Thus, the new model does not provide a valid assessment of method performance, nor a practical methodology for selecting or designing statistical quality control procedures.

Expected long-term defect rate of analytical performance in the medical laboratory: Assured Sigma versus observed Sigma.

Reliability of laboratory results is determined by the ratio of incorrect results expected in long-term. Sigma is a measure of defect ratio, therefore long-term Sigma is a measure of the reliability of laboratory results. Commonly, long-term Sigma is estimated based on the short-term Sigma. The Six Sigma methodology assumes that in long-term performances will shift up to 1.5 Sigma, and therefore the long-term Sigma is considered 1.5 Sigma less than short-term Sigma. Analytical performance in the medical laboratory is prone to shifts larger than 1.5 Sigma. Thus, the 1.5 Sigma shift assumed in the Six Sigma is not a correct estimate in the medical laboratory. On the other hand, in the medical laboratory statistical quality control procedure (SQC) is applied to detect and correct shifts. Since SQC can be planned to trap shifts of different sizes, the threshold set for SQC determines the defect rate expected for long-term.

Analytical Sigma metrics: A review of Six Sigma implementation tools for medical laboratories.

Sigma metrics have become a useful tool for all parts of the quality control (QC) design process. Through the allowable total error model of laboratory testing, analytical assay performance can be judged on the Six Sigma scale. This not only allows benchmarking the performance of methods and instruments on a universal scale, it allows laboratories to easily visualize performance, optimize the QC rules and numbers of control measurements they implement, and now even schedule the frequency of running those controls.

Selecting a Risk-Based SQC Procedure for a HbA1c Total QC Plan.

BACKGROUND: Recent US practice guidelines and laboratory regulations for quality control (QC) emphasize the development of QC plans and the application of risk management principles. The US Clinical Laboratory Improvement Amendments (CLIA) now includes an option to comply with QC regulations by developing an individualized QC plan (IQCP) based on a risk assessment of the total testing process. The Clinical and Laboratory Standards Institute (CLSI) has provided new practice guidelines for application of risk management to QC plans and statistical QC (SQC). METHODS: We describe an alternative approach for developing a total QC plan (TQCP) that includes a risk-based SQC procedure. CLIA compliance is maintained by analyzing at least 2 levels of controls per day. A Sigma-Metric SQC Run Size nomogram provides a graphical tool to simplify the selection of risk-based SQC procedures. APPLICATIONS: Current HbA1c method performance, as demonstrated by published method validation studies, is estimated to be 4-Sigma quality at best. Optimal SQC strategies require more QC than the CLIA minimum requirement of 2 levels per day. More complex control algorithms, more control measurements, and a bracketed mode of operation are needed to assure the intended quality of results. CONCLUSIONS: A total QC plan with a risk-based SQC procedure provides a simpler alternative to an individualized QC plan. A Sigma-Metric SQC Run Size nomogram provides a practical tool for selecting appropriate control rules, numbers of control measurements, and run size (or frequency of SQC). Applications demonstrate the need for continued improvement of analytical performance of HbA1c laboratory methods.

The use of error and uncertainty methods in the medical laboratory.

Error methods - compared with uncertainty methods - offer simpler, more intuitive and practical procedures for calculating measurement uncertainty and conducting quality assurance in laboratory medicine. However, uncertainty methods are preferred in other fields of science as reflected by the guide to the expression of uncertainty in measurement. When laboratory results are used for supporting medical diagnoses, the total uncertainty consists only partially of analytical variation. Biological variation, pre- and postanalytical variation all need to be included. Furthermore, all components of the measuring procedure need to be taken into account. Performance specifications for diagnostic tests should include the diagnostic uncertainty of the entire testing process. Uncertainty methods may be particularly useful for this purpose but have yet to show their strength in laboratory medicine. The purpose of this paper is to elucidate the pros and cons of error and uncertainty methods as groundwork for future consensus on their use in practical performance specifications. Error and uncertainty methods are complementary when evaluating measurement data.

Planning Risk-Based SQC Schedules for Bracketed Operation of Continuous Production Analyzers.

BACKGROUND: To minimize patient risk, "bracketed" statistical quality control (SQC) is recommended in the new CLSI guidelines for SQC (C24-Ed4). Bracketed SQC requires that a QC event both precedes and follows (brackets) a group of patient samples. In optimizing a QC schedule, the frequency of QC or run size becomes an important planning consideration to maintain quality and also facilitate responsive reporting of results from continuous operation of high production analytic systems. METHODS: Different plans for optimizing a bracketed SQC schedule were investigated on the basis of Parvin's model for patient risk and CLSI C24-Ed4's recommendations for establishing QC schedules. A Sigma-metric run size nomogram was used to evaluate different QC schedules for processes of different sigma performance. RESULTS: For high Sigma performance, an effective SQC approach is to employ a multistage QC procedure utilizing a "startup" design at the beginning of production and a "monitor" design periodically throughout production. Example QC schedules are illustrated for applications with measurement procedures having 6-σ, 5-σ, and 4-σ performance. CONCLUSIONS: Continuous production analyzers that demonstrate high σ performance can be effectively controlled with multistage SQC designs that employ a startup QC event followed by periodic monitoring or bracketing QC events. Such designs can be optimized to minimize the risk of harm to patients.

Selecting multi-rule quality control procedures based on patient risk.

BACKGROUND: Traditionally, statistical quality control (SQC) planning is aimed at preventing the error rate from exceeding a pre-defined acceptable rate (Westgard JO. Basic QC Practices, 4th ed. Westgard QC, 2016). A pivotal characteristic for planning a QC procedure with the traditional approach is the probability of rejecting an analytical run that contains critical size errors (Pedc). Multi-rule QC procedures, with fully documented power curves, are important tools for SQC. In addition, it has been recommended (Parvin CA, Gronowski AM. Effect of analytical run length on quality-control (QC) performance and the QC planning process. Clin Chem 1997;43:2149-54) to optimize the frequency of QC on the basis of the maximum expected increase in the number of unacceptable patient results reported during the presence of an undetected out-of-control error condition [Max E(Nuf)]. The relationship between Pedc and Max E(Nuf) has been studied for single rule QC procedures (Yago M, Alcover S. Selecting statistical procedures for quality control planning based on risk management. Clin Chem 2016;62:959-65), but corresponding information for multi-rule QC is lacking. METHODS: We used a statistical model to investigate the relationship between Pedc and Max E(Nuf) for multi-rules commonly used in clinical laboratories, and constructed charts relating the Max E(Nuf) and the sigma capability of the examination procedure for multi-rules which can be used as practical tools for planning SQC. RESULTS: There is a close relationship between Pedc and Max E(Nuf) for commonly used multi-rules. Common multi-rule SQC procedures traditionally designed for high Pedc will also provide low Max E(Nuf) values. CONCLUSIONS: Multi-rule SQC procedures can be used for controlling intermediate and low sigma capability method to attain a low Max E(Nuf) so that the probability of patient harm is mitigated to acceptable levels.

Planning Risk-Based Statistical Quality Control Strategies: Graphical Tools to Support the New Clinical and Laboratory Standards Institute C24-Ed4 Guidance.

BACKGROUND: Clinical and Laboratory Standards Institute (CLSI)'s new guideline for statistical quality control (SQC; C24-Ed4) (CLSI C24-Ed4, 2016; Parvin CA, 2017) recommends the implementation of risk-based SQC strategies. Important changes from earlier editions include alignment of principles and concepts with the general patient risk model in CLSI EP23A (CLSI EP23A, 2011) and a recommendation for optimizing the frequency of SQC (number of patients included in a run, or run size) on the basis of the expected number of unreliable final patient results. The guideline outlines a planning process for risk-based SQC strategies and describes 2 applications for examination procedures that provide 9-σ and 4-σ quality. A serious limitation is that there are no practical tools to help laboratories verify the results of these examples or perform their own applications. METHODS: Power curves that characterize the rejection characteristics of SQC procedures were used to predict the risk of erroneous patient results based on Parvin's MaxE(Nuf) parameter (Clin Chem 2008;54:2049-54). Run size was calculated from MaxE(Nuf) and related to the probability of error detection for the critical systematic error (Pedc). RESULTS: A plot of run size vs Pedc was prepared to provide a simple nomogram for estimating run size for common single-rule and multirule SQC procedures with Ns of 2 and 4. CONCLUSIONS: The "traditional" SQC selection process that uses power function graphs to select control rules and the number of control measurements can be extended to determine SQC frequency by use of a run size nomogram. Such practical tools are needed for planning risk-based SQC strategies.