External Quality Assessment of Institutions and Instruments Using a Linear Mixed Model

ABSTRACT

BACKGROUND: External quality assessment (EQA) uses a standard deviation index (SDI), based on a peer group, to evaluate laboratory performance. However, evaluations using peer group SDIs often have limited applicability, because they are not statistically valid unless the number of institutions in the same peer group is large. The present study proposes a statistical model for simultaneously evaluating the performance of all participating institutions, as well as the performance of instruments on the market. METHODS: By assuming that proficiency test results were affected by the manufacturer, the instrument, and the institution, the effects of those factors were estimated using a linear mixed model. We used these effect estimates to calculate manufacturer, instrument, and institution SDIs. Using simulation, we evaluated the false positive rates and efficiencies of the proposed linear mixed model. RESULTS: Simulations showed that the linear mixed model empirical type I error rates preserved the nominal significance level. This model was also more statistically efficient than the peer group SDI. Rates of unacceptability were lower when using institution SDI than they were when using peer group SDI. Additional outliers that could not be evaluated using the current system were detected by the institution SDI statistic. The instrument SDI statistic detected outliers among different instrument groups. CONCLUSIONS: Institution and instrument SDIs are robust and efficient tools for EQA, and they can replace the currently used system of peer group SDI.