ABSTRACT
OBJECTIVES: This study sought to determine whether statistical analysis of a computerized clinical diagnostic database can be used as a tool for quality assessment by determining the contribution of reader bias to variance in diagnostic output. BACKGROUND: In industry, measurement of product uniformity is a key component of quality assessment. In echocardiography, quality assessment has focused on review of small numbers of cases, or prospective determination of reader variability in selected and relatively small subsets. However, diagnostic biases in clinical practice might be discerned utilizing large computerized databases to determine interreader differences in diagnostic prevalence and, with use of appropriate statistical methods, to determine the association of reader selection with diagnostic prevalence independently of other covariates. METHODS: We analyzed 6,026 echocardiograms in a computerized database, read by one of three level 3 (American Society of Echocardiography) readers, for differences in frequency among four coded echocardiographic diagnoses: mitral valve prolapse, valvular vegetations, left ventricular (LV) thrombus, and LV regional wall-motion abnormality. RESULTS: Significant differences (up to fourfold) were found between readers, which persisted after statistical adjustment for those population characteristics, which differed slightly between readers. The low population prevalence of these conditions would have made it unlikely that these interreader differences could be detected by nonstatistical methods. Additionally, chamber dimensions differed between readers and were not normally distributed. CONCLUSIONS: Statistically based quality assessment analysis of computerized clinical databases facilitates ongoing monitoring of interreader bias despite low diagnostic prevalence, and targets opportunities for subsequent quality improvement.
