ABSTRACT
BACKGROUND: The effectiveness trial "Stress echo (SE) 2020" evaluates novel applications of SE in and beyond coronary artery disease. The core protocol also includes 4-site simplified scan of B-lines by lung ultrasound, useful to assess pulmonary congestion. PURPOSE: To provide web-based upstream quality control and harmonization of B-lines reading criteria. METHODS: 60 readers (all previously accredited for regional wall motion, 53 B-lines naive) from 52 centers of 16 countries of SE 2020 network read a set of 20 lung ultrasound video-clips selected by the Pisa lab serving as reference standard, after taking an obligatory web-based learning 2-h module ( http://se2020.altervista.org ). Each test clip was scored for B-lines from 0 (black lung, A-lines, no B-lines) to 10 (white lung, coalescing B-lines). The diagnostic gold standard was the concordant assessment of two experienced readers of the Pisa lab. The answer of the reader was considered correct if concordant with reference standard reading ±1 (for instance, reference standard reading of 5 B-lines; correct answer 4, 5, or 6). The a priori determined pass threshold was 18/20 (≥ 90%) with R value (intra-class correlation coefficient) between reference standard and recruiting center) > 0.90. Inter-observer agreement was assessed with intra-class correlation coefficient statistics. RESULTS: All 60 readers were successfully accredited: 26 (43%) on first, 24 (40%) on second, and 10 (17%) on third attempt. The average diagnostic accuracy of the 60 accredited readers was 95%, with R value of 0.95 compared to reference standard reading. The 53 B-lines naive scored similarly to the 7 B-lines expert on first attempt (90 versus 95%, p = NS). Compared to the step-1 of quality control for regional wall motion abnormalities, the mean reading time per attempt was shorter (17 ± 3 vs 29 ± 12 min, p < .01), the first attempt success rate was higher (43 vs 28%, p < 0.01), and the drop-out of readers smaller (0 vs 28%, p < .01). CONCLUSIONS: Web-based learning is highly effective for teaching and harmonizing B-lines reading. Echocardiographers without previous experience with B-lines learn quickly.
Subject(s)
Echocardiography, Stress/standards , Lung/diagnostic imaging , Pulmonary Edema/diagnosis , Quality Control , Female , Humans , Internet , Male , Middle AgedABSTRACT
BACKGROUND: The trial "Stress Echo (SE) 2020" evaluates novel applications of SE beyond coronary artery disease. The aim of the study was control quality and harmonize reading criteria. METHODS: One reader from 78 centers of the SE 2020 network asked for credentials to read a set of 20 SE video-clips selected by the core lab. All aspiring centers met the pre-requisite of high-volume and the years of experience in SE ranged from 5 to 31years (mean value 18years). The diagnostic gold standard was a reading by the core lab. The a priori determined pass threshold was 18/20 (≥90%). RESULTS: Of the initial 78 who started, 57 completed the first attempt: individual readers' score on first attempt ranged from 07/20 to 20/20 (accuracy from 35% to 100%, mean 78.7±13%) and 44 readers passed it. There was a very poor correlation between years of experience and the reader's score on first attempt (r=-0.161, p=0.231). Of the 13 readers who failed the first attempt, 12 took it again after the web-based session and their accuracy improved (74% vs. 96%, p<0.001). The kappa inter-observer agreement before and after web-based training was 0.59 on first attempt and rose to 0.91 on the last attempt. CONCLUSIONS: In SE reading, the volume of activity or years of experience is not synonymous with diagnostic quality. Qualitative analysis and operator-dependence can become a limiting weakness in clinical practice, in the absence of strict pathways of learning, credentialing and audit.
