AI do it better. Reproducibility of fully automated versus manual analysis of left ventricle function using a new software

ABSTRACT

Background/Introduction: Ejection fraction (EF) is a parameter widely used in Echolab to evaluate left ventricular function. Recently, in parallel with the growing interest in artificial intelligence (AI), attemps have been made to create automated systems for EF assessment, in order to reduce time and improve the accuracy of the analysis. Purpose: to compare results of different methods of EF assessment visual estimation (visual EF), manual and fully automated analysis. Methods: 28 consecutive pediatric patients were enrolled. This cohort of previously healthy patients was screened at our Center for cardiac evaluation within 6 months after an asymptomatic or paucisymptomatic COVID19 infection. All they were in sinus rhythm. Optimized apical 4- and 2- chamber views were collected for each patient using Canon Aplio i900. Off-line EF assessment was first evaluated visually by pediatric cardiologists with experience in echocardiography, then performed by both fully automated analysis (AI) using two different methods (Automatic Simpson -AI Simpson- and Wall Motion Tracking -AI WMT-) and pediatric cardiologists through manual tracing of endocardial border (Manual Simpson and Manual WMT respectively). Operators were blinded to the AI analysis. To measure intraobserver variability, evaluations of 16 patients' datasets were performed twice by both operators and AI. Results: Patients' demographic data were age 9,8+/-4,7 years;males 22 (78%);height 134,3+/- 34,9 cm;weight 41,8+/-28,7 kg;BSA 1,2+/-0,4 mq, HR 85+/-15/min. The time taken for off-line analysis was 0.3-0.7 minutes, 1-1.5 minutes, 1-3 minutes and 3-4 minutes, respectively for AI WMT, AI Simpson, Manual WMT and Manual Simpson. As expected, visual EF showed high intraobserver variability and a poor reproducibility (ICC 43%). AI analysis revealed a good to excellent reproducibility (ICC from 80% to 99%, depending on the method used). WMT methods had the best reproducibility both for manual tracing of endocardial border and fully automated analysis (Table 1). The comparison between different methods (Table 2) showed a good agreement between AI Simpson and AI WMT (mean bias 2,9, from -3,2 to 9,0, ICC 86%). A moderate correlation was found between different methods of AI analysis while only poor correlation was found between manual Simpson and manual WMT (Table 2). Conclusion(s) Automatic Simpson and Wall Motion Tracking are two different fully automated methods which can be used for left ventricular function assessment. AI reproducibility is high for both methods, higher for WMT. WMT method is also less time consuming and improves reproducibility of manual tracing of endocardial borderd analysis.