Shortcut learning in medical AI hinders generalization: method for estimating AI model generalization without external data.

Healthcare datasets are becoming larger and more complex, necessitating the development of accurate and generalizable AI models for medical applications. Unstructured datasets, including medical imaging, electrocardiograms, and natural language data, are gaining attention with advancements in deep convolutional neural networks and large language models. However, estimating the generalizability of these models to new healthcare settings without extensive validation on external data remains challenging. In experiments across 13 datasets including X-rays, CTs, ECGs, clinical discharge summaries, and lung auscultation data, our results demonstrate that model performance is frequently overestimated by up to 20% on average due to shortcut learning of hidden data acquisition biases (DAB). Shortcut learning refers to a phenomenon in which an AI model learns to solve a task based on spurious correlations present in the data as opposed to features directly related to the task itself. We propose an open source, bias-corrected external accuracy estimate, PEst, that better estimates external accuracy to within 4% on average by measuring and calibrating for DAB-induced shortcut learning.

Neural Networks for Prognostication of Patients With Heart Failure.

Background Prognostication of heart failure patients from cardiopulmonary exercise test (CPET) currently involves simplification of complex time-series data into summary indices. We hypothesized that prognostication could be improved by considering the totality of the data generated during a CPET, instead of using summary indices alone. Methods and Results Complete data from 1156 CPETs were used to predict clinical deterioration (characterized by initiation of mechanical circulatory support, listing for heart transplantation or mortality) 1 year post-CPET. We compared the prognostic value (area under the receiver operating characteristic curve) of (1) the most predictive summary indices, (2) staged data collected at discrete intervals using multivariable regression models, and (3) breath-by-breath data using a feedforward neural network. The top-performing models were compared with the commonly used CPET risk score, using absolute net reclassification index. All models were trained and assessed using a 100-iteration Monte Carlo cross-validation. A total of 190 (16.4%) patients experienced clinical deterioration. The summary indices demonstrated subpar discriminative value (area under the receiver operating characteristic curve ≤0.800). Each multivariable model outperformed the summary indices, with the neural network incorporating breath-by-breath data achieving the best performance (area under the receiver operating characteristic curve =0.842). When compared with the CPET risk score (area under the receiver operating characteristic curve =0.759), the top-performing model obtained a net reclassification index of 4.9%. Conclusions The current practice of considering summary indices in isolation fails to realize the full value of CPET data. This may lead to less accurate prognostication of patients and in consequence, inaccurate selection of patients for advanced therapy. Clinical practices, like CPET prognostication, must be continuously reevaluated to ensure optimal usage of valuable (and oft-underutilized) data sources.