Predicting seizure recurrence after an initial seizure-like episode from routine clinical notes using large language models: a retrospective cohort study.

BACKGROUND: The evaluation and management of first-time seizure-like events in children can be difficult because these episodes are not always directly observed and might be epileptic seizures or other conditions (seizure mimics). We aimed to evaluate whether machine learning models using real-world data could predict seizure recurrence after an initial seizure-like event. METHODS: This retrospective cohort study compared models trained and evaluated on two separate datasets between Jan 1, 2010, and Jan 1, 2020: electronic medical records (EMRs) at Boston Children's Hospital and de-identified, patient-level, administrative claims data from the IBM MarketScan research database. The study population comprised patients with an initial diagnosis of either epilepsy or convulsions before the age of 21 years, based on International Classification of Diseases, Clinical Modification (ICD-CM) codes. We compared machine learning-based predictive modelling using structured data (logistic regression and XGBoost) with emerging techniques in natural language processing by use of large language models. FINDINGS: The primary cohort comprised 14 021 patients at Boston Children's Hospital matching inclusion criteria with an initial seizure-like event and the comparison cohort comprised 15 062 patients within the IBM MarketScan research database. Seizure recurrence based on a composite expert-derived definition occurred in 57% of patients at Boston Children's Hospital and 63% of patients within IBM MarketScan. Large language models with additional domain-specific and location-specific pre-training on patients excluded from the study (F1-score 0·826 [95% CI 0·817-0·835], AUC 0·897 [95% CI 0·875-0·913]) performed best. All large language models, including the base model without additional pre-training (F1-score 0·739 [95% CI 0·738-0·741], AUROC 0·846 [95% CI 0·826-0·861]) outperformed models trained with structured data. With structured data only, XGBoost outperformed logistic regression and XGBoost models trained with the Boston Children's Hospital EMR (logistic regression: F1-score 0·650 [95% CI 0·643-0·657], AUC 0·694 [95% CI 0·685-0·705], XGBoost: F1-score 0·679 [0·676-0·683], AUC 0·725 [0·717-0·734]) performed similarly to models trained on the IBM MarketScan database (logistic regression: F1-score 0·596 [0·590-0·601], AUC 0·670 [0·664-0·675], XGBoost: F1-score 0·678 [0·668-0·687], AUC 0·710 [0·703-0·714]). INTERPRETATION: Physician's clinical notes about an initial seizure-like event include substantial signals for prediction of seizure recurrence, and additional domain-specific and location-specific pre-training can significantly improve the performance of clinical large language models, even for specialised cohorts. FUNDING: UCB, National Institute of Neurological Disorders and Stroke (US National Institutes of Health).

Towards fully automated segmentation of rat cardiac MRI by leveraging deep learning frameworks.

Automated segmentation of human cardiac magnetic resonance datasets has been steadily improving during recent years. Similar applications would be highly useful to improve and speed up the studies of cardiac function in rodents in the preclinical context. However, the transfer of such segmentation methods to the preclinical research is compounded by the limited number of datasets and lower image resolution. In this paper we present a successful application of deep architectures 3D cardiac segmentation for rats in preclinical contexts which to our knowledge has not yet been reported. We developed segmentation models that expand on the standard U-Net architecture and evaluated models separately trained for systole and diastole phases (2MSA) and a single model trained for all phases (1MSA). Furthermore, we calibrated model outputs using a Gaussian process (GP)-based prior to improve phase selection. The resulting models approach human performance in terms of left ventricular segmentation quality and ejection fraction (EF) estimation in both 1MSA and 2MSA settings (Sørensen-Dice score 0.91 ± 0.072 and 0.93 ± 0.032, respectively). 2MSA achieved a mean absolute difference between estimated and reference EF of 3.5 ± 2.5%, while 1MSA resulted in 4.1 ± 3.0%. Applying GPs to 1MSA enabled automating systole and diastole phase selection. Both segmentation approaches (1MSA and 2MSA) were statistically equivalent. Combined with a proposed cardiac phase selection strategy, our work presents an important first step towards a fully automated segmentation pipeline in the context of rat cardiac analysis.

Multimodal Continual Learning with Sonographer Eye-Tracking in Fetal Ultrasound.

Deep networks have been shown to achieve impressive accuracy for some medical image analysis tasks where large datasets and annotations are available. However, tasks involving learning over new sets of classes arriving over extended time is a different and difficult challenge due to the tendency of reduction in performance over old classes while adapting to new ones. Controlling such a 'forgetting' is vital for deployed algorithms to evolve with new arrivals of data incrementally. Usually, incremental learning approaches rely on expert knowledge in the form of manual annotations or active feedback. In this paper, we explore the role that other forms of expert knowledge might play in making deep networks in medical image analysis immune to forgetting over extended time. We introduce a novel framework for mitigation of this forgetting effect in deep networks considering the case of combining ultrasound video with point-of-gaze tracked for expert sonographers during model training. This is used along with a novel weighted distillation strategy to reduce the propagation of effects due to class imbalance.

Hierarchical Class Incremental Learning of Anatomical Structures in Fetal Echocardiography Videos.

This paper proposes an ultrasound video interpretation algorithm that enables novel classes or instances to be added over time, without significantly compromising prediction abilities on prior representations. The motivating application is diagnostic fetal echocardiography analysis. Currently in clinical practice, recording full diagnostic fetal echocardiography is not common. Diagnostic videos are typically available in varying length and summarize a number of diagnostic sub-tasks of varying difficulty. Although large clinical datasets may be available at onset to build ultrasound image-based models for automatic image analysis, data may also become available over extended time to assist in algorithm refinement. To address this scenario, we propose to use an incremental learning approach to build a hierarchical network model that allows for a parallel inclusion of previously unseen anatomical classes without requiring prior data distributions. Super classes are obtained by coarse classification followed by fine classification to allow the model to self-organize anatomical structures in a sequence of categories through a modular architecture. We show that this approach can be adapted with new variable data distributions without significantly affecting previously learned representations. Two extreme situations of new data addition are considered; (1) when new class data is available over time with volume and distribution similar to prior available classes, and (2) when imbalanced datasets arrive over future time to be learned in a few-shot setting. In either case, availability of data from prior classes is not assumed. Evolution of the learning process is validated using incremental accuracies of fine classification over novel classes and compared to results from an end-to-end transfer learning-derived model fine-tuned on a clinical dataset annotated by experienced sonographers. The modularization of subsequent learning reduces the depreciation in future accuracies over old tasks from 6.75% to 1.10% using balanced increments. The depreciation is reduced from 6.95% to 1.89% with imbalanced data distributions in future increments, while retaining competitive classification accuracies in new additions of fine classes with parameter operations in the same order of magnitude in all stages in both cases.

Efficient Ultrasound Image Analysis Models with Sonographer Gaze Assisted Distillation.

Recent automated medical image analysis methods have attained state-of-the-art performance but have relied on memory and compute-intensive deep learning models. Reducing model size without significant loss in performance metrics is crucial for time and memory-efficient automated image-based decision-making. Traditional deep learning based image analysis only uses expert knowledge in the form of manual annotations. Recently, there has been interest in introducing other forms of expert knowledge into deep learning architecture design. This is the approach considered in the paper where we propose to combine ultrasound video with point-of-gaze tracked for expert sonographers as they scan to train memory-efficient ultrasound image analysis models. Specifically we develop teacher-student knowledge transfer models for the exemplar task of frame classification for the fetal abdomen, head, and femur. The best performing memory-efficient models attain performance within 5% of conventional models that are 1000× larger in size.

Automated Myocardial Wall Motion Classification using Handcrafted Features vs a Deep CNN-based mapping.

Compared to other modalities such as computed tomography or magnetic resonance imaging, the appearance of ultrasound images is highly dependent on the expertise of the sonographer or clinician making the image acquisition, as well as the machine used, making it a challenge to analyze due to the frequent presence of artefacts, missing boundaries, attenuation, shadows, and speckle. In addition, manual contouring of the epicardial and endocardial walls exhibits large inconsistencies and variations as it is strongly dependent on the sonographer's training and expertise. Hence, in this paper we propose a fully automated image analysis framework to ultimately perform wall motion abnormality classification in 2D+T images. We explore both traditional Random Forests classification with handcrafted features and spatio-temporal hierarchical aggregation of information with a deep learning CNN-based approach. Regarding the later classifier, we also investigate the effect of local phase information retrieval through the use of Feature Asymmetry (FA), and demonstrate that pre-processing videos with FA enables the spatio-temporal CNN to better discover relevant left ventricle endocardial abstractions from low-level features to high-level representations automatically.

Improving Visual Detection of Wall Motion Abnormality with Echocardiographic Image Enhancing Methods.

Analysis of wall motion abnormality using echocardiography is an established method for detecting myocardial ischemia. We describe a hybrid approach of enhancing 2D+T echo datasets with border detection and Eulerian motion magnification to improve the visual assessment of wall motion. We implemented a local phase-based approach using the monogenic signal and its derived features, either feature asymmetry (FA) or oriented feature symmetry (OFS), to detect boundaries of the heart structure. We enhanced the 2D+T datasets using either an intensity-based or phase-based Eulerian Motion Magnification (EMM) video processing technique, and identified among eight different types of enhancements the best performing method as OFS with an accuracy of 78% versus the original B-Mode with an accuracy of 71%.