Bridging AI and Clinical Practice: Integrating Automated Sleep Scoring Algorithm with Uncertainty-Guided Physician Review.

Purpose: This study aims to enhance the clinical use of automated sleep-scoring algorithms by incorporating an uncertainty estimation approach to efficiently assist clinicians in the manual review of predicted hypnograms, a necessity due to the notable inter-scorer variability inherent in polysomnography (PSG) databases. Our efforts target the extent of review required to achieve predefined agreement levels, examining both in-domain (ID) and out-of-domain (OOD) data, and considering subjects' diagnoses. Patients and Methods: A total of 19,578 PSGs from 13 open-access databases were used to train U-Sleep, a state-of-the-art sleep-scoring algorithm. We leveraged a comprehensive clinical database of an additional 8832 PSGs, covering a full spectrum of ages (0-91 years) and sleep-disorders, to refine the U-Sleep, and to evaluate different uncertainty-quantification approaches, including our novel confidence network. The ID data consisted of PSGs scored by over 50 physicians, and the two OOD sets comprised recordings each scored by a unique senior physician. Results: U-Sleep demonstrated robust performance, with Cohen's kappa (K) at 76.2% on ID and 73.8-78.8% on OOD data. The confidence network excelled at identifying uncertain predictions, achieving AUROC scores of 85.7% on ID and 82.5-85.6% on OOD data. Independently of sleep-disorder status, statistical evaluations revealed significant differences in confidence scores between aligning vs discording predictions, and significant correlations of confidence scores with classification performance metrics. To achieve κ ≥ 90% with physician intervention, examining less than 29.0% of uncertain epochs was required, substantially reducing physicians' workload, and facilitating near-perfect agreement. Conclusion: Inter-scorer variability limits the accuracy of the scoring algorithms to ~80%. By integrating an uncertainty estimation with U-Sleep, we enhance the review of predicted hypnograms, to align with the scoring taste of a responsible physician. Validated across ID and OOD data and various sleep-disorders, our approach offers a strategy to boost automated scoring tools' usability in clinical settings.

Deep Generative Models: The winning key for large and easily accessible ECG datasets?

Large high-quality datasets are essential for building powerful artificial intelligence (AI) algorithms capable of supporting advancement in cardiac clinical research. However, researchers working with electrocardiogram (ECG) signals struggle to get access and/or to build one. The aim of the present work is to shed light on a potential solution to address the lack of large and easily accessible ECG datasets. Firstly, the main causes of such a lack are identified and examined. Afterward, the potentials and limitations of cardiac data generation via deep generative models (DGMs) are deeply analyzed. These very promising algorithms have been found capable not only of generating large quantities of ECG signals but also of supporting data anonymization processes, to simplify data sharing while respecting patients' privacy. Their application could help research progress and cooperation in the name of open science. However several aspects, such as a standardized synthetic data quality evaluation and algorithm stability, need to be further explored.

U-Sleep's resilience to AASM guidelines.

AASM guidelines are the result of decades of efforts aiming at standardizing sleep scoring procedure, with the final goal of sharing a worldwide common methodology. The guidelines cover several aspects from the technical/digital specifications, e.g., recommended EEG derivations, to detailed sleep scoring rules accordingly to age. Automated sleep scoring systems have always largely exploited the standards as fundamental guidelines. In this context, deep learning has demonstrated better performance compared to classical machine learning. Our present work shows that a deep learning-based sleep scoring algorithm may not need to fully exploit the clinical knowledge or to strictly adhere to the AASM guidelines. Specifically, we demonstrate that U-Sleep, a state-of-the-art sleep scoring algorithm, can be strong enough to solve the scoring task even using clinically non-recommended or non-conventional derivations, and with no need to exploit information about the chronological age of the subjects. We finally strengthen a well-known finding that using data from multiple data centers always results in a better performing model compared with training on a single cohort. Indeed, we show that this latter statement is still valid even by increasing the size and the heterogeneity of the single data cohort. In all our experiments we used 28528 polysomnography studies from 13 different clinical studies.

Multi-scored sleep databases: how to exploit the multiple-labels in automated sleep scoring.

STUDY OBJECTIVES: Inter-scorer variability in scoring polysomnograms is a well-known problem. Most of the existing automated sleep scoring systems are trained using labels annotated by a single-scorer, whose subjective evaluation is transferred to the model. When annotations from two or more scorers are available, the scoring models are usually trained on the scorer consensus. The averaged scorer's subjectivity is transferred into the model, losing information about the internal variability among different scorers. In this study, we aim to insert the multiple-knowledge of the different physicians into the training procedure. The goal is to optimize a model training, exploiting the full information that can be extracted from the consensus of a group of scorers. METHODS: We train two lightweight deep learning-based models on three different multi-scored databases. We exploit the label smoothing technique together with a soft-consensus (LSSC) distribution to insert the multiple-knowledge in the training procedure of the model. We introduce the averaged cosine similarity metric (ACS) to quantify the similarity between the hypnodensity-graph generated by the models with-LSSC and the hypnodensity-graph generated by the scorer consensus. RESULTS: The performance of the models improves on all the databases when we train the models with our LSSC. We found an increase in ACS (up to 6.4%) between the hypnodensity-graph generated by the models trained with-LSSC and the hypnodensity-graph generated by the consensus. CONCLUSION: Our approach definitely enables a model to better adapt to the consensus of the group of scorers. Future work will focus on further investigations on different scoring architectures and hopefully large-scale-heterogeneous multi-scored datasets.

Postoperative Epileptic Seizures in Children.

Background: Postoperative seizures (PS) occur in 10−15% of patients. This study aims to provide an update on the role of surgery in PS. Methods: All children undergoing a craniotomy for supratentorial lesions in the last 10 years were considered except those with preoperative seizures, perioperative antiepileptic drugs prophylaxis, head-injury and infections, repeated surgery, or preoperative hyponatremia. Children undergoing surgery for intra-axial lesions (Group 1, 74 cases) were compared with those harboring extra-axial lesions (Group 2, 91 cases). Results: PS occurred in 9% of 165 cases and epilepsy in 3% of 165 cases (mean follow-up: 5.7 years). There was no difference between the two study groups with regard to demographic data or tumor size. Group 1 showed a higher rate of gross total tumor resection (p = 0.002), while Group 2 had a higher rate of postoperative hyponatremia (p < 0.0001). There were no differences between the two groups in the occurrence of seizures (6.7% vs. 11%) or epilepsy (2.7% vs. 3.2%). No correlations were found between seizures and age, tumor location, histotype, tumor size, or the extent of tumor resection. Hyponatremia affected the risk of PS in Group 2 (p = 0.02). Conclusions: This study shows a lower rate of PS and epilepsy than series including children with preoperative seizures. Hyponatremia has a significant role. Neurosurgery is safe but surgical complications may cause late epilepsy.

Wearables in the home-based assessment of abnormal movements in Parkinson's disease: a systematic review of the literature.

At present, the standard practices for home-based assessments of abnormal movements in Parkinson's disease (PD) are based either on subjective tools or on objective measures that often fail to capture day-to-day fluctuations and long-term information in real-life conditions in a way that patient's compliance and privacy are secured. The employment of wearable technologies in PD represents a great paradigm shift in healthcare remote diagnostics and therapeutics monitoring. However, their applicability in everyday clinical practice seems to be still limited. We carried out a systematic search across the Medline Database. In total, 246 publications, published until 1 June 2020, were identified. Among them, 26 reports met the inclusion criteria and were included in the present review. We focused more on clinically relevant aspects of wearables' application including feasibility and efficacy of the assessment, the number, type and body position of the wearable devices, type of PD motor symptom, environment and duration of assessments and validation methodology. The aim of this review is to provide a systematic overview of the current knowledge and state-of-the-art of the home-based assessment of motor symptoms and fluctuations in PD patients using wearable technology, highlighting current problems and laying foundations for future works.

DeepSleepNet-Lite: A Simplified Automatic Sleep Stage Scoring Model With Uncertainty Estimates.

Deep learning is widely used in the most recent automatic sleep scoring algorithms. Its popularity stems from its excellent performance and from its ability to process raw signals and to learn feature directly from the data. Most of the existing scoring algorithms exploit very computationally demanding architectures, due to their high number of training parameters, and process lengthy time sequences in input (up to 12 minutes). Only few of these architectures provide an estimate of the model uncertainty. In this study we propose DeepSleepNet-Lite, a simplified and lightweight scoring architecture, processing only 90-seconds EEG input sequences. We exploit, for the first time in sleep scoring, the Monte Carlo dropout technique to enhance the performance of the architecture and to also detect the uncertain instances. The evaluation is performed on a single-channel EEG Fpz-Cz from the open source Sleep-EDF expanded database. DeepSleepNet-Lite achieves slightly lower performance, if not on par, compared to the existing state-of-the-art architectures, in overall accuracy, macro F1-score and Cohen's kappa (on Sleep-EDF v1-2013 ±30mins: 84.0%, 78.0%, 0.78; on Sleep-EDF v2-2018 ±30mins: 80.3%, 75.2%, 0.73). Monte Carlo dropout enables the estimate of the uncertain predictions. By rejecting the uncertain instances, the model achieves higher performance on both versions of the database (on Sleep-EDF v1-2013 ±30mins: 86.1.0%, 79.6%, 0.81; on Sleep-EDF v2-2018 ±30mins: 82.3%, 76.7%, 0.76). Our lighter sleep scoring approach paves the way to the application of scoring algorithms for sleep analysis in real-time.

Personalized Sleep Spindle Detection in Whole Night Polysomnography.

The present study proposes a new personalized sleep spindle detection algorithm, suggesting the importance of an individualized approach. We identify an optimal set of features that characterize the spindle and exploit a support vector machine to distinguish between spindle and nonspindle patterns. The algorithm is assessed on the open source DREAMS database, that contains only selected part of the polysomnography, and on whole night polysomnography recordings from the SPASH database. We show that on the former database the personalization can boost sensitivity, from 84.2% to 89.8%, with a slight increase in specificity, from 97.6% to 98.1%. On a whole night polysomnography instead, the algorithm reaches a sensitivity of 98.6% and a specificity of 98.1%, thanks to the personalization approach. Future work will address the integration of the spindle detection algorithm within a sleep scoring automated procedure.

Temporal dependency in automatic sleep scoring via deep learning based architectures: An empirical study.

The present study evaluates how effectively a deep learning based sleep scoring system does encode the temporal dependency from raw polysomnography signals. An exhaustive range of neural networks, including state of the art architecture, have been used in the evaluation. The architectures have been assessed using a single-channel EEG Fpz-Cz from the open source Sleep-EDF expanded database. The best performing model reached an overall accuracy of 85.2% and a Cohen's kappa of 0.8, with an F1-score of stage N1 equal to 50.2%. We have introduced a new metric, δnorm, to better evaluate temporal dependencies. A simple feed forward architecture not only achieves comparable performance to most up-to-date complex architectures, but also does better encode the continuous temporal characteristics of sleep.Clinical relevance - A better understanding of the capability of the network in encoding sleep temporal patterns could lead to improve the automatic sleep scoring.

Automated sleep scoring: A review of the latest approaches.

Clinical sleep scoring involves a tedious visual review of overnight polysomnograms by a human expert, according to official standards. It could appear then a suitable task for modern artificial intelligence algorithms. Indeed, machine learning algorithms have been applied to sleep scoring for many years. As a result, several software products offer nowadays automated or semi-automated scoring services. However, the vast majority of the sleep physicians do not use them. Very recently, thanks to the increased computational power, deep learning has also been employed with promising results. Machine learning algorithms can undoubtedly reach a high accuracy in specific situations, but there are many difficulties in their introduction in the daily routine. In this review, the latest approaches that are applying deep learning for facilitating and accelerating sleep scoring are thoroughly analyzed and compared with the state of the art methods. Then the obstacles in introducing automated sleep scoring in the clinical practice are examined. Deep learning algorithm capabilities of learning from a highly heterogeneous dataset, in terms both of human data and of scorers, are very promising and should be further investigated.