L-SeqSleepNet: Whole-cycle Long Sequence Modeling for Automatic Sleep Staging.

Human sleep is cyclical with a period of approximately 90 minutes, implying long temporal dependency in the sleep data. Yet, exploring this long-term dependency when developing sleep staging models has remained untouched. In this work, we show that while encoding the logic of a whole sleep cycle is crucial to improve sleep staging performance, the sequential modelling approach in existing state-of-the-art deep learning models are inefficient for that purpose. We thus introduce a method for efficient long sequence modelling and propose a new deep learning model, L-SeqSleepNet, which takes into account whole-cycle sleep information for sleep staging. Evaluating L-SeqSleepNet on four distinct databases of various sizes, we demonstrate state-of-the-art performance obtained by the model over three different EEG setups, including scalp EEG in conventional Polysomnography (PSG), in-ear EEG, and around-the-ear EEG (cEEGrid), even with a single EEG channel input. Our analyses also show that L-SeqSleepNet is able to alleviate the predominance of N2 sleep (the major class in terms of classification) to bring down errors in other sleep stages. Moreover the network becomes much more robust, meaning that for all subjects where the baseline method had exceptionally poor performance, their performance are improved significantly. Finally, the computation time only grows at a sub-linear rate when the sequence length increases.

Pediatric Automatic Sleep Staging: A Comparative Study of State-of-the-Art Deep Learning Methods.

BACKGROUND: Despite the tremendous prog- ress recently made towards automatic sleep staging in adults, it is currently unknown if the most advanced algorithms generalize to the pediatric population, which displays distinctive characteristics in overnight polysomnography (PSG). METHODS: To answer the question, in this work, we conduct a large-scale comparative study on the state-of-the-art deep learning methods for pediatric automatic sleep staging. Six different deep neural networks with diverging features are adopted to evaluate a sample of more than 1,200 children across a wide spectrum of obstructive sleep apnea (OSA) severity. RESULTS: Our experimental results show that the individual performance of automated pediatric sleep stagers when evaluated on new subjects is equivalent to the expert-level one reported on adults. Combining the six stagers into ensemble models further boosts the staging accuracy, reaching an overall accuracy of 88.8%, a Cohen's kappa of 0.852, and a macro F1-score of 85.8%. At the same time, the ensemble models lead to reduced predictive uncertainty. The results also show that the studied algorithms and their ensembles are robust to concept drift when the training and test data were recorded seven months apart and after clinical intervention. CONCLUSION: However, we show that the improvements in the staging performance are not necessarily clinically significant although the ensemble models lead to more favorable clinical measures than the six standalone models. SIGNIFICANCE: Detailed analyses further demonstrate "almost perfect" agreement between the automatic stagers to one another and their similar patterns on the staging errors, suggesting little room for improvement.

SleepTransformer: Automatic Sleep Staging With Interpretability and Uncertainty Quantification.

BACKGROUND: Black-box skepticism is one of the main hindrances impeding deep-learning-based automatic sleep scoring from being used in clinical environments. METHODS: Towards interpretability, this work proposes a sequence-to-sequence sleep-staging model, namely SleepTransformer. It is based on the transformer backbone and offers interpretability of the model's decisions at both the epoch and sequence level. We further propose a simple yet efficient method to quantify uncertainty in the model's decisions. The method, which is based on entropy, can serve as a metric for deferring low-confidence epochs to a human expert for further inspection. RESULTS: Making sense of the transformer's self-attention scores for interpretability, at the epoch level, the attention scores are encoded as a heat map to highlight sleep-relevant features captured from the input EEG signal. At the sequence level, the attention scores are visualized as the influence of different neighboring epochs in an input sequence (i.e. the context) to recognition of a target epoch, mimicking the way manual scoring is done by human experts. CONCLUSION: Additionally, we demonstrate that SleepTransformer performs on par with existing methods on two databases of different sizes. SIGNIFICANCE: Equipped with interpretability and the ability of uncertainty quantification, SleepTransformer holds promise for being integrated into clinical settings.

Direct multi-dimensional Chebyshev polynomial based reconstruction for magnetic particle imaging.

Magnetic Particle Imaging is a tomographic imaging technique that measures the voltage induced due to magnetization changes of magnetic nanoparticle distributions. The relationship between the received signal and the distribution of the nanoparticels is described by the system function. A common method for image reconstruction is using a measured system function to create a system matrix and set up a regularized linear system of equations. Since the measurement of the system matrix is time-consuming, different methods for acceleration have been proposed. These include modeling the system matrix or using a direct reconstruction method in time, known as X-space reconstruction. In this work, based on the simplified Langevin model of paramagnetism and certain approximations, a direct reconstruction technique for Magnetic Particle Imaging in the frequency domain with two- and three-dimensional Lissajous trajectory excitation is presented. The approach uses Chebyshev polynomials of second kind. During reconstruction, they are weighted with the frequency components of the voltage signal and additional factors and then summed up. To obtain the final nanoparticle distribution, this result is rescaled and deconvolved. It is shown that the approach works for both simulated data and real measurements. The obtained image quality is comparable to a modeled system matrix approach using the same simplified physical assumptions and no relaxation effects. The reconstruction of a 31 × 31 × 31 volume takes less than a second and is up to 25 times faster than the state-of-the-art Kaczmarz reconstruction. Besides, the derivation of the proposed method shows some new theoretical aspects of the system function and its well-known observed similarity to tensor products of Chebyshev polynomials of second kind.

XSleepNet: Multi-View Sequential Model for Automatic Sleep Staging.

Automating sleep staging is vital to scale up sleep assessment and diagnosis to serve millions experiencing sleep deprivation and disorders and enable longitudinal sleep monitoring in home environments. Learning from raw polysomnography signals and their derived time-frequency image representations has been prevalent. However, learning from multi-view inputs (e.g., both the raw signals and the time-frequency images) for sleep staging is difficult and not well understood. This work proposes a sequence-to-sequence sleep staging model, XSleepNet,1 that is capable of learning a joint representation from both raw signals and time-frequency images. Since different views may generalize or overfit at different rates, the proposed network is trained such that the learning pace on each view is adapted based on their generalization/overfitting behavior. In simple terms, the learning on a particular view is speeded up when it is generalizing well and slowed down when it is overfitting. View-specific generalization/overfitting measures are computed on-the-fly during the training course and used to derive weights to blend the gradients from different views. As a result, the network is able to retain the representation power of different views in the joint features which represent the underlying distribution better than those learned by each individual view alone. Furthermore, the XSleepNet architecture is principally designed to gain robustness to the amount of training data and to increase the complementarity between the input views. Experimental results on five databases of different sizes show that XSleepNet consistently outperforms the single-view baselines and the multi-view baseline with a simple fusion strategy. Finally, XSleepNet also outperforms prior sleep staging methods and improves previous state-of-the-art results on the experimental databases.

Inception-Based Network and Multi-Spectrogram Ensemble Applied To Predict Respiratory Anomalies and Lung Diseases.

This paper presents an inception-based deep neural network for detecting lung diseases using respiratory sound input. Recordings of respiratory sound collected from patients are first transformed into spectrograms where both spectral and temporal information are well represented, in a process referred to as front-end feature extraction. These spectrograms are then fed into the proposed network, in a process referred to as back-end classification, for detecting whether patients suffer from lung-related diseases. Our experiments, conducted over the ICBHI benchmark metadataset of respiratory sound, achieve competitive ICBHI scores of 0.53/0.45 and 0.87/0.85 regarding respiratory anomaly and disease detection, respectively.

sEMG-Based Hand Movement Regression by Prediction of Joint Angles With Recurrent Neural Networks.

This work takes a step towards a better biosignal based hand gesture recognition by investigating the strategies for a reliable prediction of hand joint angles. Those strategies are especially important for medical applications in order to achieve e.g. good acceptance of hand prostheses among amputees. A recurrent neural network with a small footprint is deployed to estimate the joint positions from surface electromyography data measured at the forearm. As the predictions are expected to be not smooth, different post processing methods and a regularisation term for the objective function of the network are proposed. The experiments were conducted on publicly available databases. The results reveal that both post processing strategies and regularisation have a positive impact on the results with a maximal relative improvement of 6.13 %. On the one hand post processing strategies introduce an additional delay, consequently, the improvement is analysed in context of the caused delay. On the other hand the regularisation strategy does not cause a delay and can be adjusted easily to cope with different ground truths or compensate for certain problems in the hand tracking.

Segmented simultaneous multi-slice diffusion-weighted imaging with navigated 3D rigid motion correction.

PURPOSE: To improve the robustness of diffusion-weighted imaging (DWI) data acquired with segmented simultaneous multi-slice (SMS) echo-planar imaging (EPI) against in-plane and through-plane rigid motion. THEORY AND METHODS: The proposed algorithm incorporates a 3D rigid motion correction and wavelet denoising into the image reconstruction of segmented SMS-EPI diffusion data. Low-resolution navigators are used to estimate shot-specific diffusion phase corruptions and 3D rigid motion parameters through SMS-to-volume registration. The shot-wise rigid motion and phase parameters are integrated into a SENSE-based full-volume reconstruction for each diffusion direction. The algorithm is compared to a navigated SMS reconstruction without gross motion correction in simulations and in vivo studies with four-fold interleaved 3-SMS diffusion tensor acquisitions. RESULTS: Simulations demonstrate high fidelity was achieved in the SMS-to-volume registration, with submillimeter registration errors and improved image reconstruction quality. In vivo experiments validate successful artifact reduction in 3D motion-compromised in vivo scans with a temporal motion resolution of approximately 0.3 s. CONCLUSION: This work demonstrates the feasibility of retrospective 3D rigid motion correction from shot navigators for segmented SMS DWI.

CNN-MoE Based Framework for Classification of Respiratory Anomalies and Lung Disease Detection.

This paper presents and explores a robust deep learning framework for auscultation analysis. This aims to classify anomalies in respiratory cycles and detect diseases, from respiratory sound recordings. The framework begins with front-end feature extraction that transforms input sound into a spectrogram representation. Then, a back-end deep learning network is used to classify the spectrogram features into categories of respiratory anomaly cycles or diseases. Experiments, conducted over the ICBHI benchmark dataset of respiratory sounds, confirm three main contributions towards respiratory-sound analysis. Firstly, we carry out an extensive exploration of the effect of spectrogram types, spectral-time resolution, overlapping/non-overlapping windows, and data augmentation on final prediction accuracy. This leads us to propose a novel deep learning system, built on the proposed framework, which outperforms current state-of-the-art methods. Finally, we apply a Teacher-Student scheme to achieve a trade-off between model performance and model complexity which holds promise for building real-time applications.

Towards More Accurate Automatic Sleep Staging via Deep Transfer Learning.

BACKGROUND: Despite recent significant progress in the development of automatic sleep staging methods, building a good model still remains a big challenge for sleep studies with a small cohort due to the data-variability and data-inefficiency issues. This work presents a deep transfer learning approach to overcome these issues and enable transferring knowledge from a large dataset to a small cohort for automatic sleep staging. METHODS: We start from a generic end-to-end deep learning framework for sequence-to-sequence sleep staging and derive two networks as the means for transfer learning. The networks are first trained in the source domain (i.e. the large database). The pretrained networks are then finetuned in the target domain (i.e. the small cohort) to complete knowledge transfer. We employ the Montreal Archive of Sleep Studies (MASS) database consisting of 200 subjects as the source domain and study deep transfer learning on three different target domains: the Sleep Cassette subset and the Sleep Telemetry subset of the Sleep-EDF Expanded database, and the Surrey-cEEGrid database. The target domains are purposely adopted to cover different degrees of data mismatch to the source domains. RESULTS: Our experimental results show significant performance improvement on automatic sleep staging on the target domains achieved with the proposed deep transfer learning approach. CONCLUSIONS: These results suggest the efficacy of the proposed approach in addressing the above-mentioned data-variability and data-inefficiency issues. SIGNIFICANCE: As a consequence, it would enable one to improve the quality of automatic sleep staging models when the amount of data is relatively small.11The source code and the pretrained models are published at https://github.com/pquochuy/sleep_transfer_learning.

Regression of Hand Movements from sEMG Data with Recurrent Neural Networks.

Most wearable human-machine interfaces concerning hand movements only focus on classifying a limited number of hand gestures. With the introduction of deep learning, surface electromyography based hand gesture classification systems improved drastically. Therefore, it is worth investigating whether the classification can be replaced by a movement regression of all the different movable hand parts. As recurrent neural networks based approaches have proven their abilities of solving the classification problem we also choose them for the regression problem. Experiments were conducted with multiple different network architectures on several databases. Furthermore, due to the lack of a reliable measure to compare different gesture regression approaches we propose an interpretable and reproducible new error measure that can even handle noisy ground truth data. The results reveal the general possibility of regressing detailed hand movements. Even with the relatively simple networks the hand gestures can be regressed quite accurately.

Personalized automatic sleep staging with single-night data: a pilot study with Kullback-Leibler divergence regularization.

OBJECTIVE: Brain waves vary between people. This work aims to improve automatic sleep staging for longitudinal sleep monitoring via personalization of algorithms based on individual characteristics extracted from sleep data recorded during the first night. APPROACH: As data from a single night are very small, thereby making model training difficult, we propose a Kullback-Leibler (KL) divergence regularized transfer learning approach to address this problem. We employ the pretrained SeqSleepNet (i.e. the subject independent model) as a starting point and finetune it with the single-night personalization data to derive the personalized model. This is done by adding the KL divergence between the output of the subject independent model and it of the personalized model to the loss function during finetuning. In effect, KL-divergence regularization prevents the personalized model from overfitting to the single-night data and straying too far away from the subject independent model. MAIN RESULTS: Experimental results on the Sleep-EDF Expanded database consisting of 75 subjects show that sleep staging personalization with single-night data is possible with help of the proposed KL-divergence regularization. On average, we achieve a personalized sleep staging accuracy of 79.6%, a Cohen's kappa of 0.706, a macro F1-score of 73.0%, a sensitivity of 71.8%, and a specificity of 94.2%. SIGNIFICANCE: We find both that the approach is robust against overfitting and that it improves the accuracy by 4.5 percentage points compared to the baseline method without personalization and 2.2 percentage points compared to it with personalization but without regularization.

Effects of hunger, satiety and oral glucose on effective connectivity between hypothalamus and insular cortex.

The hypothalamus and insular cortex play an essential role in the integration of endocrine and homeostatic signals and their impact on food intake. Resting-state functional connectivity alterations of the hypothalamus, posterior insula (PINS) and anterior insula (AINS) are modulated by metabolic states and caloric intake. Nevertheless, a deeper understanding of how these factors affect the strength of connectivity between hypothalamus, PINS and AINS is missing. This study investigated whether effective (directed) connectivity within this network varies as a function of prandial states (hunger vs. satiety) and energy availability (glucose levels and/or hormonal modulation). To address this question, we measured twenty healthy male participants of normal weight twice: once after 36 â€‹h of fasting (except water consumption) and once under satiated conditions. During each session, resting-state functional MRI (rs-fMRI) and hormone concentrations were recorded before and after glucose administration. Spectral dynamic causal modeling (spDCM) was used to assess the effective connectivity between the hypothalamus and anterior and posterior insula. Using Bayesian model selection, we observed that the same model was identified as the most likely model for each rs-fMRI recording. Compared to satiety, the hunger condition enhanced the strength of the forward connections from PINS to AINS and reduced the strength of backward connections from AINS to PINS. Furthermore, the strength of connectivity from PINS to AINS was positively related to plasma cortisol levels in the hunger condition, mainly before glucose administration. However, there was no direct relationship between glucose treatment and effective connectivity. Our findings suggest that prandial states modulate connectivity between PINS and AINS and relate to theories of interoception and homeostatic regulation that invoke hierarchical relations between posterior and anterior insula.

Segmented diffusion imaging with iterative motion-corrected reconstruction (SEDIMENT) for brain echo-planar imaging.

Multi-shot techniques offer improved resolution and signal-to-noise ratio for diffusion- weighted imaging, but make the acquisition vulnerable to shot-specific phase variations and inter-shot macroscopic motion. Several model-based reconstruction approaches with iterative phase correction have been proposed, but robust macroscopic motion estimation is still challenging. Segmented diffusion imaging with iterative motion-corrected reconstruction (SEDIMENT) uses iteratively refined data-driven shot navigators based on sensitivity encoding to cure phase and rigid in-plane motion artifacts. The iterative scheme is compared in simulations and in vivo with a non-iterative reference algorithm for echo-planar imaging with up to sixfold segmentation. The SEDIMENT framework supports partial Fourier acquisitions and furthermore includes options for data rejection and learning-based modules to improve robustness and convergence.

Multiresolution vessel detection in magnetic particle imaging using wavelets and a Gaussian mixture model.

PURPOSE: Magnetic particle imaging is a tomographic imaging technique that allows one to measure the spatial distribution of superparamagnetic nanoparticles, which are used as tracer. The magnetic particle imaging scanner measures the voltage induced due to the nonlinear magnetization behavior of the nanoparticles. The tracer distribution can be reconstructed from the voltage signal by solving an inverse problem. A possible application is the imaging of vessel structures. In this and many other cases, the tracer is only located inside the structures and a large part of the image is related to background. A detection of the tracer support in early stages of the reconstruction process could improve reconstruction results. METHODS: In this work, a multiresolution wavelet-based reconstruction combined with a segmentation of the foreground structures is performed. For this, different wavelets are compared with respect to their reconstruction quality. For the detection of the foreground, a segmentation with a Gaussian mixture model is performed, which leads to a threshold-based binary segmentation. This segmentation is done on a coarse level of the reconstruction and then transferred to the next finer level, where it is used as prior knowledge for the reconstruction. This is repeated until the finest resolution is reached. RESULTS: The approach is evaluated on simulated vessel phantoms and on two real measurements. The results show that this method improves the structural similarity index of the reconstructed images significantly. Among the compared wavelets, the 9/7 wavelets led to the best reconstruction results. CONCLUSIONS: The early detection of the vessel structures at low resolution helps to improve the image quality. For the wavelet decomposition, the use of 9/7 wavelets is recommended.

Impact of Hunger, Satiety, and Oral Glucose on the Association Between Insulin and Resting-State Human Brain Activity.

To study the interplay of metabolic state (hungry vs. satiated) and glucose administration (including hormonal modulation) on brain function, resting-state functional magnetic resonance imaging (rs-fMRI) and blood samples were obtained in 24 healthy normal-weight men in a repeated measurement design. Participants were measured twice: once after a 36 h fast (except water) and once under satiation (three meals/day for 36 h). During each session, rs-fMRI and hormone concentrations were recorded before and after a 75 g oral dose of glucose. We calculated the amplitude map from blood-oxygen-level-dependent (BOLD) signals by using the fractional amplitude of low-frequency fluctuation (fALFF) approach for each volunteer per condition. Using multiple linear regression analysis (MLRA) the interdependence of brain activity, plasma insulin and blood glucose was investigated. We observed a modulatory impact of fasting state on intrinsic brain activity in the posterior cingulate cortex (PCC). Strikingly, differences in plasma insulin levels between hunger and satiety states after glucose administration at the time of the scan were negatively related to brain activity in the posterior insula and superior frontal gyrus (SFG), while plasma glucose levels were positively associated with activity changes in the fusiform gyrus. Furthermore, we could show that changes in plasma insulin enhanced the connectivity between the posterior insula and SFG. Our results indicate that hormonal signals like insulin alleviate an acute hemostatic energy deficit by modifying the homeostatic and frontal circuitry of the human brain.

Machine Learning Based Classification of Resting-State fMRI Features Exemplified by Metabolic State (Hunger/Satiety).

OBJECTIVE: Resting-state functional magnetic resonance imaging (rs-fMRI) has become an essential measure to investigate the human brain's spontaneous activity and intrinsic functional connectivity. Several studies including our own previous work have shown that the brain controls the regulation of energy expenditure and food intake behavior. Accordingly, we expected different metabolic states to influence connectivity and activity patterns in neuronal networks. METHODS: The influence of hunger and satiety on rs-fMRI was investigated using three connectivity models (local connectivity, global connectivity and amplitude rs-fMRI signals). After extracting the connectivity parameters of 90 brain regions for each model, we used sequential forward floating selection strategy in conjunction with a linear support vector machine classifier and permutation tests to reveal which connectivity model differentiates best between metabolic states (hunger vs. satiety). RESULTS: We found that the amplitude of rs-fMRI signals is slightly more precise than local and global connectivity models in order to detect resting brain changes during hunger and satiety with a classification accuracy of 81%. CONCLUSION: The amplitude of rs-fMRI signals serves as a suitable basis for machine learning based classification of brain activity. This opens up the possibility to apply this combination of algorithms to similar research questions, such as the characterization of brain states (e.g., sleep stages) or disease conditions (e.g., Alzheimer's disease, minimal cognitive impairment).

Fusion of End-to-End Deep Learning Models for Sequence-to-Sequence Sleep Staging.

Sleep staging, a process of identifying the sleep stages associated with polysomnography (PSG) epochs, plays an important role in sleep monitoring and diagnosing sleep disorders. We present in this work a model fusion approach to automate this task. The fusion model is composed of two base sleep-stage classifiers, SeqSleepNet and DeepSleepNet, both of which are state-of-the-art end-to-end deep learning models complying to the sequence-to-sequence sleep staging scheme. In addition, in the light of ensemble methods, we reason and demonstrate that these two networks form a good ensemble of models due to their high diversity. Experiments show that the fusion approach is able to preserve the strength of the base networks in the fusion model, leading to consistent performance gains over the two base networks. The fusion model obtain the best modelling results we have observed so far on the Montreal Archive of Sleep Studies (MASS) dataset with 200 subjects, achieving an overall accuracy of 88.0%, a macro F1-score of 84.3%, and a Cohen's kappa of 0.828.

A Recurrent Neural Network for Hand Gesture Recognition based on Accelerometer Data.

For many applications, hand gesture recognition systems that rely on biosignal data exclusively are mandatory. Usually, theses systems have to be affordable, reliable as well as mobile. The hand is moved due to muscle contractions that cause motions of the forearm skin. Theses motions can be captured with cheap and reliable accelerometers placed around the forearm. Since accelerometers can also be integrated into mobile systems easily, the possibility of a robust hand gesture recognition based on accelerometer signals is evaluated in this work. For this, a neural network architecture consisting of two different kinds of recurrent neural network (RNN) cells is proposed. Experiments on three databases reveal that this relatively small network outperforms by far state-of-the-art hand gesture recognition approaches that rely on multi-modal data. The combination of accelerometer data and an RNN forms a robust hand gesture classification system, i.e., the performance of the network does not vary a lot between subjects and it is outstanding for amputees. Furthermore, the proposed network uses only 5 ms short windows to classify the hand gestures. Consequently, this approach allows for a quick, and potentially delay-free hand gesture detection.

Recurrent Neural Network Based Early Prediction of Future Hand Movements.

This work focuses on a system for hand prostheses that can overcome the delay problem introduced by classical approaches while being reliable. The proposed approach based on a recurrent neural network enables us to incorporate the sequential nature of the surface electromyogram data and the proposed system can be used either for classification or early prediction of hand movements. Especially the latter is a key to a latency free steering of a prosthesis. The experiments conducted on the first three Ninapro databases reveal that the prediction up to 200 ms ahead in the future is possible without a significant drop in accuracy. Furthermore, for classification, our proposed approach outperforms the state of the art classifiers even though we used significantly shorter windows for feature extraction.