Personalization of Automatic Sleep Scoring: How Best to Adapt Models to Personal Domains in Wearable EEG.

Wearable EEG enables us to capture large amounts of high-quality sleep data for diagnostic purposes. To make full use of this capacity we need high-performance automatic sleep scoring models. To this end, it has been noted that domain mismatch between recording equipment can be considerable, e.g. PSG to wearable EEG, but a previously observed benefit from personalizing models to individual subjects further indicates a personal domain in sleep EEG. In this work, we have investigated the extent of such a personal domain in wearable EEG, and review supervised and unsupervised approaches to personalization as found in the literature. We investigated the personalization effect of the unsupervised Adversarial Domain Adaptation and implemented an unsupervised method based on statistics alignment. No beneficial personalization effect was observed using these unsupervised methods. We find that supervised personalization leads to a substantial performance improvement on the target subject ranging from 15% Cohen's Kappa for subjects with poor performance ( ) to roughly 2% on subjects with high performance ( ). This improvement was present for models trained on both small and large data sets, indicating that even high-performance models benefit from supervised personalization. We found that this personalization can be beneficially regularized using Kullback-Leibler regularization, leading to lower variance with negligible cost to improvement. Based on the experiments, we recommend model personalization using Kullback-Leibler regularization.

L-SeqSleepNet: Whole-cycle Long Sequence Modeling for Automatic Sleep Staging.

Human sleep is cyclical with a period of approximately 90 minutes, implying long temporal dependency in the sleep data. Yet, exploring this long-term dependency when developing sleep staging models has remained untouched. In this work, we show that while encoding the logic of a whole sleep cycle is crucial to improve sleep staging performance, the sequential modelling approach in existing state-of-the-art deep learning models are inefficient for that purpose. We thus introduce a method for efficient long sequence modelling and propose a new deep learning model, L-SeqSleepNet, which takes into account whole-cycle sleep information for sleep staging. Evaluating L-SeqSleepNet on four distinct databases of various sizes, we demonstrate state-of-the-art performance obtained by the model over three different EEG setups, including scalp EEG in conventional Polysomnography (PSG), in-ear EEG, and around-the-ear EEG (cEEGrid), even with a single EEG channel input. Our analyses also show that L-SeqSleepNet is able to alleviate the predominance of N2 sleep (the major class in terms of classification) to bring down errors in other sleep stages. Moreover the network becomes much more robust, meaning that for all subjects where the baseline method had exceptionally poor performance, their performance are improved significantly. Finally, the computation time only grows at a sub-linear rate when the sequence length increases.

Long-term ear-EEG monitoring of sleep - A case study during shift work.

The interest in sleep as a potential clinical biomarker is growing, but the standard method of sleep assessment, polysomnography, is expensive, time consuming, and requires a lot of expert assistance for both set-up and interpretation. To make sleep analysis more available both in research and in the clinic, there is a need for a reliable wearable device for sleep staging. In this case study, we test ear-electroencephalography. A wearable, where electrodes are placed in the outer ear, as a platform for longitudinal at-home recording of sleep. We explore the usability of the ear-electroencephalography in a shift work case with alternating sleep conditions. We find the ear-electroencephalography platform to be reliable both in terms of showing substantial agreement to polysomnography after long-time use (with an overall agreement, using Cohen's kappa, of 0.72) and by being unobtrusive enough to wear during night shift conditions. We find that fractions of non-rapid eye movement sleep and transition probability between sleep stages show great potential as sleep metrics when exploring quantitative differences in sleep architecture between shifting sleep conditions. This study shows that the ear-electroencephalography platform holds great potential as a reliable wearable for quantifying sleep "in the wild", pushing this technology further towards clinical adaptation.

At-home sleep monitoring using generic ear-EEG.

Introduction: A device comprising two generic earpieces with embedded dry electrodes for ear-centered electroencephalography (ear-EEG) was developed. The objective was to provide ear-EEG based sleep monitoring to a wide range of the population without tailoring the device to the individual. Methods: To validate the device ten healthy subjects were recruited for a 12-night sleep study. The study was divided into two parts; part A comprised two nights with both ear-EEG and polysomnography (PSG), and part B comprised 10 nights using only ear-EEG. In addition to the electrophysiological measurements, subjects filled out a questionnaire after each night of sleep. Results: The subjects reported that the ear-EEG system was easy to use, and that the comfort was better in part B. The performance of the system was validated by comparing automatic sleep scoring based on ear-EEG with PSG-based sleep scoring performed by a professional trained sleep scorer. Cohen's kappa was used to assess the agreement between the manual and automatic sleep scorings, and the study showed an average kappa value of 0.71. The majority of the 20 recordings from part A yielded a kappa value above 0.7. The study was compared to a companioned study conducted with individualized earpieces. To compare the sleep across the two studies and two parts, 7 different sleeps metrics were calculated based on the automatic sleep scorings. The ear-EEG nights were validated through linear mixed model analysis in which the effects of equipment (individualized vs. generic earpieces), part (PSG and ear-EEG vs. only ear-EEG) and subject were investigated. We found that the subject effect was significant for all computed sleep metrics. Furthermore, the equipment did not show any statistical significant effect on any of the sleep metrics. Discussion: These results corroborate that generic ear-EEG is a promising alternative to the gold standard PSG for sleep stage monitoring. This will allow sleep stage monitoring to be performed in a less obtrusive way and over longer periods of time, thereby enabling diagnosis and treatment of diseases with associated sleep disorders.

Self-applied ear-EEG for sleep monitoring at home.

High quality sleep monitoring is done using EEG electrodes placed on the skin. This has traditionally required assistance by an expert when the equipment needed to mounted. However, this creates a limitation in how cheap and easy it can be to record sleep in the subject's own home. Here we present a data set of 120 home recordings of sleep, in which subjects use self-applied ear-EEG monitoring equipment. We compare this data set to a previously recorded data set with both ear-EEG and polysomnography, which was applied by an expert. Clinical relevance - On all tested metrics, self applied sleep recordings behaved the same as expert applied. This indicates that ear-EEG can reliably be used as a home sleep monitor, even when subjects apply the equipment themselves.

Automatic sleep staging of EEG signals: recent development, challenges, and future directions.

Modern deep learning holds a great potential to transform clinical studies of human sleep. Teaching a machine to carry out routine tasks would be a tremendous reduction in workload for clinicians. Sleep staging, a fundamental step in sleep practice, is a suitable task for this and will be the focus in this article. Recently, automatic sleep-staging systems have been trained to mimic manual scoring, leading to similar performance to human sleep experts, at least on scoring of healthy subjects. Despite tremendous progress, we have not seen automatic sleep scoring adopted widely in clinical environments. This review aims to provide the shared view of the authors on the most recent state-of-the-art developments in automatic sleep staging, the challenges that still need to be addressed, and the future directions needed for automatic sleep scoring to achieve clinical value.

SleepTransformer: Automatic Sleep Staging With Interpretability and Uncertainty Quantification.

BACKGROUND: Black-box skepticism is one of the main hindrances impeding deep-learning-based automatic sleep scoring from being used in clinical environments. METHODS: Towards interpretability, this work proposes a sequence-to-sequence sleep-staging model, namely SleepTransformer. It is based on the transformer backbone and offers interpretability of the model's decisions at both the epoch and sequence level. We further propose a simple yet efficient method to quantify uncertainty in the model's decisions. The method, which is based on entropy, can serve as a metric for deferring low-confidence epochs to a human expert for further inspection. RESULTS: Making sense of the transformer's self-attention scores for interpretability, at the epoch level, the attention scores are encoded as a heat map to highlight sleep-relevant features captured from the input EEG signal. At the sequence level, the attention scores are visualized as the influence of different neighboring epochs in an input sequence (i.e. the context) to recognition of a target epoch, mimicking the way manual scoring is done by human experts. CONCLUSION: Additionally, we demonstrate that SleepTransformer performs on par with existing methods on two databases of different sizes. SIGNIFICANCE: Equipped with interpretability and the ability of uncertainty quantification, SleepTransformer holds promise for being integrated into clinical settings.

Repeated automatic sleep scoring based on ear-EEG is a valuable alternative to manually scored polysomnography.

While polysomnography (PSG) is the gold standard to quantify sleep, modern technology allows for new alternatives. PSG is obtrusive, affects the sleep it is set out to measure and requires technical assistance for mounting. A number of less obtrusive solutions based on alternative methods have been introduced, but few have been clinically validated. Here we validate one of these solutions, the ear-EEG method, against concurrently recorded PSG in twenty healthy subjects each measured for four nights. Two trained technicians scored the 80 nights of PSG independently, while an automatic algorithm scored the ear-EEG. The sleep stages and eight sleep metrics (Total Sleep Time (TST), Sleep Onset Latency, Sleep Efficiency, Wake After Sleep Onset, REM latency, REM fraction of TST, N2 fraction of TST, and N3 fraction of TST) were used in the further analysis. We found the sleep metrics: Total Sleep Time, Sleep Onset Latency, Sleep Efficiency, Wake After Sleep Onset were estimated with high accuracy and precision between automatic sleep scoring and manual sleep scoring. However, the REM latency and REM fraction of sleep showed high accuracy but low precision. Further, the automatic sleep scoring systematically overestimated the N2 fraction of sleep and slightly underestimated the N3 fraction of sleep. We demonstrate that sleep metrics estimated from automatic sleep scoring based on repeated ear-EEG in some cases are more reliably estimated with repeated nights of automatically scored ear-EEG than with a single night of manually scored PSG. Thus, given the obtrusiveness and cost of PSG, ear-EEG seems to be a useful alternative for sleep staging for the single night recording and an advantageous choice for several nights of sleep monitoring.

Sleep Monitoring Using Ear-Centered Setups: Investigating the Influence From Electrode Configurations.

Modern sleep monitoring development is shifting towards the use of unobtrusive sensors combined with algorithms for automatic sleep scoring. Many different combinations of wet and dry electrodes, ear-centered, forehead-mounted or headband-inspired designs have been proposed, alongside an ever growing variety of machine learning algorithms for automatic sleep scoring. OBJECTIVE: Among candidate positions, those in the facial area and around the ears have the benefit of being relatively hairless, and in our view deserve extra attention. In this paper, we seek to determine the limits to sleep monitoring quality within this spatial constraint. METHODS: We compare 13 different, realistic sensor setups derived from the same data set and analysed with the same pipeline. RESULTS: All setups which include both a lateral and an EOG derivation show similar, state-of-the-art performance, with average Cohen's kappa values of at least 0.80. CONCLUSION: If large electrode distances are used, positioning is not critical for achieving large sleep-related signal-to-noise-ratio, and hence accurate sleep scoring. SIGNIFICANCE: We argue that with the current competitive performance of automated staging approaches, there is a need for establishing an improved benchmark beyond current single human rater scoring.

Investigation of low dimensional feature spaces for automatic sleep staging.

BACKGROUND: Automatic sleep stage classification depends crucially on the selection of a good set of descriptive features. However, the selection of a feature set with an appropriate low computational cost without compromising classification performance is still a challenge. This study attempts to represent sleep EEG patterns using a minimum number of features, without significant performance loss. METHODS: Three feature selection algorithms were applied to a high dimensional feature space comprising 84 features. These methods were based on a bootstrapping approach guided by Gini ranking and mutual information between the features. The algorithms were tested on three scalp electroencephalography (EEG) and one ear-EEG datasets. The relationship between the information carried by different features was investigated using mutual information and illustrated by a graphical clustering tool. RESULTS: The minimum number of features that can represent the whole feature set without performance loss was found to range between 5 and 11 for different datasets. In ear-EEG, 7 features based on Continuous Wavelet Transform (CWT) resulted in similar performance as the whole set whereas in two scalp EEG datasets, the difference between minimal CWT set and the whole set was statistically significant (0.008 and 0.017 difference in average kappa). Features were divided into groups according to the type of information they carry. The group containing relative power features was identified as the most informative feature group in sleep stage classification, whereas the group containing non-linear features was found to be the least informative. CONCLUSIONS: This study showed that EEG sleep staging can be performed based on a low dimensional feature space without significant decrease in sleep staging performance. This is especially important in the case of wearable devices like ear-EEG where low computational complexity is needed. The division of the feature space into groups of features, and the analysis of the distribution of feature groups for different feature set sizes, is helpful in the selection of an appropriate feature set.

EEGs Vary Less Between Lab and Home Locations Than They Do Between People.

Given the rapid development of light weight EEG devices which we have witnessed the past decade, it is reasonable to ask to which extent neuroscience could now be taken outside the lab. In this study, we have designed an EEG paradigm well suited for deployment "in the wild." The paradigm is tested in repeated recordings on 20 subjects, on eight different occasions (4 in the laboratory, 4 in the subject's own home). By calculating the inter subject, intra subject and inter location variance, we find that the inter location variation for this paradigm is considerably less than the inter subject variation. We believe the paradigm is representative of a large group of other relevant paradigms. This means that given the positive results in this study, we find that if a research paradigm would benefit from being performed in less controlled environments, we expect limited problems in doing so.

Ear-EEG for sleep assessment: a comparison with actigraphy and PSG.

PURPOSE: To assess automatic sleep staging of three ear-EEG setups with different electrode configurations and compare performance with concurrent polysomnography and wrist-worn actigraphy recordings. METHODS: Automatic sleep staging was performed for single-ear, single-ear with ipsilateral mastoid, and cross-ear electrode configurations, and for actigraphy data. The polysomnography data were manually scored and used as the gold standard. The automatic sleep staging was tested on 80 full-night recordings from 20 healthy subjects. The scoring performance and sleep metrics were determined for all ear-EEG setups and the actigraphy device. RESULTS: The single-ear, the single-ear with ipsilateral mastoid setup, and the cross-ear setup performed five class sleep staging with kappa values 0.36, 0.63, and 0.72, respectively. For the single-ear with mastoid electrode and the cross-ear setup, the performance of the sleep metrics, in terms of mean absolute error, was better than the sleep metrics estimated from the actigraphy device in the current study, and also better than current state-of-the-art actigraphy studies. CONCLUSION: A statistically significant improvement in both accuracy and kappa was observed from single-ear to single-ear with ipsilateral mastoid, and from single-ear with ipsilateral mastoid to cross-ear configurations for both two and five-sleep stage classification. In terms of sleep metrics, the results were more heterogeneous, but in general, actigraphy and single-ear with ipsilateral mastoid configuration were better than the single-ear configuration; and the cross-ear configuration was consistently better than both the actigraphy device and the single-ear configuration.

Predicting Sleep Classification Performance without Labels.

When generating automatic sleep reports with mobile sleep monitoring devices, it is crucial to have a good grasp of the reliability of the result. In this paper, we feed features derived from the output of a sleep scoring algorithm to a 'regression ensemble' to estimate the quality of the automatic sleep scoring. We compare this estimate to the actual quality, calculated using a manual scoring of a concurrent polysomnography recording. We find that it is generally possible to estimate the quality of a sleep scoring, but with some uncertainty ('root mean squared error' between estimated and true Cohen's kappa is 0.078). We expect that this method could be useful in situations with many scored nights from the same subject, where an overall picture of scoring quality is needed, but where uncertainty on single nights is less of an issue.

Muscle Activity Detection during Sleep by Ear-EEG.

Muscle activation during sleep is an important biomarker in the diagnosis of several sleep disorders and neurodegenerative diseases. Muscle activity is typically assessed manually based on the EMG channels from polysomnography recordings. Ear-EEG provides a mobile and comfortable alternative for sleep assessment. In this study, ear-EEG was used to automatically detect muscle activities during sleep. The study was based on a dataset comprising four full night recordings from 20 healthy subjects with concurrent polysomnography and ear-EEG. A binary label, active or relax, extracted from the chin EMG was assigned to selected 30 s epoch of the sleep recordings in order to train a classifier to predict muscle activation. We found that the ear-EEG based classifier detected muscle activity with an accuracy of 88% and a Cohen's kappa value of 0.71 relative to the labels derived from the chin EMG channels. The analysis also showed a significant difference in the distribution of muscle activity between REM and non-REM sleep.

Personalized automatic sleep staging with single-night data: a pilot study with Kullback-Leibler divergence regularization.

OBJECTIVE: Brain waves vary between people. This work aims to improve automatic sleep staging for longitudinal sleep monitoring via personalization of algorithms based on individual characteristics extracted from sleep data recorded during the first night. APPROACH: As data from a single night are very small, thereby making model training difficult, we propose a Kullback-Leibler (KL) divergence regularized transfer learning approach to address this problem. We employ the pretrained SeqSleepNet (i.e. the subject independent model) as a starting point and finetune it with the single-night personalization data to derive the personalized model. This is done by adding the KL divergence between the output of the subject independent model and it of the personalized model to the loss function during finetuning. In effect, KL-divergence regularization prevents the personalized model from overfitting to the single-night data and straying too far away from the subject independent model. MAIN RESULTS: Experimental results on the Sleep-EDF Expanded database consisting of 75 subjects show that sleep staging personalization with single-night data is possible with help of the proposed KL-divergence regularization. On average, we achieve a personalized sleep staging accuracy of 79.6%, a Cohen's kappa of 0.706, a macro F1-score of 73.0%, a sensitivity of 71.8%, and a specificity of 94.2%. SIGNIFICANCE: We find both that the approach is robust against overfitting and that it improves the accuracy by 4.5 percentage points compared to the baseline method without personalization and 2.2 percentage points compared to it with personalization but without regularization.

Automatic sleep stage classification based on subcutaneous EEG in patients with epilepsy.

BACKGROUND: The interplay between sleep structure and seizure probability has previously been studied using electroencephalography (EEG). Combining sleep assessment and detection of epileptic activity in ultralong-term EEG could potentially optimize seizure treatment and sleep quality of patients with epilepsy. However, the current gold standard polysomnography (PSG) limits sleep recording to a few nights. A novel subcutaneous device was developed to record ultralong-term EEG, and has been shown to measure events of clinical relevance for patients with epilepsy. We investigated whether subcutaneous EEG recordings can also be used to automatically assess the sleep architecture of epilepsy patients. METHOD: Four adult inpatients with probable or definite temporal lobe epilepsy were monitored simultaneously with long-term video scalp EEG (LTV EEG) and subcutaneous EEG. In total, 11 nights with concurrent recordings were obtained. The sleep EEG in the two modalities was scored independently by a trained expert according to the American Academy of Sleep Medicine (AASM) rules. By using the sleep stage labels from the LTV EEG as ground truth, an automatic sleep stage classifier based on 30 descriptive features computed from the subcutaneous EEG was trained and tested. RESULTS: An average Cohen's kappa of [Formula: see text] was achieved using patient specific leave-one-night-out cross validation. When merging all sleep stages into a single class and thereby evaluating an awake-sleep classifier, we achieved a sensitivity of 94.8% and a specificity of 96.6%. Compared to manually labeled video-EEG, the model underestimated total sleep time and sleep efficiency by 8.6 and 1.8 min, respectively, and overestimated wakefulness after sleep onset by 13.6 min. CONCLUSION: This proof-of-concept study shows that it is possible to automatically sleep score patients with epilepsy based on two-channel subcutaneous EEG. The results are comparable with the methods currently used in clinical practice. In contrast to comparable studies with wearable EEG devices, several nights were recorded per patient, allowing for the training of patient specific algorithms that can account for the individual brain dynamics of each patient. Clinical trial registered at ClinicalTrial.gov on 19 October 2016 (ID:NCT02946151).

Accurate whole-night sleep monitoring with dry-contact ear-EEG.

Sleep is a key phenomenon to both understanding, diagnosing and treatment of many illnesses, as well as for studying health and well being in general. Today, the only widely accepted method for clinically monitoring sleep is the polysomnography (PSG), which is, however, both expensive to perform and influences the sleep. This has led to investigations into light weight electroencephalography (EEG) alternatives. However, there has been a substantial performance gap between proposed alternatives and PSG. Here we show results from an extensive study of 80 full night recordings of healthy participants wearing both PSG equipment and ear-EEG. We obtain automatic sleep scoring with an accuracy close to that achieved by manual scoring of scalp EEG (the current gold standard), using only ear-EEG as input, attaining an average Cohen's kappa of 0.73. In addition, this high performance is present for all 20 subjects. Finally, 19/20 subjects found that the ear-EEG had little to no negative effect on their sleep, and subjects were generally able to apply the equipment without supervision. This finding marks a turning point on the road to clinical long term sleep monitoring: the question should no longer be whether ear-EEG could ever be used for clinical home sleep monitoring, but rather when it will be.

Discrimination of Sleep Spindles in Ear-EEG.

Sleep spindles are brief oscillatory events observed in EEG measurements during sleep, related to both sleep staging and basic neuroscience. The objective of this study was to investigate to which extent sleep spindles are observable from ear-EEG. The analysis was based on single-night recordings from 12 subjects, wearing both a polysomnography setup and two light-weight mobile EEG devices (ear-EEG). By introducing a sleep spindle index capable of discriminating between epochs with distinct spindles and distinctly spindle-free epochs, we describe to which extent the most clear cut sleep spindles (as labeled using scalp EEG) can be detected using ear-EEG. We find that ear-EEG can be used to detect sleep spindles, at a performance level similar to scalp derivations. We speculate that part of the observed discrepancy between ear-EEG and the gold standard (scalp EEG) could be caused by the visibility of different spindles in the ear-EEG.

Machine-learning-derived sleep-wake staging from around-the-ear electroencephalogram outperforms manual scoring and actigraphy.

Quantification of sleep is important for the diagnosis of sleep disorders and sleep research. However, the only widely accepted method to obtain sleep staging is by visual analysis of polysomnography (PSG), which is expensive and time consuming. Here, we investigate automated sleep scoring based on a low-cost, mobile electroencephalogram (EEG) platform consisting of a lightweight EEG amplifier combined with flex-printed cEEGrid electrodes placed around the ear, which can be implemented as a fully self-applicable sleep system. However, cEEGrid signals have different amplitude characteristics to normal scalp PSG signals, which might be challenging for visual scoring. Therefore, this study evaluates the potential of automatic scoring of cEEGrid signals using a machine learning classifier ("random forests") and compares its performance with manual scoring of standard PSG. In addition, the automatic scoring of cEEGrid signals is compared with manual annotation of the cEEGrid recording and with simultaneous actigraphy. Acceptable recordings were obtained in 15 healthy volunteers (aged 35 ± 14.3 years) during an extended nocturnal sleep opportunity, which induced disrupted sleep with a large inter-individual variation in sleep parameters. The results demonstrate that machine-learning-based scoring of around-the-ear EEG outperforms actigraphy with respect to sleep onset and total sleep time assessments. The automated scoring outperforms human scoring of cEEGrid by standard criteria. The accuracy of machine-learning-based automated scoring of cEEGrid sleep recordings compared with manual scoring of standard PSG was satisfactory. The findings show that cEEGrid recordings combined with machine-learning-based scoring holds promise for large-scale sleep studies.

Sleep EEG Derived From Behind-the-Ear Electrodes (cEEGrid) Compared to Standard Polysomnography: A Proof of Concept Study.

Electroencephalography (EEG) recordings represent a vital component of the assessment of sleep physiology, but the methodology presently used is costly, intrusive to participants, and laborious in application. There is a recognized need to develop more easily applicable yet reliable EEG systems that allow unobtrusive long-term recording of sleep-wake EEG ideally away from the laboratory setting. cEEGrid is a recently developed flex-printed around-the-ear electrode array, which holds great potential for sleep-wake monitoring research. It is comfortable to wear, simple to apply, and minimally intrusive during sleep. Moreover, it can be combined with a smartphone-controlled miniaturized amplifier and is fully portable. Evaluation of cEEGrid as a motion-tolerant device is ongoing, but initial findings clearly indicate that it is very well suited for cognitive research. The present study aimed to explore the suitability of cEEGrid for sleep research, by testing whether cEEGrid data affords the signal quality and characteristics necessary for sleep stage scoring. In an accredited sleep laboratory, sleep data from cEEGrid and a standard PSG system were acquired simultaneously. Twenty participants were recorded for one extended nocturnal sleep opportunity. Fifteen data sets were scored manually. Sleep parameters relating to sleep maintenance and sleep architecture were then extracted and statistically assessed for signal quality and concordance. The findings suggest that the cEEGrid system is a viable and robust recording tool to capture sleep and wake EEG. Further research is needed to fully determine the suitability of cEEGrid for basic and applied research as well as sleep medicine.