Brain Signals to Rescue Aphasia, Apraxia and Dysarthria Speech Recognition.

In this paper, we propose a deep learning-based algorithm to improve the performance of automatic speech recognition (ASR) systems for aphasia, apraxia, and dysarthria speech by utilizing electroencephalography (EEG) features recorded synchronously with aphasia, apraxia, and dysarthria speech. We demonstrate a significant decoding performance improvement by more than 50% during test time for isolated speech recognition task and we also provide preliminary results indicating performance improvement for the more challenging continuous speech recognition task by utilizing EEG features. The results presented in this paper show the first step towards demonstrating the possibility of utilizing non-invasive neural signals to design a real-time robust speech prosthetic for stroke survivors recovering from aphasia, apraxia, and dysarthria. Our aphasia, apraxia, and dysarthria speech-EEG data set will be released to the public to help further advance this interesting and crucial research.

Deep J-Sense: Accelerated MRI Reconstruction via Unrolled Alternating Optimization.

Accelerated multi-coil magnetic resonance imaging reconstruction has seen a substantial recent improvement combining compressed sensing with deep learning. However, most of these methods rely on estimates of the coil sensitivity profiles, or on calibration data for estimating model parameters. Prior work has shown that these methods degrade in performance when the quality of these estimators are poor or when the scan parameters differ from the training conditions. Here we introduce Deep J-Sense as a deep learning approach that builds on unrolled alternating minimization and increases robustness: our algorithm refines both the magnetization (image) kernel and the coil sensitivity maps. Experimental results on a subset of the knee fastMRI dataset show that this increases reconstruction performance and provides a significant degree of robustness to varying acceleration factors and calibration region sizes.

Overcoming Long-Term Variability in Local Field Potentials Using an Adaptive Decoder.

Long-term variability remains one of the major hurdles in using intracortical recordings like local field potentials for brain computer interfaces (BCI). Practical neural decoders need to overcome time instability of neural signals to estimate subject behavior accurately and faithfully over the long term. This paper presents a novel decoder that 1) characterizes each behavioral task (i.e., different movement directions under different force conditions) with multiple neural patterns and 2) adapts to the long-term variations in neural features by identifying the stable neural patterns. This adaptation can be performed in both an unsupervised and a semisupervised learning framework requiring minimal feedback from the user. To achieve generalization over time, the proposed decoder uses redundant sparse regression models that adapt to day-to-day variations in neural patterns. While this update requires no explicit feedback from the BCI user, any feedback (explicit or derived) to the BCI improves its performance. With this adaptive decoder, we investigated the effects of long-term neural modulation especially when subjects encountered new external forces against movement. The proposed decoder predicted eight hand-movement directions with an accuracy of 95% over two weeks (when there was no external forces); and 85% in later acquisition sessions spanning up to 42 days (when the monkeys countered external field forces). Since the decoder can operate with or without manual intervention, it could alleviate user frustration associated with BCI.

Block design enhances classification of 3D reach targets from electroencephalographic signals.

To date, decoding accuracy of actual or imagined pointing movements to targets in 3D space from electroencephalographic (EEG) signals has remained modest. The reason may pertain to the fact that these movements activate essentially the same neural networks. In this study, we aimed at testing whether repetitive pointing movements to each of the targets promotes the development of segregated neural patterns, resulting in enhanced decoding accuracy. Six human subjects generated slow or fast repetitive pointing movements with their right dominant arm to one of five targets distributed in 3D space, followed by repetitive imagery of movements to the same target or to a different target. Nine naive subjects generated both repetitive and non-repetitive slow actual movements to each of the five targets to test the effect of block design on decoding accuracy. In order to assure that base line drift and low frequency motion artifacts do not contaminate the data, the data were high-pass filtered in 4-30Hz, leaving out the delta and gamma band. For the repetitive trials, the model decoded target location with 81% accuracy, which is significantly higher than chance level. The average decoding rate of target location was only 30% for the non-repetitive trials, which is not significantly different than chance level. A subset of electrodes, mainly over the contralateral sensorimotor areas, was found to provide most of the discriminative features for all tested conditions. Time proximity between trained and tested blocks was found to enhance decoding accuracy of target location both by target non-specific and specific mechanisms. Our findings suggest that movement repetition promotes the development of distinct neural patterns, presumably by the formation of target-specific kinesthetic memory.

Long-term decoding of arm movement using Spatial Distribution of Neural Patterns.

Day to day variability and non-stationarity caused by changes in subject motivation, learning and behavior pose a challenge in using local field potentials (LFP) for practical Brain Computer Interfaces. Pattern recognition algorithms require that the features possess little to no variation from the training to test data. As such models developed on one day fail to represent the characteristics on the other day. This paper provides a solution in the form of adaptive spatial features. We propose an algorithm to capture the local spatial variability of LFP patterns and provide accurate long-term decoding. This algorithm achieved more than 95% decoding of eight movement directions two weeks after its initial training.

Source localization techniques for direction decoding from local field potentials.

Local Field Potential (LFP) recordings are one type of intracortical recordings, (besides Single Unit Activity) that can help decode movement direction successfully. In the longterm however, using LFPs for decoding presents some major challenges like inherent instability and non-stationarity. Our approach to overcome this challenge bases around the hypothesis that each task has a signature source-location pattern. The methodology involves introduction of source localization, and tracking of sources over a period of time that enables us to decode movement direction in an eight-direction center-out-reach-task. We establish that such tracking can be used for long term decoding, with preliminary results indicating consistent patterns. In fact, tracking task related source locations render up to 66% accuracy in decoding movement direction one week after the decoding model was learnt.

Robust movement direction decoders from local field potentials using spatio-temporal qualitative patterns.

A major drawback of using Local Field Potentials (LFP) for Brain Computer Interface (BCI) is their inherent instability and non-stationarity. Specifically, even when a well-trained subject performs the same task over a period of time, the neural data observed are unstable. To overcome this problem in decoding movement direction, this paper proposes the use of qualitative information in the form of spatial patterns of inter-channel ranking of multi-channel LFP recordings. The quality of the decoding was further refined by concentrating on the statistical distributions of the top powered channels. Decoding of movement direction was performed using Support Vector Machines (SVM) to construct decoders, instead of the traditional spatial patterns. Our algorithm provides a decoding power of up to 74% on average over a period of two weeks, compared with the state-of-the-art methods in the literature that yield only 33%. Furthermore, it provides 62.5% direction decoding in novel motor environments, compared with 29.5% with conventional methods. Finally, a comparison with the traditional methods and other surveyed literature is presented.

Real time 3D visualization of intraoperative organ deformations using structured dictionary.

Restricted visualization of the surgical field is one of the most critical challenges for minimally invasive surgery (MIS). Current intraoperative visualization systems are promising. However, they can hardly meet the requirements of high resolution and real time 3D visualization of the surgical scene to support the recognition of anatomic structures for safe MIS procedures. In this paper, we present a new approach for real time 3D visualization of organ deformations based on optical imaging patches with limited field-of-view and a single preoperative scan of magnetic resonance imaging (MRI) or computed tomography (CT). The idea for reconstruction is motivated by our empirical observation that the spherical harmonic coefficients corresponding to distorted surfaces of a given organ lie in lower dimensional subspaces in a structured dictionary that can be learned from a set of representative training surfaces. We provide both theoretical and practical designs for achieving these goals. Specifically, we discuss details about the selection of limited optical views and the registration of partial optical images with a single preoperative MRI/CT scan. The design proposed in this paper is evaluated with both finite element modeling data and ex vivo experiments. The ex vivo test is conducted on fresh porcine kidneys using 3D MRI scans with 1.2 mm resolution and a portable laser scanner with an accuracy of 0.13 mm. Results show that the proposed method achieves a sub-3 mm spatial resolution in terms of Hausdorff distance when using only one preoperative MRI scan and the optical patch from the single-sided view of the kidney. The reconstruction frame rate is between 10 frames/s and 39 frames/s depending on the complexity of the test model.

Sparse Representation of Deformable 3D Organs with Spherical Harmonics and Structured Dictionary.

This paper proposed a novel algorithm to sparsely represent a deformable surface (SRDS) with low dimensionality based on spherical harmonic decomposition (SHD) and orthogonal subspace pursuit (OSP). The key idea in SRDS method is to identify the subspaces from a training data set in the transformed spherical harmonic domain and then cluster each deformation into the best-fit subspace for fast and accurate representation. This algorithm is also generalized into applications of organs with both interior and exterior surfaces. To test the feasibility, we first use the computer models to demonstrate that the proposed approach matches the accuracy of complex mathematical modeling techniques and then both ex vivo and in vivo experiments are conducted using 3D magnetic resonance imaging (MRI) scans for verification in practical settings. All results demonstrated that the proposed algorithm features sparse representation of deformable surfaces with low dimensionality and high accuracy. Specifically, the precision evaluated as maximum error distance between the reconstructed surface and the MRI ground truth is better than 3 mm in real MRI experiments.

Prediction of pharmacologically induced baroreflex sensitivity from local time and frequency domain indices of R-R interval and systolic blood pressure signals obtained during deep breathing.

Pharmacological measurement of baroreflex sensitivity (BRS) is widely accepted and used in clinical practice. Following the introduction of pharmacologically induced BRS (p-BRS), alternative assessment methods eliminating the use of drugs were in the center of interest of the cardiovascular research community. In this study we investigated whether p-BRS using phenylephrine injection can be predicted from non-pharmacological time and frequency domain indices computed from electrocardiogram (ECG) and blood pressure (BP) data acquired during deep breathing. In this scheme, ECG and BP data were recorded from 16 subjects in a two-phase experiment. In the first phase the subjects performed irregular deep breaths and in the second phase the subjects received phenylephrine injection. From the first phase of the experiment, a large pool of predictors describing the local characteristic of beat-to-beat interval tachogram (RR) and systolic blood pressure (SBP) were extracted in time and frequency domains. A subset of these indices was selected using twelve subjects with an exhaustive search fused with a leave one subject out cross validation procedure. The selected indices were used to predict the p-BRS on the remaining four test subjects. A multivariate regression was used in all prediction steps. The algorithm achieved best prediction accuracy with only two features extracted from the deep breathing data, one from the frequency and the other from the time domain. The normalized L2-norm error was computed as 22.9% and the correlation coefficient was 0.97 (p=0.03). These results suggest that the p-BRS can be estimated from non-pharmacological indices computed from ECG and invasive BP data related to deep breathing.

Intragenic rearrangement and altered RNA splicing of the androgen receptor in a cell-based model of prostate cancer progression.

Androgen depletion for advanced prostate cancer (PCa) targets activity of the androgen receptor (AR), a steroid receptor transcription factor required for PCa growth. The emergence of lethal castration-resistant PCa (CRPCa) is marked by aberrant reactivation of the AR despite ongoing androgen depletion. Recently, alternative splicing has been described as a mechanism giving rise to COOH-terminally truncated, constitutively active AR isoforms that can support the CRPCa phenotype. However, the pathologic origin of these truncated AR isoforms is unknown. The goal of this study was to investigate alterations in AR expression arising in a cell-based model of PCa progression driven by truncated AR isoform activity. We show that stable, high-level expression of truncated AR isoforms in 22Rv1 CRPCa cells is associated with intragenic rearrangement of an approximately 35-kb AR genomic segment harboring a cluster of previously described alternative AR exons. Analysis of genomic data from clinical specimens indicated that related AR intragenic copy number alterations occurred in CRPCa in the context of AR amplification. Cloning of the break fusion junction in 22Rv1 cells revealed long interspersed nuclear elements (LINE-1) flanking the rearranged segment and a DNA repair signature consistent with microhomology-mediated, break-induced replication. This rearrangement served as a marker for the emergence of a rare subpopulation of CRPCa cells expressing high levels of truncated AR isoforms during PCa progression in vitro. Together, these data provide the first report of AR intragenic rearrangements in CRPCa and an association with pathologic expression of truncated AR isoforms in a cell-based model of PCa progression.

A subspace approach to learning recurrent features from brain activity.

This paper introduces a novel technique to address the instability and time variability challenges associated with brain activity recorded on different days. A critical challenge when working with brain signal activity is the variability in their characteristics when the signals are collected in different sessions separated by a day or more. Such variability is due to the acute and chronic responses of the brain tissue after implantation, variations as the subject learns to optimize performance, physiological changes in a subject due to prior activity or rest periods and environmental conditions. We propose a novel approach to tackle signal variability by focusing on learning subspaces which are recurrent over time. Furthermore, we illustrate how we can use projections on those subspaces to improve classification for an application such as brain-machine interface (BMI). In this paper, we illustrate the merits of finding recurrent subspaces in the context of movement direction decoding using local field potential (LFP). We introduce two methods for using the learned subspaces in movement direction decoding and show a decoding power improvement from 76% to 88% for a particularly unstable subject and consistent decoding across subjects.

Semi-supervised event detection using higher order statistics for multidimensional time series accelerometer data.

In this study, we target to automatically detect stereotypical behavioral patterns (stereotypy) and self-injurious behaviors (SIB) of Autistic children which can lead to critical damages or wounds as they tend to repeatedly harm oneself. Our custom designed accelerometer based wearable sensors are placed at wrists, ankles and upper body to detect stereotypy and SIB. The analysis was done on four children diagnosed with ASD who showed repeated behaviors that involve part of the body such as flapping arms, body rocking and self-injurious behaviors such as punching their face, or hitting their legs. Our goal of detecting novel events relies on the fact that the limitation of training data and variability in the possible combination of signals and events also make it impossible to design a single algorithm to understand all events in natural setting. Therefore, a semi-supervised method to discover and track unknown events in a multidimensional sensor data rises as a very important topic in classification and detection problems. In this paper, we show how the Higher Order Statistics (HOS) features can be used to design dictionaries and to detect novel events in a multichannel time series data. We explain our methods to detect novel events in a multidimensional time series data and combine the proposed semi-supervised learning method to improve the adaptability of the system while maintaining comparable detection accuracy as the supervised method. We, compare our results to the supervised methods that we have previously developed and show that although semi-supervised method do not achieve better performance compared to supervised methods, it can efficiently find new events and anomalies in multidimensional time series data with similar performance of the supervised method. We show that our proposed method achieves recall rate of 93.3% compared to 94.1% for the supervised method studied earlier.

A real-time cardiac surface tracking system using Subspace Clustering.

Catheter based radio frequency ablation of atrial fibrillation requires real-time 3D tracking of cardiac surfaces with sub-millimeter accuracy. To best of our knowledge, there are no commercial or non-commercial systems capable to do so. In this paper, a system for high-accuracy 3D tracking of cardiac surfaces in real-time is proposed and results applied to a real patient dataset are presented. Proposed system uses Subspace Clustering algorithm to identify the potential deformation subspaces for cardiac surfaces during the training phase from pre-operative MRI scan based training set. In Tracking phase, using low-density outer cardiac surface samples, active deformation subspace is identified and complete inner & outer cardiac surfaces are reconstructed in real-time under a least squares formulation.

Automatic characterization and detection of behavioral patterns using linear predictive coding of accelerometer sensor data.

In this study, we target to automatically detect behavioral patterns of patients with autism. Many stereotypical behavioral patterns may hinder their learning ability as a child and patterns such as self-injurious behaviors (SIB) can lead to critical damages or wounds as they tend to repeatedly harm one single location. Our custom designed accelerometer based wearable sensor can be placed at various locations of the body to detect stereotypical self-stimulatory behaviors (stereotypy) and self-injurious behaviors of patients with Autism Spectrum Disorder (ASD). A microphone was used to record sounds so that we may understand the surrounding environment and video provided ground truth for analysis. The analysis was done on four children diagnosed with ASD who showed repeated self-stimulatory behaviors that involve part of the body such as flapping arms, body rocking and self-injurious behaviors such as punching their face, or hitting their legs. The goal of this study is to devise novel algorithms to detect these events and open possibility for design of intervention methods. In this paper, we have shown time domain pattern matching with linear predictive coding (LPC) of data to design detection and classification of these ASD behavioral events. We observe clusters of pole locations from LPC roots to select candidates and apply pattern matching for classification. We also show novel event detection using online dictionary update method. We show that our proposed method achieves recall rate of 95.5% for SIB, 93.5% for flapping, and 95.5% for rocking which is an increase of approximately 5% compared to flapping events detected by using wrist worn sensors in our previous study.

Time robust movement direction decoding in Local Field Potentials using channel ranking.

Movement direction for Brain Machine Interface (BMI) can be decoded successfully using Local Field Potentials (LFP) and Single Unit Activity (SUA). A major challenge when dealing with the intra-cortical recordings is to develop decoders that are robust in time. In this paper we present for the first time a technique that uses the qualitative information derived from multiple LFP channels rather than the absolute power of the recorded signals. In this novel method, we use a power based inter-channel ranking system to define the quality of a channel in multi-channel LFP. This representation enables us to bypass the problems associated with the dynamic ranges of absolute power. We also introduce a parameter based ranking system that provides the same rank to channels that have comparable powers. We show that using our algorithms, we can develop models that provide stable decoding of eight movement directions with an average efficiency of above 56% over a period of two weeks. Moreover, the decoding power using this method is 46% at the end of two weeks versus the 13% using the traditional approaches. We also applied these models to decoding movements performed in a force field and again achieved significantly higher decoding power than the existing methods.

Neural oscillations associated with the primacy and recency effects of verbal working memory.

For sequential information, the first (primacy) and last (recency) items are better remembered than items in the middle of the sequence. The cognitive operations and neural correlates for the primacy and recency effects are unclear. In this paper, we investigate brain oscillations associated with these effects. MEG recordings were obtained on 19 subjects performing a modified Sternberg paradigm. Correlation analyses were performed between brain oscillatory activity and primacy and recency indices. Oscillatory activity during information maintenance, not encoding, was correlated with the primacy and recency effects. The primacy effect was associated with occipital post-desynchrony, and temporal post-synchrony. The recency effect was associated with parietal and temporal desynchrony. Differences were also observed according to the maintenance strategy. These data indicate that the primacy and recency effects are related to different neural, and likely cognitive, operations that are dependant on the strategy for information maintenance.

High accuracy decoding of movement target direction in non-human primates based on common spatial patterns of local field potentials.

BACKGROUND: The current development of brain-machine interface technology is limited, among other factors, by concerns about the long-term stability of single- and multi-unit neural signals. In addition, the understanding of the relation between potentially more stable neural signals, such as local field potentials, and motor behavior is still in its early stages. METHODOLOGY/PRINCIPAL FINDINGS: We tested the hypothesis that spatial correlation patterns of neural data can be used to decode movement target direction. In particular, we examined local field potentials (LFP), which are thought to be more stable over time than single unit activity (SUA). Using LFP recordings from chronically implanted electrodes in the dorsal premotor and primary motor cortex of non-human primates trained to make arm movements in different directions, we made the following observations: (i) it is possible to decode movement target direction with high fidelity from the spatial correlation patterns of neural activity in both primary motor (M1) and dorsal premotor cortex (PMd); (ii) the decoding accuracy of LFP was similar to the decoding accuracy obtained with the set of SUA recorded simultaneously; (iii) directional information varied with the LFP frequency sub-band, being greater in low (0.3-4 Hz) and high (48-200 Hz) frequency bands than in intermediate bands; (iv) the amount of directional information was similar in M1 and PMd; (v) reliable decoding was achieved well in advance of movement onset; and (vi) LFP were relatively stable over a period of one week. CONCLUSIONS/SIGNIFICANCE: The results demonstrate that the spatial correlation patterns of LFP signals can be used to decode movement target direction. This finding suggests that parameters of movement, such as target direction, have a stable spatial distribution within primary motor and dorsal premotor cortex, which may be used for brain-machine interfaces.

Optimal sensor location for body sensor network to detect self-stimulatory behaviors of children with Autism Spectrum Disorder.

In this study, we investigate various locations of sensor positions to detect stereotypical self-stimulatory behavioral patterns of children with Autism Spectrum Disorder (ASD). The study is focused on finding optimal detection performance based on sensor location and number of sensors. To perform this study, we developed a wearable sensor system that uses a 3 axis accelerometer. A microphone was used to understand the surrounding environment and video provided ground truth for analysis. The recordings were done on 2 children diagnosed with ASD who showed repeated self-stimulatory behaviors that involve part of the body such as flapping arms, body rocking and vocalization of non-word sounds. We used time-frequency methods to extract features and sparse signal representation methods to design over-complete dictionary for data analysis, detection and classification of these ASD behavioral events. We show that using single sensor on the back achieves 95.5% classification rate for rocking and 80.5% for flapping. In contrast, flapping events can be recognized with 86.5% accuracy using wrist worn sensors.

Overcoming measurement time variability in brain machine interface.

We introduce a subspace learning approach for multi-channel Local Field Potentials (LFP), and demonstrate its application in movement direction decoding for 8 directions movement. We show that the subspace learning method can effectively address the issue of signal instability across recording sessions by extracting recurrent features from the data. We present results for movement direction decoding, where we trained on two recording sessions, and evaluated decoding performance on a third session. We combine our method with a classifier based on Error-Correcting Output Codes (ECOC) and Common Spatial Patterns (CSP) and found improvement in Decoding Power (DP) from 76% to 88% for a subject known to have strong inter-session variability. Furthermore, we saw an increase from 86% to 90% DP with another subject which exhibited significantly less variability.