High-Gamma Activity Is Coupled to Low-Gamma Oscillations in Precentral Cortices and Modulates with Movement and Speech.

Planning and executing motor behaviors requires coordinated neural activity among multiple cortical and subcortical regions of the brain. Phase-amplitude coupling between the high-gamma band amplitude and the phase of low frequency oscillations (theta, alpha, beta) has been proposed to reflect neural communication, as has synchronization of low-gamma oscillations. However, coupling between low-gamma and high-gamma bands has not been investigated. Here, we measured phase-amplitude coupling between low- and high-gamma in monkeys performing a reaching task and in humans either performing finger-flexion or word-reading tasks. We found significant coupling between low-gamma phase and high-gamma amplitude in multiple sensorimotor and premotor cortices of both species during all tasks. This coupling modulated with the onset of movement. These findings suggest that interactions between the low and high gamma bands are markers of network dynamics related to movement and speech generation.

Identifying Naturalistic Movies from Human Brain Activity with High-Density Diffuse Optical Tomography.

Modern neuroimaging modalities, particularly functional MRI (fMRI), can decode detailed human experiences. Thousands of viewed images can be identified or classified, and sentences can be reconstructed. Decoding paradigms often leverage encoding models that reduce the stimulus space into a smaller yet generalizable feature set. However, the neuroimaging devices used for detailed decoding are non-portable, like fMRI, or invasive, like electrocorticography, excluding application in naturalistic use. Wearable, non-invasive, but lower-resolution devices such as electroencephalography and functional near-infrared spectroscopy (fNIRS) have been limited to decoding between stimuli used during training. Herein we develop and evaluate model-based decoding with high-density diffuse optical tomography (HD-DOT), a higher-resolution expansion of fNIRS with demonstrated promise as a surrogate for fMRI. Using a motion energy model of visual content, we decoded the identities of novel movie clips outside the training set with accuracy far above chance for single-trial decoding. Decoding was robust to modulations of testing time window, different training and test imaging sessions, hemodynamic contrast, and optode array density. Our results suggest that HD-DOT can translate detailed decoding into naturalistic use.

Ultra-high density imaging arrays for diffuse optical tomography of human brain improve resolution, signal-to-noise, and information decoding.

Functional magnetic resonance imaging (fMRI) has dramatically advanced non-invasive human brain mapping and decoding. Functional near-infrared spectroscopy (fNIRS) and high-density diffuse optical tomography (HD-DOT) non-invasively measure blood oxygen fluctuations related to brain activity, like fMRI, at the brain surface, using more-lightweight equipment that circumvents ergonomic and logistical limitations of fMRI. HD-DOT grids have smaller inter-optode spacing (∼13 mm) than sparse fNIRS (∼30 mm) and therefore provide higher image quality, with spatial resolution ∼1/2 that of fMRI. Herein, simulations indicated reducing inter-optode spacing to 6.5 mm would further improve image quality and noise-resolution tradeoff, with diminishing returns below 6.5 mm. We then constructed an ultra-high-density DOT system (6.5-mm spacing) with 140 dB dynamic range that imaged stimulus-evoked activations with 30-50% higher spatial resolution and repeatable multi-focal activity with excellent agreement with participant-matched fMRI. Further, this system decoded visual stimulus position with 19-35% lower error than previous HD-DOT, throughout occipital cortex.

Scalable, modular continuous wave functional near-infrared spectroscopy system (Spotlight).

Significance: We present a fiberless, portable, and modular continuous wave-functional near-infrared spectroscopy system, Spotlight, consisting of multiple palm-sized modules-each containing high-density light-emitting diode and silicon photomultiplier detector arrays embedded in a flexible membrane that facilitates optode coupling to scalp curvature. Aim: Spotlight's goal is to be a more portable, accessible, and powerful functional near-infrared spectroscopy (fNIRS) device for neuroscience and brain-computer interface (BCI) applications. We hope that the Spotlight designs we share here can spur more advances in fNIRS technology and better enable future non-invasive neuroscience and BCI research. Approach: We report sensor characteristics in system validation on phantoms and motor cortical hemodynamic responses in a human finger-tapping experiment, where subjects wore custom 3D-printed caps with two sensor modules. Results: The task conditions can be decoded offline with a median accuracy of 69.6%, reaching 94.7% for the best subject, and at a comparable accuracy in real time for a subset of subjects. We quantified how well the custom caps fitted to each subject and observed that better fit leads to more observed task-dependent hemodynamic response and better decoding accuracy. Conclusions: The advances presented here should serve to make fNIRS more accessible for BCI applications.

High-gamma activity is coupled to low-gamma oscillations in precentral cortices and modulates with movement and speech.

Planning and executing motor behaviors requires coordinated neural activity among multiple cortical and subcortical regions of the brain. Phase-amplitude coupling between the high-gamma band amplitude and the phase of low frequency oscillations (theta, alpha, beta) has been proposed to reflect neural communication, as has synchronization of low-gamma oscillations. However, coupling between low-gamma and high-gamma bands has not been investigated. Here, we measured phase-amplitude coupling between low- and high-gamma in monkeys performing a reaching task and in humans either performing finger movements or speaking words aloud. We found significant coupling between low-gamma phase and high-gamma amplitude in multiple sensorimotor and premotor cortices of both species during all tasks. This coupling modulated with the onset of movement. These findings suggest that interactions between the low and high gamma bands are markers of network dynamics related to movement and speech generation.

Myoelectric interface training enables targeted reduction in abnormal muscle co-activation.

BACKGROUND: Abnormal patterns of muscle co-activation contribute to impaired movement after stroke. Previously, we developed a myoelectric computer interface (MyoCI) training paradigm to improve stroke-induced arm motor impairment by reducing the abnormal co-activation of arm muscle pairs. However, it is unclear to what extent the paradigm induced changes in the overall intermuscular coordination in the arm, as opposed to changing just the muscles trained with the MyoCI. This study examined the intermuscular coordination patterns of thirty-two stroke survivors who participated in 6 weeks of MyoCI training. METHODS: We used non-negative matrix factorization to identify the arm muscle synergies (coordinated patterns of muscle activity) during a reaching task before and after the training. We examined the extent to which synergies changed as the training reduced motor impairment. In addition, we introduced a new synergy analysis metric, disparity index (DI), to capture the changes in the individual muscle weights within a synergy. RESULTS: There was no consistent pattern of change in the number of synergies across the subjects after the training. The composition of muscle synergies, calculated using a traditional synergy similarity metric, also did not change after the training. However, the disparity of muscle weights within synergies increased after the training in the participants who responded to MyoCI training-that is, the specific muscles that the MyoCI was targeting became less correlated within a synergy. This trend was not observed in participants who did not respond to the training. CONCLUSIONS: These findings suggest that MyoCI training reduced arm impairment by decoupling only the muscles trained while leaving other muscles relatively unaffected. This suggests that, even after injury, the nervous system is capable of motor learning on a highly fractionated level. It also suggests that MyoCI training can do what it was designed to do-enable stroke survivors to reduce abnormal co-activation in targeted muscles. Trial registration This study was registered at ClinicalTrials.gov (NCT03579992, Registered 09 July 2018-Retrospectively registered, https://clinicaltrials.gov/ct2/show/NCT03579992?term=NCT03579992&draw=2&rank=1 ).

Multivariate Kalman filter regression of confounding physiological signals for real-time classification of fNIRS data.

Significance: Functional near-infrared spectroscopy (fNIRS) is a noninvasive technique for measuring hemodynamic changes in the human cortex related to neural function. Due to its potential for miniaturization and relatively low cost, fNIRS has been proposed for applications, such as brain-computer interfaces (BCIs). The relatively large magnitude of the signals produced by the extracerebral physiology compared with the ones produced by evoked neural activity makes real-time fNIRS signal interpretation challenging. Regression techniques incorporating physiologically relevant auxiliary signals such as short separation channels are typically used to separate the cerebral hemodynamic response from the confounding components in the signal. However, the coupling of the extra-cerebral signals is often noninstantaneous, and it is necessary to find the proper delay to optimize nuisance removal. Aim: We propose an implementation of the Kalman filter with time-embedded canonical correlation analysis for the real-time regression of fNIRS signals with multivariate nuisance regressors that take multiple delays into consideration. Approach: We tested our proposed method on a previously acquired finger tapping dataset with the purpose of classifying the neural responses as left or right. Results: We demonstrate computationally efficient real-time processing of 24-channel fNIRS data (400 samples per second per channel) with a two order of selective magnitude decrease in cardiac signal power and up to sixfold increase in the contrast-to-noise ratio compared with the nonregressed signals. Conclusion: The method provides a way to obtain better distinction of brain from non-brain signals in real time for BCI application with fNIRS.

Generating Natural, Intelligible Speech From Brain Activity in Motor, Premotor, and Inferior Frontal Cortices.

Neural interfaces that directly produce intelligible speech from brain activity would allow people with severe impairment from neurological disorders to communicate more naturally. Here, we record neural population activity in motor, premotor and inferior frontal cortices during speech production using electrocorticography (ECoG) and show that ECoG signals alone can be used to generate intelligible speech output that can preserve conversational cues. To produce speech directly from neural data, we adapted a method from the field of speech synthesis called unit selection, in which units of speech are concatenated to form audible output. In our approach, which we call Brain-To-Speech, we chose subsequent units of speech based on the measured ECoG activity to generate audio waveforms directly from the neural recordings. Brain-To-Speech employed the user's own voice to generate speech that sounded very natural and included features such as prosody and accentuation. By investigating the brain areas involved in speech production separately, we found that speech motor cortex provided more information for the reconstruction process than the other cortical areas.

Hemicraniectomy in Traumatic Brain Injury: A Noninvasive Platform to Investigate High Gamma Activity for Brain Machine Interfaces.

Brain-machine interfaces (BMIs) translate brain signals into control signals for an external device, such as a computer cursor or robotic limb. These signals can be obtained either noninvasively or invasively. Invasive recordings, using electrocorticography (ECoG) or intracortical microelectrodes, provide higher bandwidth and more informative signals. Rehabilitative BMIs, which aim to drive plasticity in the brain to enhance recovery after brain injury, have almost exclusively used non-invasive recordings, such electroencephalography (EEG) or magnetoencephalography (MEG), which have limited bandwidth and information content. Invasive recordings provide more information and spatiotemporal resolution, but do incur risk, and thus are not usually investigated in people with stroke or traumatic brain injury (TBI). Here, in this paper, we describe a new BMI paradigm to investigate the use of higher frequency signals in brain-injured subjects without incurring significant risk. We recorded EEG in TBI subjects who required hemicraniectomies (removal of a part of the skull). EEG over the hemicraniectomy (hEEG) contained substantial information in the high gamma frequency range (65-115 Hz). Using this information, we decoded continuous finger flexion force with moderate to high accuracy (variance accounted for 0.06 to 0.52), which at best approaches that using epidural signals. These results indicate that people with hemicraniectomies can provide a useful resource for developing BMI therapies for the treatment of brain injury.

Myoelectric Computer Interface Training for Reducing Co-Activation and Enhancing Arm Movement in Chronic Stroke Survivors: A Randomized Trial.

BACKGROUND: Abnormal muscle co-activation contributes to impairment after stroke. We developed a myoelectric computer interface (MyoCI) training paradigm to reduce abnormal co-activation. MyoCI provides intuitive feedback about muscle activation patterns, enabling decoupling of these muscles. OBJECTIVE: To investigate tolerability and effects of MyoCI training of 3 muscle pairs on arm motor recovery after stroke, including effects of training dose and isometric versus movement-based training. METHODS: We randomized chronic stroke survivors with moderate-to-severe arm impairment to 3 groups. Two groups tested different doses of isometric MyoCI (60 vs 90 minutes), and one group tested MyoCI without arm restraint (90 minutes), over 6 weeks. Primary outcome was arm impairment (Fugl-Meyer Assessment). Secondary outcomes included function, spasticity, and elbow range-of-motion at weeks 6 and 10. RESULTS: Over all 32 subjects, MyoCI training of 3 muscle pairs significantly reduced impairment (Fugl-Meyer Assessment) by 3.3 ± 0.6 and 3.1 ± 0.7 ( P < 10-4) at weeks 6 and 10, respectively. Each group improved significantly from baseline; no significant differences were seen between groups. Participants' lab-based and home-based function also improved at weeks 6 and 10 ( P ≤ .01). Spasticity also decreased over all subjects, and elbow range-of-motion improved. Both moderately and severely impaired patients showed significant improvement. No participants had training-related adverse events. MyoCI reduced abnormal co-activation, which appeared to transfer to reaching in the movement group. CONCLUSIONS: MyoCI is a well-tolerated, novel rehabilitation tool that enables stroke survivors to reduce abnormal co-activation. It may reduce impairment and spasticity and improve arm function, even in severely impaired patients.

Speech synthesis from ECoG using densely connected 3D convolutional neural networks.

OBJECTIVE: Direct synthesis of speech from neural signals could provide a fast and natural way of communication to people with neurological diseases. Invasively-measured brain activity (electrocorticography; ECoG) supplies the necessary temporal and spatial resolution to decode fast and complex processes such as speech production. A number of impressive advances in speech decoding using neural signals have been achieved in recent years, but the complex dynamics are still not fully understood. However, it is unlikely that simple linear models can capture the relation between neural activity and continuous spoken speech. APPROACH: Here we show that deep neural networks can be used to map ECoG from speech production areas onto an intermediate representation of speech (logMel spectrogram). The proposed method uses a densely connected convolutional neural network topology which is well-suited to work with the small amount of data available from each participant. MAIN RESULTS: In a study with six participants, we achieved correlations up to r = 0.69 between the reconstructed and original logMel spectrograms. We transfered our prediction back into an audible waveform by applying a Wavenet vocoder. The vocoder was conditioned on logMel features that harnessed a much larger, pre-existing data corpus to provide the most natural acoustic output. SIGNIFICANCE: To the best of our knowledge, this is the first time that high-quality speech has been reconstructed from neural recordings during speech production using deep neural networks.

The effect of myoelectric computer interface training on arm kinematics and function after stroke.

Abnormal co-activation patterns of arm muscles is a substantial cause of impaired arm function after stroke. We designed a myoelectric computer interface (MCI) training paradigm to help stroke survivors reduce this abnormal coactivation. Here, we evaluated the effects of MCI training on function and arm kinematics in 32 chronic stroke survivors. We compared the effects of training duration and isometric vs. movement-based training conditions in 3 different groups. All groups reduced abnormal co-activation in targeted muscles, and showed reduced arm impairment after 6 weeks of training. They also showed improvements in arm kinematics as well as functional scores. Moreover, the gains persisted, though most were reduced, at one month after training stopped. These results suggest that MCI training holds promise to improve arm function after stroke.

Differential Representation of Articulatory Gestures and Phonemes in Precentral and Inferior Frontal Gyri.

Speech is a critical form of human communication and is central to our daily lives. Yet, despite decades of study, an understanding of the fundamental neural control of speech production remains incomplete. Current theories model speech production as a hierarchy from sentences and phrases down to words, syllables, speech sounds (phonemes), and the actions of vocal tract articulators used to produce speech sounds (articulatory gestures). Here, we investigate the cortical representation of articulatory gestures and phonemes in ventral precentral and inferior frontal gyri in men and women. Our results indicate that ventral precentral cortex represents gestures to a greater extent than phonemes, while inferior frontal cortex represents both gestures and phonemes. These findings suggest that speech production shares a common cortical representation with that of other types of movement, such as arm and hand movements. This has important implications both for our understanding of speech production and for the design of brain-machine interfaces to restore communication to people who cannot speak.SIGNIFICANCE STATEMENT Despite being studied for decades, the production of speech by the brain is not fully understood. In particular, the most elemental parts of speech, speech sounds (phonemes) and the movements of vocal tract articulators used to produce these sounds (articulatory gestures), have both been hypothesized to be encoded in motor cortex. Using direct cortical recordings, we found evidence that primary motor and premotor cortices represent gestures to a greater extent than phonemes. Inferior frontal cortex (part of Broca's area) appears to represent both gestures and phonemes. These findings suggest that speech production shares a similar cortical organizational structure with the movement of other body parts.

Identification of the S100 fused-type protein hornerin as a regulator of tumor vascularity.

Sustained angiogenesis is essential for the development of solid tumors and metastatic disease. Disruption of signaling pathways that govern tumor vascularity provide a potential avenue to thwart cancer progression. Through phage display-based functional proteomics, immunohistochemical analysis of human pancreatic ductal carcinoma (PDAC) specimens, and in vitro validation, we reveal that hornerin, an S100 fused-type protein, is highly expressed on pancreatic tumor endothelium in a vascular endothelial growth factor (VEGF)-independent manner. Murine-specific hornerin knockdown in PDAC xenografts results in tumor vessels with decreased radii and tortuosity. Hornerin knockdown tumors have significantly reduced leakiness, increased oxygenation, and greater apoptosis. Additionally, these tumors show a significant reduction in growth, a response that is further heightened when therapeutic inhibition of VEGF receptor 2 (VEGFR2) is utilized in combination with hornerin knockdown. These results indicate that hornerin is highly expressed in pancreatic tumor endothelium and alters tumor vessel parameters through a VEGF-independent mechanism.Angiogenesis is essential for solid tumor progression. Here, the authors interrogate the proteome of pancreatic cancer endothelium via phage display and identify hornerin as a critical protein whose expression is essential to maintain the pancreatic cancer vasculature through a VEGF-independent mechanism.

Evaluation of pharmacokinetic and pharmacodynamic profiles of liposomes for the cell type-specific delivery of small molecule drugs.

Liposome-based drug formulations represent an exciting avenue of research as they increase efficacy to toxicity ratios. Current formulations rely on passive accumulation to the disease site where drug is taken up by the cells. Ligand mediated targeting increases the net accumulation of liposomes, however, an unexplored benefit is to potentially refine pharmacodynamics (PD) of a drug specifically to different cell types within diseased tissue. As a model system, we engineered cardiomyocyte- (I-1) and endothelial-targeted (B-40) liposomes to carry a VEGFR2 inhibitor (PTK787), and examined the effect of cell type-specific delivery on both pharmacokinetics (PK) and PD. Neovascularization in post-myocardial infarction was significantly reduced by B-40 liposomes loaded with PTK787 as compared to animals injected with I-1 liposomes, and profoundly more as compared to free PTK787. This study thus shows that the intraorgan targeting of drugs through cell type-specific delivery holds substantial promise towards lowering the minimal efficacious dose administered systemically.

Corneal Potential Maps Measured With Multi-Electrode Electroretinography in Rat Eyes With Experimental Lesions.

Purpose: Conventional full-field flash electroretinography (ERG) yields a single response waveform that can be useful in the early detection and diagnosis of many diseases affecting the retina. It is an objective measurement that probes the entire retina. However, localized areas of dysfunction have relatively small influence on ERG amplitudes compared to normal ranges. Here we evaluate the use of corneal potential maps obtained in response to full-field flash stimuli for sensitivity to local areas of retinal damage. Methods: A contact lens electrode array was used to record 25 ERG waveforms simultaneously following saturating full-field flash stimuli (multi-electrode electroretinography, meERG) in rats. Waveforms were evaluated for a-wave and b-wave amplitudes; these values were normalized and further evaluated for spatial differences across the corneal surface. Cluster analysis and a support vector machine approach were used to classify meERG responses from healthy eyes and eyes with central (photocoagulation) or peripheral (cryocoagulation) experimental lesions. Results: A normative normalized corneal potential map was obtained from healthy eyes (n = 26). Corneal potential maps from eyes with experimental lesions (n = 13) could be classified with sensitivity and specificity of approximately 80% based solely on the normalized spatial distribution of corneal potentials, that is, with no knowledge of absolute amplitudes. Conclusions: Corneal potential maps obtained in response to full-field flash stimuli are altered in eyes with scotomas in the central and far-peripheral retina. The meERG approach yields useful spatial information following a single brief flash, analogous to body-surface potential maps used to evaluate heart and brain.

Decoding of articulatory gestures during word production using speech motor and premotor cortical activity.

Brain-machine interfaces that directly translate attempted speech from the speech motor areas could change the lives of people with complete paralysis. However, it remains uncertain exactly how speech production is encoded in cortex. Improving this understanding could greatly improve brain-machine interface design. Specifically, it is not clear to what extent the different levels of speech production (phonemes, or speech sounds, and articulatory gestures, which describe the movements of the articulator muscles) are represented in the motor cortex. Using electrocorticographic (ECoG) electrodes on the cortical surface, we recorded neural activity from speech motor and premotor areas during speech production. We decoded both gestures and phonemes using the neural signals. Overall classification accuracy was higher for gestures than phonemes. In particular, gestures were better represented in the primary sensorimotor cortices, while phonemes were better represented in more anterior areas.

Direct classification of all American English phonemes using signals from functional speech motor cortex.

OBJECTIVE: Although brain-computer interfaces (BCIs) can be used in several different ways to restore communication, communicative BCI has not approached the rate or efficiency of natural human speech. Electrocorticography (ECoG) has precise spatiotemporal resolution that enables recording of brain activity distributed over a wide area of cortex, such as during speech production. In this study, we sought to decode elements of speech production using ECoG. APPROACH: We investigated words that contain the entire set of phonemes in the general American accent using ECoG with four subjects. Using a linear classifier, we evaluated the degree to which individual phonemes within each word could be correctly identified from cortical signal. MAIN RESULTS: We classified phonemes with up to 36% accuracy when classifying all phonemes and up to 63% accuracy for a single phoneme. Further, misclassified phonemes follow articulation organization described in phonology literature, aiding classification of whole words. Precise temporal alignment to phoneme onset was crucial for classification success. SIGNIFICANCE: We identified specific spatiotemporal features that aid classification, which could guide future applications. Word identification was equivalent to information transfer rates as high as 3.0 bits s(-1) (33.6 words min(-1)), supporting pursuit of speech articulation for BCI control.

Cortical encoding of phonemic context during word production.

Brain-computer interfaces that directly decode speech could restore communication to locked-in individuals. However, decoding speech from brain signals still faces many challenges. We investigated decoding of phonemes - the smallest separable parts of speech - from ECoG signals during word production. We expanded on previous efforts to identify specific phoneme by identifying phonemes by where in the word they were formed. We evaluated how the context of phonemes in words affects classification results using linear discriminant analysis. The decoding accuracy of our linear classifier indicated the degree to which the context of a phoneme can be determined from the cortical signal significantly greater than chance. Further, we identified the spectrotemporal features that contributed most to successful decoding of phonemic classes. Finally, we discuss how this can augment speech decoding for neural interfaces.

Design and implementation of a P300-based brain-computer interface for controlling an internet browser.

An electroencephalographic (EEG) brain-computer interface (BCI) internet browser was designed and evaluated with 10 healthy volunteers and three individuals with advanced amyotrophic lateral sclerosis (ALS), all of whom were given tasks to execute on the internet using the browser. Participants with ALS achieved an average accuracy of 73% and a subsequent information transfer rate (ITR) of 8.6 bits/min and healthy participants with no prior BCI experience over 90% accuracy and an ITR of 14.4 bits/min. We define additional criteria for unrestricted internet access for evaluation of the presented and future internet browsers, and we provide a review of the existing browsers in the literature. The P300-based browser provides unrestricted access and enables free web surfing for individuals with paralysis.