Classification of Articulator Movements and Movement Direction from Sensorimotor Cortex Activity.

For people suffering from severe paralysis, communication can be difficult or nearly impossible. Technology systems called brain-computer interfaces (BCIs) are being developed to assist these people with communication by using their brain activity to control a computer without any muscle activity. To benefit the development of BCIs that employ neural activity related to speech, we investigated if neural activity patterns related to different articulator movements can be distinguished from each other. We recorded with electrocorticography (ECoG), the neural activity related to different articulator movements in 4 epilepsy patients and classified which articulator participants moved based on the sensorimotor cortex activity patterns. The same was done for different movement directions of a single articulator, the tongue. In both experiments highly accurate classification was obtained, on average 92% for different articulators and 85% for different tongue directions. Furthermore, the data show that only a small part of the sensorimotor cortex is needed for classification (ca. 1 cm2). We show that recordings from small parts of the sensorimotor cortex contain information about different articulator movements which might be used for BCI control. Our results are of interest for BCI systems that aim to decode neural activity related to (actual or attempted) movements from a contained cortical area.

Repeated Vowel Production Affects Features of Neural Activity in Sensorimotor Cortex.

The sensorimotor cortex is responsible for the generation of movements and interest in the ability to use this area for decoding speech by brain-computer interfaces has increased recently. Speech decoding is challenging however, since the relationship between neural activity and motor actions is not completely understood. Non-linearity between neural activity and movement has been found for instance for simple finger movements. Despite equal motor output, neural activity amplitudes are affected by preceding movements and the time between movements. It is unknown if neural activity is also affected by preceding motor actions during speech. We addressed this issue, using electrocorticographic high frequency band (HFB; 75-135 Hz) power changes in the sensorimotor cortex during discrete vowel generation. Three subjects with temporarily implanted electrode grids produced the /i/ vowel at repetition rates of 1, 1.33 and 1.66 Hz. For every repetition, the HFB power amplitude was determined. During the first utterance, most electrodes showed a large HFB power peak, which decreased for subsequent utterances. This result could not be explained by differences in performance. With increasing duration between utterances, more electrodes showed an equal response to all repetitions, suggesting that the duration between vowel productions influences the effect of previous productions on sensorimotor cortex activity. Our findings correspond with previous studies for finger movements and bear relevance for the development of brain-computer interfaces that employ speech decoding based on brain signals, in that past utterances will need to be taken into account for these systems to work accurately.

The influence of prior pronunciations on sensorimotor cortex activity patterns during vowel production.

OBJECTIVE: In recent years, brain-computer interface (BCI) systems have been investigated for their potential as a communication device to assist people with severe paralysis. Decoding speech sensorimotor cortex activity is a promising avenue for the generation of BCI control signals, but is complicated by variability in neural patterns, leading to suboptimal decoding. We investigated whether neural pattern variability associated with sound pronunciation can be explained by prior pronunciations and determined to what extent prior speech affects BCI decoding accuracy. APPROACH: Neural patterns in speech motor areas were evaluated with electrocorticography in five epilepsy patients, who performed a simple speech task that involved pronunciation of the /i/ sound, preceded by either silence, the /a/ sound or the /u/ sound. MAIN RESULTS: The neural pattern related to the /i/ sound depends on previous sounds and is therefore associated with multiple distinct sensorimotor patterns, which is likely to reflect differences in the movements towards this sound. We also show that these patterns still contain a commonality that is distinct from the other vowel sounds (/a/ and /u/). Classification accuracies for the decoding of different sounds do increase, however, when the multiple patterns for the /i/ sound are taken into account. Simply including multiple forms of the /i/ vowel in the training set for the creation of a single /i/ model performs as well as training individual models for each /i/ variation. SIGNIFICANCE: Our results are of interest for the development of BCIs that aim to decode speech sounds from the sensorimotor cortex, since they argue that a multitude of cortical activity patterns associated with speech movements can be reduced to a basis set of models which reflect meaningful language units (vowels), yet it is important to account for the variety of neural patterns associated with a single sound in the training process.

Spatial-Temporal Dynamics of the Sensorimotor Cortex: Sustained and Transient Activity.

How the sensorimotor cortex is organized with respect to controlling different features of movement is unclear. One unresolved question concerns the relation between the duration of an action and the duration of the associated neuronal activity change in the sensorimotor cortex. Using subdural electrocorticography electrodes, we investigated in five subjects, whether high frequency band (HFB; 75-135 Hz) power changes have a transient or sustained relation to speech duration, during pronunciation of the Dutch /i/ vowel with different durations. We showed that the neuronal activity patterns recorded from the sensorimotor cortex can be directly related to action duration in some locations, whereas in other locations, during the same action, neuronal activity is transient, with a peak in HFB activity at movement onset and/or offset. This data sheds light on the neural underpinnings of motor actions and we discuss the possible mechanisms underlying these different response types.

Decoding spoken phonemes from sensorimotor cortex with high-density ECoG grids.

For people who cannot communicate due to severe paralysis or involuntary movements, technology that decodes intended speech from the brain may offer an alternative means of communication. If decoding proves to be feasible, intracranial Brain-Computer Interface systems can be developed which are designed to translate decoded speech into computer generated speech or to instructions for controlling assistive devices. Recent advances suggest that such decoding may be feasible from sensorimotor cortex, but it is not clear how this challenge can be approached best. One approach is to identify and discriminate elements of spoken language, such as phonemes. We investigated feasibility of decoding four spoken phonemes from the sensorimotor face area, using electrocorticographic signals obtained with high-density electrode grids. Several decoding algorithms including spatiotemporal matched filters, spatial matched filters and support vector machines were compared. Phonemes could be classified correctly at a level of over 75% with spatiotemporal matched filters. Support Vector machine analysis reached a similar level, but spatial matched filters yielded significantly lower scores. The most informative electrodes were clustered along the central sulcus. Highest scores were achieved from time windows centered around voice onset time, but a 500 ms window before onset time could also be classified significantly. The results suggest that phoneme production involves a sequence of robust and reproducible activity patterns on the cortical surface. Importantly, decoding requires inclusion of temporal information to capture the rapid shifts of robust patterns associated with articulator muscle group contraction during production of a phoneme. The high classification scores are likely to be enabled by the use of high density grids, and by the use of discrete phonemes. Implications for use in Brain-Computer Interfaces are discussed.

Classification of mouth movements using 7 T fMRI.

OBJECTIVE: A brain-computer interface (BCI) is an interface that uses signals from the brain to control a computer. BCIs will likely become important tools for severely paralyzed patients to restore interaction with the environment. The sensorimotor cortex is a promising target brain region for a BCI due to the detailed topography and minimal functional interference with other important brain processes. Previous studies have shown that attempted movements in paralyzed people generate neural activity that strongly resembles actual movements. Hence decodability for BCI applications can be studied in able-bodied volunteers with actual movements. APPROACH: In this study we tested whether mouth movements provide adequate signals in the sensorimotor cortex for a BCI. The study was executed using fMRI at 7 T to ensure relevance for BCI with cortical electrodes, as 7 T measurements have been shown to correlate well with electrocortical measurements. Twelve healthy volunteers executed four mouth movements (lip protrusion, tongue movement, teeth clenching, and the production of a larynx activating sound) while in the scanner. Subjects performed a training and a test run. Single trials were classified based on the Pearson correlation values between the activation patterns per trial type in the training run and single trials in the test run in a 'winner-takes-all' design. MAIN RESULTS: Single trial mouth movements could be classified with 90% accuracy. The classification was based on an area with a volume of about 0.5 cc, located on the sensorimotor cortex. If voxels were limited to the surface, which is accessible for electrode grids, classification accuracy was still very high (82%). Voxels located on the precentral cortex performed better (87%) than the postcentral cortex (72%). SIGNIFICANCE: The high reliability of decoding mouth movements suggests that attempted mouth movements are a promising candidate for BCI in paralyzed people.

SU-E-T-627: Optimal Partial Arcs in VMAT Planning.

PURPOSE: To describe a method for producing minimal delivery time partial arc VMAT plans. METHODS: We begin with the assumption that dose quality is the primary treatment planning goal. Therefore the first step in the partial arc computation is a 180 beam equi-spaced IMRT multi-criteria optimized treatment plan, which serves as an ideal plan, along with a set of user- specified allowable deviations from this plan. This defines a set of target coverage and healthy organ sparing constraints. We then seek a partial arc plan which recovers this ideal plan but is minimal in delivery time. The search for the optimal partial arc which fulfills the hard constraints is done by wrapping a VMAT fluence map optimization/merging/simplification algorithm called VMERGE. The search is performed over all possible partial arcs, with start and end locations discretized to 20 degree increments, and respecting that the gantry cannot pass underneath the couch. This results in 169 partial arcs. For the ones that yield feasible plans, the complete VMERGE algorithm is run, which minimizes the delivery time for that arc. The minimal delivery time plan that fulfills the dosimetric requirements is returned. RESULTS: We apply the method to a lung and liver case. The time savings are as follows: (full arc time, optimal partial arc time): lung (185 s, 94 s), liver (263 s, 165 s). The optimal arc for the lung lesion, a left anterior target, is 140 degrees centered at 50 degrees. The optimal arc for the liver lesion is 160 degrees centered at -90 degrees. CONCLUSIONS: By wrapping a fast VMAT optimization/sequencing routine by an exhaustive search over 169 possible partial arcs, we are able to determine the fastest delivery partial arc. The use of partial arcs can significantly shorten delivery time in VMAT delivery. The project described was supported by Award Number R01CA103904 from the National Cancer Institute. The content is solely the responsibility of the authors and does not necessarily represent the ocial views of the National Cancer Institute or the National Institutes of Health.

SU-E-T-619: A Network-Flow Solution Approach to VMAT Treatment Plan Optimization.

PURPOSE: To add mathematical rigor to the merging phase of the recently published two-stage VMAT optimization method called VMERGE. Using an exact merging method, we are able to better characterize the tradeoff between delivery efficiency and dose quality. METHODS: VMERGE begins with an IMRT plan that uses 180 equi-spaced beams and yields the "ideal" dose. Neighboring fluence maps are successively merged, meaning they are added together and delivered as one map. The merging process improves the delivery time at the expense of deviating from the initial high-quality dose distribution. We replace the original heuristic merging method by considering the merging problem as a bi-criteria optimization problem: maximize treatment efficiency and minimize the deviation from the ideal dose. We formulate this using a network-flow model where nodes represent the beam angles along with the starting MLC leaf position and arcs represent the possible merges. Since the problem is non-convex, we employ a customized box algorithm to obtain the Pareto approximation. We also evaluate the performance of several simple heuristics. RESULTS: We test our exact and heuristic solution approaches on a pancreas and a prostate case. For both cases, the shape of the Pareto frontier suggests that starting from a high quality plan, we can obtain efficient VMAT plans through merging neighboring arcs without substantially deviating from the initial dose distribution. The trade-off curves obtained by the various heuristics are contrasted and shown to all be equally capable of initial plan simplifications, but to deviate in quality for more drastic efficiency improvements. CONCLUSIONS: This work presents a bi-criteria network-flow solution approach to the merging problem. The obtained Pareto-frontier approximation is used as a benchmark to evaluate the performance of the proposed merging heuristics. The results validate that one of the heuristics in particular can achieve high-quality solutions.

Successive elimination algorithm for motion estimation.

The correspondence presents a fast exhaustive search algorithm for motion estimation. The basic idea is to obtain the best estimate of the motion vectors by successively eliminating the search positions in the search window and thus decreasing the number of matching evaluations that require very intensive computations. Simulation results demonstrate that although the performance of the proposed algorithm is the same as that using the exhaustive search, the computation time has been reduced significantly.