An adaptive closed-loop ECoG decoder for long-term and stable bimanual control of an exoskeleton by a tetraplegic.

Objective.The article aims at addressing 2 challenges to step motor brain-computer interface (BCI) out of laboratories: asynchronous control of complex bimanual effectors with large numbers of degrees of freedom, using chronic and safe recorders, and the decoding performance stability over time without frequent decoder recalibration.Approach.Closed-loop adaptive/incremental decoder training is one strategy to create a model stable over time. Adaptive decoders update their parameters with new incoming data, optimizing the model parameters in real time. It allows cross-session training with multiple recording conditions during closed loop BCI experiments. In the article, an adaptive tensor-based recursive exponentially weighted Markov-switching multi-linear model (REW-MSLM) decoder is proposed. REW-MSLM uses a mixture of expert (ME) architecture, mixing or switching independent decoders (experts) according to the probability estimated by a 'gating' model. A Hidden Markov model approach is employed as gating model to improve the decoding robustness and to provide strong idle state support. The ME architecture fits the multi-limb paradigm associating an expert to a particular limb or action.Main results.Asynchronous control of an exoskeleton by a tetraplegic patient using a chronically implanted epidural electrocorticography (EpiCoG) recorder is reported. The stable over a period of six months (without decoder recalibration) eight-dimensional alternative bimanual control of the exoskeleton and its virtual avatar is demonstrated.Significance.Based on the long-term (>36 months) chronic bilateral EpiCoG recordings in a tetraplegic (ClinicalTrials.gov, NCT02550522), we addressed the poorly explored field of asynchronous bimanual BCI. The new decoder was designed to meet to several challenges: the high-dimensional control of a complex effector in experiments closer to real-world behavior (point-to-point pursuit versus conventional center-out tasks), with the ability of the BCI system to act as a stand-alone device switching between idle and control states, and a stable performance over a long period of time without decoder recalibration.

Long-term stability of the chronic epidural wireless recorder WIMAGINE in tetraplegic patients.

Objective.The evaluation of the long-term stability of ElectroCorticoGram (ECoG) signals is an important scientific question as new implantable recording devices can be used for medical purposes such as Brain-Computer Interface (BCI) or brain monitoring.Approach.The long-term clinical validation of wireless implantable multi-channel acquisition system for generic interface with neurons (WIMAGINE), a wireless 64-channel epidural ECoG recorder was investigated. The WIMAGINE device was implanted in two quadriplegic patients within the context of a BCI protocol. This study focused on the ECoG signal stability in two patients bilaterally implanted in June 2017 (P1) and in November 2019 (P2).Methods. The ECoG signal was recorded at rest prior to each BCI session resulting in a 32 month and in a 14 month follow-up for P1 and P2 respectively. State-of-the-art signal evaluation metrics such as root mean square (RMS), the band power (BP), the signal to noise ratio (SNR), the effective bandwidth (EBW) and the spectral edge frequency (SEF) were used to evaluate stability of signal over the implantation time course. The time-frequency maps obtained from task-related motor activations were also studied to investigate the long-term selectivity of the electrodes.Mainresults.Based on temporal linear regressions, we report a limited decrease of the signal average level (RMS), spectral distribution (BP) and SNR, and a remarkable steadiness of the EBW and SEF. Time-frequency maps obtained during motor imagery, showed a high level of discrimination 1 month after surgery and also after 2 years.Conclusions.The WIMAGINE epidural device showed high stability over time. The signal evaluation metrics of two quadriplegic patients during 32 months and 14 months respectively provide strong evidence that this wireless implant is well-suited for long-term ECoG recording.Significance.These findings are relevant for the future of implantable BCIs, and could benefit other patients with spinal cord injury, amyotrophic lateral sclerosis, neuromuscular diseases or drug-resistant epilepsy.

The Riemannian spatial pattern method: mapping and clustering movement imagery using Riemannian geometry.

Objective. Over the last decade, Riemannian geometry has shown promising results for motor imagery classification. However, extracting the underlying spatial features is not as straightforward as for applying common spatial pattern (CSP) filtering prior to classification. In this article, we propose a simple way to extract the spatial patterns obtained from Riemannian classification: the Riemannian spatial pattern (RSP) method, which is based on the backward channel selection procedure.Approach. The RSP method was compared to the CSP approach on ECoG data obtained from a quadriplegic patient while performing imagined movements of arm articulations and fingers.Main results.Similar results were found between the RSP and CSP methods for mapping each motor imagery task with activations following the classical somatotopic organization. Clustering obtained by pairwise comparisons of imagined motor movements however, revealed higher differentiation for the RSP method compared to the CSP approach. Importantly, the RSP approach could provide a precise comparison of the imagined finger flexions which added supplementary information to the mapping results.Significance.Our new RSP method illustrates the interest of the Riemannian framework in the spatial domain and as such offers new avenues for the neuroimaging community. This study is part of an ongoing clinical trial registered with ClinicalTrials.gov, NCT02550522.

Space-Time-Frequency Multi-Sensor Analysis for Motor Cortex Localization Using Magnetoencephalography.

Brain source imaging and time frequency mapping (TFM) are commonly used in magneto/electro encephalography (M/EEG) imaging. However, these methods suffer from important limitations. Source imaging is based on an ill-posed inverse problem leading to instability of source localization solutions, has a limited capacity to localize high frequency oscillations and loses its robustness for induced responses (ill-defined trigger). The drawback of TFM is that it involves independent analysis of signals from a number of frequency bands, and from co-localized sensors. In the present article, a regression-based multi-sensor space-time-frequency analysis (MSA) approach, which integrates co-localized sensors and/or multi-frequency information, is proposed. To estimate task-specific brain activations, MSA uses cross-validated, shifted, multiple Pearson correlation, calculated from the time-frequency transformed brain signal and the binary signal of stimuli. The results are projected from the sensor space onto the cortical surface. To assess MSA performance, the proposed method was compared to the weighted minimum norm estimate (wMNE) source imaging method, in terms of spatial selectivity and robustness against an ill-defined trigger. Magnetoencephalography (MEG) recordings were performed in fourteen subjects during two motor tasks: finger tapping and elbow flexion/extension. In particular, our results show that the MSA approach provides good localization performance when compared to wMNE and statistically significant improvement of robustness against ill-defined trigger.

Tracking Your Mind's Eye during Recollection: Decoding the Long-Term Recall of Short Audiovisual Clips.

Unlike familiarity, recollection involves the ability to reconstruct mentally previous events that results in a strong sense of reliving. According to the reinstatement hypothesis, this specific feature emerges from the reactivation of cortical patterns involved during information exposure. Over time, the retrieval of specific details becomes more difficult, and memories become increasingly supported by familiarity judgments. The multiple trace theory (MTT) explains the gradual loss of episodic details by a transformation in the memory representation, a view that is not shared by the standard consolidation model. In this study, we tested the MTT in light of the reinstatement hypothesis. The temporal dynamics of mental imagery from long-term memory were investigated and tracked over the passage of time. Participant EEG activity was recorded during the recall of short audiovisual clips that had been watched 3 weeks, 1 day, or a few hours beforehand. The recall of the audiovisual clips was assessed using a Remember/Know/New procedure, and snapshots of clips were used as recall cues. The decoding matrices obtained from the multivariate pattern analyses revealed sustained patterns that occurred at long latencies (>500 msec poststimulus onset) that faded away over the retention intervals and that emerged from the same neural processes. Overall, our data provide further evidence toward the MTT and give new insights into the exploration of our "mind's eye."

Extremely long-term memory and familiarity after 12 years.

In 2006 Mitchell demonstrated that implicit memory was robust to decay. He showed that the ability to identify fragments of pictures seen 17 years before was significantly higher than for new stimuli. Is this true only for implicit memory? In this study, we tested whether explicit memory was still possible for drawings (n = 144) that had been presented once or three times, two seconds each time on average, approximately 12 years earlier. Surprisingly, our data reveal that our participants were able to recognize pictures above chance level. Preserved memory was mainly observed in the youngest subjects, for stimuli seen three times. Despite the fact that confidence judgments were low, reports suggest that recognition could be based on a strong sense of familiarity. These data extend Mitchell's findings and show that familiarity can also be robust to decay.

Waking Up Buried Memories of Old TV Programs.

Although it has been demonstrated that visual and auditory stimuli can be recalled decades after the initial exposure, previous studies have generally not ruled out the possibility that the material may have been seen or heard during the intervening period. Evidence shows that reactivations of a long-term memory trace play a role in its update and maintenance. In the case of remote or very long-term memories, it is most likely that these reactivations are triggered by the actual re-exposure to the stimulus. In this study we decided to explore whether it is possible to recall stimuli that could not have been re-experienced in the intervening period. We tested the ability of French participants (N = 34, 31 female) to recall 50 TV programs broadcast on average for the last time 44 years ago (from the 60's and early 70's). Potential recall was elicited by the presentation of short audiovisual excerpts of these TV programs. The absence of potential re-exposure to the material was strictly controlled by selecting TV programs that have never been rebroadcast and were not available in the public domain. Our results show that six TV programs were particularly well identified on average across the 34 participants with a median percentage of 71.7% (SD = 13.6, range: 48.5-87.9%). We also obtained 50 single case reports with associated information about the viewing of 23 TV programs including the 6 previous ones. More strikingly, for two cases, retrieval of the title was made spontaneously without the need of a four-proposition choice. These results suggest that re-exposures to the stimuli are not necessary to maintain a memory for a lifetime. These new findings raise fundamental questions about the underlying mechanisms used by the brain to store these very old sensory memories.