Exploring Self-Paced Embodiable Neurofeedback for Post-stroke Motor Rehabilitation.

Neurofeedback-guided motor-imagery training (NF-MIT) has been proposed as a promising intervention following upper limb motor impairment. In this intervention, paretic stroke patients receive online feedback about their brain activity while conducting a motor-imagery (MI) task with the paretic limb. Typically, the feedback provided in NF-MIT protocols is an abstract visual signal based on a fixed trial. Here we developed a self-paced NF-MIT paradigm with an embodiable feedback signal (EFS), which was designed to resemble the content of the mental act as closely as possible. To this end, the feedback was delivered via an embodiable, anthropomorphic robotic hand (RH), which was integrated into a closed-looped EEG-based brain-computer interface (BCI). Whenever the BCI identified a new instance of a hand-flexion or hand-extension imagination by the participant, the RH carried out the corresponding movement with minimum delay. Nine stroke patients and nine healthy participants were instructed to control RH movements as accurately as possible, using mental activity alone. We evaluated the general feasibility of our paradigm on electrophysiological, subjective and performance levels. Regarding electrophysiological measures, individuals showed the predicted event-related desynchronization (ERD) patterns over sensorimotor brain areas. On the subjective level, we found that most individuals integrated the RH into their body scheme. With respect to RH control, none of our participants achieved a high level of control, but most managed to control the RH actions to some degree. Importantly, patients and controls achieved similar performance levels. The results support the view that self-paced embodiable NF-MIT is feasible for stroke patients and can complement classical NF-MIT.

Dynamic phase alignment of ongoing auditory cortex oscillations.

Neural oscillations can synchronize to external rhythmic stimuli, as for example in speech and music. While previous studies have mainly focused on elucidating the fundamental concept of neural entrainment, less is known about the time course of entrainment. In this human electroencephalography (EEG) study, we unravel the temporal evolution of neural entrainment by contrasting short and long periods of rhythmic stimulation. Listeners had to detect short silent gaps that were systematically distributed with respect to the phase of a 3 Hz frequency-modulated tone. We found that gap detection performance was modulated by the stimulus stream with a consistent stimulus phase across participants for short and long stimulation. Electrophysiological analysis confirmed neural entrainment effects at 3 Hz and the 6 Hz harmonic for both short and long stimulation lengths. 3 Hz source level analysis revealed that longer stimulation resulted in a phase shift of a participant's neural phase relative to the stimulus phase. Phase coupling increased over the first second of stimulation, but no effects for phase coupling strength were observed over time. The dynamic evolution of phase alignment suggests that the brain attunes to external rhythmic stimulation by adapting the brain's internal representation of incoming environmental stimuli.

Embodied neurofeedback with an anthropomorphic robotic hand.

Neurofeedback-guided motor imagery training (NF-MIT) has been suggested as a promising therapy for stroke-induced motor impairment. Whereas much NF-MIT research has aimed at signal processing optimization, the type of sensory feedback given to the participant has received less attention. Often the feedback signal is highly abstract and not inherently coupled to the mental act performed. In this study, we asked whether an embodied feedback signal is more efficient for neurofeedback operation than a non-embodiable feedback signal. Inspired by the rubber hand illusion, demonstrating that an artificial hand can be incorporated into one's own body scheme, we used an anthropomorphic robotic hand to visually guide the participants' motor imagery act and to deliver neurofeedback. Using two experimental manipulations, we investigated how a participant's neurofeedback performance and subjective experience were influenced by the embodiability of the robotic hand, and by the neurofeedback signal's validity. As pertains to embodiment, we found a promoting effect of robotic-hand embodiment in subjective, behavioral, electrophysiological and electrodermal measures. Regarding neurofeedback signal validity, we found some differences between real and sham neurofeedback in terms of subjective and electrodermal measures, but not in terms of behavioral and electrophysiological measures. This study motivates the further development of embodied feedback signals for NF-MIT.

Cross-Modal Functional Reorganization of Visual and Auditory Cortex in Adult Cochlear Implant Users Identified with fNIRS.

Cochlear implant (CI) users show higher auditory-evoked activations in visual cortex and higher visual-evoked activation in auditory cortex compared to normal hearing (NH) controls, reflecting functional reorganization of both visual and auditory modalities. Visual-evoked activation in auditory cortex is a maladaptive functional reorganization whereas auditory-evoked activation in visual cortex is beneficial for speech recognition in CI users. We investigated their joint influence on CI users' speech recognition, by testing 20 postlingually deafened CI users and 20 NH controls with functional near-infrared spectroscopy (fNIRS). Optodes were placed over occipital and temporal areas to measure visual and auditory responses when presenting visual checkerboard and auditory word stimuli. Higher cross-modal activations were confirmed in both auditory and visual cortex for CI users compared to NH controls, demonstrating that functional reorganization of both auditory and visual cortex can be identified with fNIRS. Additionally, the combined reorganization of auditory and visual cortex was found to be associated with speech recognition performance. Speech performance was good as long as the beneficial auditory-evoked activation in visual cortex was higher than the visual-evoked activation in the auditory cortex. These results indicate the importance of considering cross-modal activations in both visual and auditory cortex for potential clinical outcome estimation.

The auditory dynamic attending theory revisited: A closer look at the pitch comparison task.

The dynamic attending theory as originally proposed by Jones, 1976. Psychol. Rev. 83(5), 323-355 posits that tone sequences presented at a regular rhythm entrain attentional oscillations and thereby facilitate the processing of sounds presented in phase with this rhythm. The increased interest in neural correlates of dynamic attending requires robust behavioral indicators of the phenomenon. Here we aimed to replicate and complement the most prominent experimental implementation of dynamic attending (Jones et al., 2002. Psychol. Sci. 13(4), 313-319). The paradigm uses a pitch comparison task in which two tones, the initial and the last of a longer series, have to be compared. In-between the two, distractor tones with variable pitch are presented, at a regular pace. A comparison tone presented in phase with the entrained rhythm is hypothesized to lead to better behavioral performance. Aiming for a conceptual replication, four different variations of the original paradigm were created which were followed by an exact replication attempt. Across all five experiments, only 40 of the 140 tested participants showed the hypothesized pattern of an inverted U-shaped profile in task accuracy, and the group average effects did not replicate the pattern reported by Jones et al., 2002. Psychol. Sci. 13(4), 313-319 in any of the five experiments. However, clear evidence for a relationship between musicality and overall behavioral performance was found. This study casts doubt on the suitability of the pitch comparison task for demonstrating auditory dynamic attending. We discuss alternative tasks that have been shown to support dynamic attending theory, thus lending themselves more readily to studying its neural correlates. This article is part of a Special Issue entitled SI: Prediction and Attention.

The beneficial effects of sounds on attentional blink performance: An ERP study.

Auditory information can affect the processing of visual information. The present study used event-related potentials (ERPs) to characterize the temporal dynamics of this interaction in a rapid serial visual presentation (RSVP) task that created an attentional blink (AB), that is, a profound deficit in detecting or identifying the second of two visual targets embedded in an RSVP stream. When a sound preceded the second target, T2, by 250ms or was played simultaneously with T2, T2 identification accuracy increased to a comparable degree. Sound-related modulations of the visual response to T2 emerged from around 140ms when the sound preceded T2, and from around 100ms when the sound was played simultaneously with T2. Timing and topography of this earliest modulation indicated that it is not related to early sensory areas, but suggest an attentional filter mechanism. Subsequent modulations are compatible with an improvement of early visual processing of T2 and of processes related to target selection both when the sound precedes and when it coincides with T2. For the latter condition our results also indicated processes related to a convergence of stimulus-driven and goal-directed attention. We also observed some overlap between neural correlates of sound-related modulations of the visual response and of correct vs. incorrect performance. Thus, external task manipulations and internal factors contributing to performance in the AB may rely on overlapping though not identical mechanisms.

Association of Concurrent fNIRS and EEG Signatures in Response to Auditory and Visual Stimuli.

Functional near-infrared spectroscopy (fNIRS) has been proven reliable for investigation of low-level visual processing in both infants and adults. Similar investigation of fundamental auditory processes with fNIRS, however, remains only partially complete. Here we employed a systematic three-level validation approach to investigate whether fNIRS could capture fundamental aspects of bottom-up acoustic processing. We performed a simultaneous fNIRS-EEG experiment with visual and auditory stimulation in 24 participants, which allowed the relationship between changes in neural activity and hemoglobin concentrations to be studied. In the first level, the fNIRS results showed a clear distinction between visual and auditory sensory modalities. Specifically, the results demonstrated area specificity, that is, maximal fNIRS responses in visual and auditory areas for the visual and auditory stimuli respectively, and stimulus selectivity, whereby the visual and auditory areas responded mainly toward their respective stimuli. In the second level, a stimulus-dependent modulation of the fNIRS signal was observed in the visual area, as well as a loudness modulation in the auditory area. Finally in the last level, we observed significant correlations between simultaneously-recorded visual evoked potentials and deoxygenated hemoglobin (DeoxyHb) concentration, and between late auditory evoked potentials and oxygenated hemoglobin (OxyHb) concentration. In sum, these results suggest good sensitivity of fNIRS to low-level sensory processing in both the visual and the auditory domain, and provide further evidence of the neurovascular coupling between hemoglobin concentration changes and non-invasive brain electrical activity.

Interplay of agency and ownership: the intentional binding and rubber hand illusion paradigm combined.

The sense of agency (SoA) refers to the phenomenal experience of initiating and controlling an action, whereas the sense of ownership (SoO) describes the feeling of myness an agent experiences towards his or her own body parts. SoA has been investigated with intentional binding paradigms, and the sense of ownership (SoO) with the rubber-hand illusion (RHI). We investigated the relationship between SoA and SoO by incorporating intentional binding into the RHI. Explicit and implicit measures of agency (SoA-questionnaire, intentional binding) and ownership (SoO-questionnaire, proprioceptive drift) were used. Artificial hand position (congruent/incongruent) and mode of agent (self-agent/other-agent) were systematically varied. Reported SoO varied mainly with position (higher in congruent conditions), but also with agent (higher in self-agent conditions). Reported SoA was modulated by agent (higher in self-agent conditions), and moderately by position (higher in congruent conditions). Implicit and explicit agency measures were not significantly correlated. Finally, intentional binding tended to be stronger in self-generated than observed voluntary actions. Results provide further evidence for a partial double dissociation between SoA and SoO, empirically distinct agency levels, and moderate intentional binding differences between self-generated and observed voluntary actions.

Neuroticism focuses attention: evidence from SSVEPs.

Neuroticism and negative affect have been associated with an increase in attentional investment and the greater processing of irrelevant stimuli. Previous research proposes the overinvestment of attention and a focused mental state as the mechanism of this effect. We investigated the neural correlates of this idea using a dual-stream rapid serial visual presentation paradigm with centrally presented, overlapping streams of letters that changed at different frequencies. Participants attended one stream at a time. We predicted that the more focused cognitive style associated with higher neuroticism would be reflected in the overinvestment of attention in the irrelevant stream of to-be-ignored letters, in particular, when the ignored stream was the more salient one. This was expected to lead to a smaller difference in power between the attended and unattended frequencies. Results showed that power differences between attended and unattended streams were negatively correlated with neuroticism scores in direct support of our hypothesis. Exploratory correlations also showed that extraversion was positively related to the attention difference. As extraversion has been contrasted to neuroticism and linked to increased cognitive flexibility and control in previous studies, it is possible that this trait may help in disengagement from salient stimuli. Together, these results provide the first neural correlates of the focused cognitive style idea. That the effect of extraversion is seen in the centro-parietal region and the effect of neuroticism is seen in the occipital region, indicate that these personality traits may affect the hierarchy of visual information processing. These findings provide new insight into the influence of personality traits on attention mechanisms and open up questions regarding the relationship between neuroticism, extraversion and information processing.

Endogenous and rapid serial visual presentation-induced alpha band oscillations in the attentional blink.

The attentional blink (AB) is a deficit in conscious perception of the second of two targets if it follows the first within 200-500 msec. The AB phenomenon has been linked to pre-target oscillatory alpha activity. However, this is based on paradigms that use a rapid serial visual presentation (RSVP) stimulus stream in which the targets are embedded. This distracter stream is usually presented at a frequency of 10 Hz and thus generates a steady-state visual-evoked potential (ssVEP) at the center of the alpha frequency band. This makes the interpretation of alpha findings in the AB difficult. To be able to relate these findings either to the presence of the ssVEP or to an effect of endogenously generated alpha activity, we compared AB paradigms with and without different pre-target distracter streams. The distracter stream was always presented at 12 Hz, and power and intertrial phase coherence were analyzed in the alpha range (8-12 Hz). Without a distracter stream alpha power dropped before target presentation, whereas coherence did not change. Presence of a distracter stream was linked to stronger pre-target power reduction and increased coherence, which were both modulated by distracter stream characteristics. With regard to the AB results indicated that, whereas ssVEP-related power tended to be higher when both targets were detected, endogenous alpha power tended to be lower. We argue that the pattern of results indicates that in the pre-target interval several processes act in parallel. The balance between these processes relates to the occurrence of an AB.

Source localisation of visual evoked potentials in congenitally deaf individuals.

Previous studies have suggested that individuals deprived of auditory input can compensate with specific superior abilities in the remaining sensory modalities. To better understand the neural basis of deafness-induced changes, the present study used electroencephalography to examine visual functions and cross-modal reorganization of the auditory cortex in deaf individuals. Congenitally deaf participants and hearing controls were presented with reversing chequerboard stimuli that were systematically modulated in luminance ratio. The two groups of participants showed similar modulation of visual evoked potential (VEP) amplitudes (N85, P110) and latencies (P110) as a function of luminance ratio. Analysis of VEPs revealed faster neural processing in deaf participants compared with hearing controls at early stages of cortical visual processing (N85). Deaf participants also showed higher amplitudes (P110) than hearing participants. In contrast to our expectations, the results from VEP source analysis revealed no clear evidence for cross-modal reorganization in the auditory cortex of deaf participants. However, deaf participants tended to show higher activation in posterior parietal cortex (PPC). Moreover, modulation of PPC responses as a function of luminance was also stronger in deaf than in hearing participants. Taken together, these findings are an indication of more efficient neural processing of visual information in the deaf, which may relate to functional changes, in particular in multisensory parietal cortex, as a consequence of early auditory deprivation.

Look now and hear what's coming: on the functional role of cross-modal phase reset.

In our multisensory environment our sensory systems are continuously receiving information that is often interrelated and must be integrated. Recent work in animals and humans has demonstrated that input to one sensory modality can reset the phase of ambient cortical oscillatory activity in another. The periodic fluctuations in neuronal excitability reflected in these oscillations can thereby be aligned to forthcoming anticipated sensory input. In the auditory domain, the example par excellence is speech, because of its inherently rhythmic structure. In contrast, fluctuations of oscillatory phase in the visual system are argued to reflect periodic sampling of the environment. Thus rhythmic structure is imposed on, rather than extracted from, the visual sensory input. Given this distinction, we suggest that cross-modal phase reset subserves separate functions in the auditory and visual systems. We propose a modality-dependent role for cross-modal input in temporal prediction whereby an auditory event signals the visual system to look now, but a visual event signals the auditory system that it needs to hear what is coming. This article is part of a Special Issue entitled .

Visuo-tactile interactions in the congenitally deaf: a behavioral and event-related potential study.

Auditory deprivation is known to be accompanied by alterations in visual processing. Yet not much is known about tactile processing and the interplay of the intact sensory modalities in the deaf. We presented visual, tactile, and visuo-tactile stimuli to congenitally deaf and hearing individuals in a speeded detection task. Analyses of multisensory responses showed a redundant signals effect that was attributable to a coactivation mechanism in both groups, although the redundancy gain was less in the deaf. In line with these behavioral results, on a neural level, there were multisensory interactions in both groups that were again weaker in the deaf. In hearing but not deaf participants, somatosensory event-related potential N200 latencies were modulated by simultaneous visual stimulation. A comparison of unisensory responses between groups revealed larger N200 amplitudes for visual and shorter N200 latencies for tactile stimuli in the deaf. Furthermore, P300 amplitudes were also larger in the deaf. This group difference was significant for tactile and approached significance for visual targets. The differences in visual and tactile processing between deaf and hearing participants, however, were not reflected in behavior. Both the behavioral and electroencephalography (EEG) results suggest more pronounced multisensory interaction in hearing than in deaf individuals. Visuo-tactile enhancements could not be explained by perceptual deficiency, but could be partly attributable to inverse effectiveness.

The influence of pretarget distracter stream properties on the skeletal attentional blink.

The attentional blink (AB) occurs when the limits of temporal processing are reached. In a typical AB experiment, two targets (T1, T2) are presented within a stream of distracters, and detection of T2 is impaired when presented shortly after T1. Several theories focus on the mechanism underlying this deficit, among them proposals that see the similarity between distracters and targets as a crucial factor. The present study aimed to gain a better understanding of the effect of distracter stream properties on performance in the AB paradigm. A skeletal AB task was combined with a pretarget distracter stream. The presence and familiarity of the distracter stream and its similarityto the targets were manipulated in three separate experiments. The last distracter before T1 was sufficient to impair T1 detection in the standard rapid serial visual presentation (RSVP) paradigm. Second, T2|T1 performance was impaired by an RSVP stream with low target-distracter similarity but unfamiliar stimuli. Finally, a single mask before T1 was shown to impair T2|T1. The results suggest that it is the last distracter before T1, rather than the RSVP stream per se, that impairs T1 detection performance. T2|T1 performance is influenced by a combination of transient attentional capture mechanisms that include but are not limited to target-distracter similarity. Thus, the size of an AB is determined not only by target-distracter similarity but by the overall processing demands of the distracters.

Simultaneous and preceding sounds enhance rapid visual targets: Evidence from the attentional blink.

Presenting two targets in a rapid visual stream will frequently result in the second target (T2) being missed when presented shortly after the first target (T1). This so-called attentional blink (AB) phenomenon can be reduced by various experimental manipulations. This study investigated the effect of combining T2 with a non-specific sound, played either simultaneously with T2 or preceding T2 by a fixed latency. The reliability of the observed effects and their correlation with potential predictors were studied. The tone significantly improved T2 identification rates regardless of tone condition and of the delay between targets, suggesting that the crossmodal facilitation of T2 identification is not limited to visual-perceptual enhancement. For the simultaneous condition, an additional time-on-task effect was observed in form of a reduction of the AB that occurred within an experimental session. Thus, audition-driven enhancement of visual perception may need some time for its full potential to evolve. Split-half and test-retest reliability were found consistently only for a condition without additional sound. AB magnitude obtained in this condition was related to AB magnitudes obtained in both sound conditions. Self-reported distractibility and performance in tests of divided attention and of cognitive flexibility correlated with the AB magnitudes of a subset but never all conditions under study. Reliability and correlation results suggest that not only dispositional abilities but also state factors exert an influence on AB magnitude. These findings extend earlier work on audition-driven enhancement of target identification in the AB and on the reliability and behavioural correlates of the AB.

Visual movement perception in deaf and hearing individuals.

A number of studies have investigated changes in the perception of visual motion as a result of altered sensory experiences. An animal study has shown that auditory-deprived cats exhibit enhanced performance in a visual movement detection task compared to hearing cats (Lomber, Meredith, & Kral, 2010). In humans, the behavioural evidence regarding the perception of motion is less clear. The present study investigated deaf and hearing adult participants using a movement localization task and a direction of motion task employing coherently-moving and static visual dot patterns. Overall, deaf and hearing participants did not differ in their movement localization performance, although within the deaf group, a left visual field advantage was found. When discriminating the direction of motion, however, deaf participants responded faster and tended to be more accurate when detecting small differences in direction compared with the hearing controls. These results conform to the view that visual abilities are enhanced after auditory deprivation and extend previous findings regarding visual motion processing in deaf individuals.

Electrophysiological correlates of auditory change detection and change deafness in complex auditory scenes.

Change deafness describes the failure to perceive even intense changes within complex auditory input, if the listener does not attend to the changing sound. Remarkably, previous psychophysical data provide evidence that this effect occurs independently of successful stimulus encoding, indicating that undetected changes are processed to some extent in auditory cortex. Here we investigated cortical representations of detected and undetected auditory changes using electroencephalographic (EEG) recordings and a change deafness paradigm. We applied a one-shot change detection task, in which participants listened successively to three complex auditory scenes, each of them consisting of six simultaneously presented auditory streams. Listeners had to decide whether all scenes were identical or whether the pitch of one stream was changed between the last two presentations. Our data show significantly increased middle-latency Nb responses for both detected and undetected changes as compared to no-change trials. In contrast, only successfully detected changes were associated with a later mismatch response in auditory cortex, followed by increased N2, P3a and P3b responses, originating from hierarchically higher non-sensory brain regions. These results strengthen the view that undetected changes are successfully encoded at sensory level in auditory cortex, but fail to trigger later change-related cortical responses that lead to conscious perception of change.

Tough doughnuts: affect and the modulation of attention.

Positive affect has been associated with improvement in performance in various attentional domains. Negative affect has been associated with narrowing of attention and lowering of performance in attentional tasks. Previous behavioral studies have put forth the diffuse mental state idea as the mechanism of these effects, where attentional resources are more evenly distributed during positive affect and more focused during negative affect. To explore neural correlates of this mechanism, a two-stream rapid serial visual presentation (RSVP) paradigm with centrally presented, overlapping streams was used. Participants attended one of the streams at a time and steady-state visual evoked potentials (ssVEP) in response to the attended and unattended streams were recorded in a positive, negative or neutral affect state. We predicted that in the positive affect condition, ssVEP responses to the attended and the unattended stream would be more alike than in a neutral condition. In the negative affect condition, as an expression of a less diffuse mental state, ssVEP responses were predicted to be more dissimilar. Self-assessments confirmed the effectiveness of the emotional manipulation. In the negative affect condition power was found to be higher than in the neutral condition. However, the modulations in the ssVEP did not reflect the predicted neural correlate of the diffuse mental state mechanism. Thus, the results provide evidence for negative affect modulating attention but suggest that the diffuse mental state is not a spatially oriented phenomenon.

Let's face it, from trial to trial: comparing procedures for N170 single-trial estimation.

The estimation of event-related single trial EEG activity is notoriously difficult but is of growing interest in various areas of cognitive neuroscience, such as multimodal neuroimaging and EEG-based brain computer interfaces. However, an objective evaluation of different approaches is lacking. The present study therefore compared four frequently-used single-trial data filtering procedures: raw sensor amplitudes, regression-based estimation, bandpass filtering, and independent component analysis (ICA). High-density EEG data were recorded from 20 healthy participants in a face recognition task and were analyzed with a focus on the face-selective N170 single-trial event-related potential. Linear discriminant analysis revealed significantly better single-trial estimation for ICA compared to raw sensor amplitudes, whereas the other two approaches did not improve classification accuracy. Further analyses suggested that ICA enabled extraction of a face-sensitive independent component in each participant, which led to the superior performance in single trial estimation. Additionally, we show that the face-sensitive component does not directly represent activity from a neuronal population exclusively involved in face-processing, but rather the activity of a network involved in general visual processing. We conclude that ICA effectively facilitates the separation of physiological trial-by-trial fluctuations from measurement noise, in particular when the process of interest is reliably reflected in components representing the neural signature of interest.

Perceptual load alters visual excitability.

Increasing perceptual load reduces the processing of visual stimuli outside the focus of attention, but the mechanism underlying these effects remains unclear. Here we tested an account attributing the effects of perceptual load to modulations of visual cortex excitability. In contrast to stimulus competition accounts, which propose that load should affect simultaneous, but not sequential, stimulus presentations, the visual excitability account makes the novel prediction that load should affect detection sensitivity for both simultaneous and sequential presentations. Participants fixated a stimulus stream, responding to targets defined by either a color (low load) or color and orientation conjunctions (high load). Additionally, detection sensitivity was measured for a peripheral critical stimulus (CS) presented occasionally. Increasing load at fixation reduced sensitivity to the peripheral CSs; this effect was similar regardless of whether CSs were presented simultaneously with central stimuli or during the (otherwise empty) interval between them. Controls ruled out explanations of the results in terms of strategic task prioritization. These findings support a cortical excitability account for perceptual load, challenging stimulus competition accounts.