Immersive virtual reality gameplay detects visuospatial atypicality, including unilateral spatial neglect, following brain injury: a pilot study.

BACKGROUND: In neurorehabilitation, problems with visuospatial attention, including unilateral spatial neglect, are prevalent and routinely assessed by pen-and-paper tests, which are limited in accuracy and sensitivity. Immersive virtual reality (VR), which motivates a much wider (more intuitive) spatial behaviour, promises new futures for identifying visuospatial atypicality in multiple measures, which reflects cognitive and motor diversity across individuals with brain injuries. METHODS: In this pilot study, we had 9 clinician controls (mean age 43 years; 4 males) and 13 neurorehabilitation inpatients (mean age 59 years; 9 males) recruited a mean of 41 days post-injury play a VR visual search game. Primary injuries included 7 stroke, 4 traumatic brain injury, 2 other acquired brain injury. Three patients were identified as having left sided neglect prior to taking part in the VR. Response accuracy, reaction time, and headset and controller raycast orientation quantified gameplay. Normative modelling identified the typical gameplay bounds, and visuospatial atypicality was defined as gameplay beyond these bounds. RESULTS: The study found VR to be feasible, with only minor instances of motion sickness, positive user experiences, and satisfactory system usability. Crucially, the analytical method, which emphasized identifying 'visuospatial atypicality,' proved effective. Visuospatial atypicality was more commonly observed in patients compared to controls and was prevalent in both groups of patients-those with and without neglect. CONCLUSION: Our research indicates that normative modelling of VR gameplay is a promising tool for identifying visuospatial atypicality after acute brain injury. This approach holds potential for a detailed examination of neglect.

Optimising the classification of feature-based attention in frequency-tagged electroencephalography data.

Brain-computer interfaces (BCIs) are a rapidly expanding field of study and require accurate and reliable real-time decoding of patterns of neural activity. These protocols often exploit selective attention, a neural mechanism that prioritises the sensory processing of task-relevant stimulus features (feature-based attention) or task-relevant spatial locations (spatial attention). Within the visual modality, attentional modulation of neural responses to different inputs is well indexed by steady-state visual evoked potentials (SSVEPs). These signals are reliably present in single-trial electroencephalography (EEG) data, are largely resilient to common EEG artifacts, and allow separation of neural responses to numerous concurrently presented visual stimuli. To date, efforts to use single-trial SSVEPs to classify visual attention for BCI control have largely focused on spatial attention rather than feature-based attention. Here, we present a dataset that allows for the development and benchmarking of algorithms to classify feature-based attention using single-trial EEG data. The dataset includes EEG and behavioural responses from 30 healthy human participants who performed a feature-based motion discrimination task on frequency tagged visual stimuli.

Implicit Neurofeedback Training of Feature-Based Attention Promotes Biased Sensory Processing during Integrative Decision-Making.

Complex perceptual decisions, in which information must be integrated across multiple sources of evidence, are ubiquitous but are not well understood. Such decisions rely on sensory processing of each individual source of evidence, and are therefore vulnerable to bias if sensory processing resources are disproportionately allocated among visual inputs. To investigate this, we developed an implicit neurofeedback protocol embedded within a complex decision-making task to bias sensory processing in favor of one source of evidence over another. Human participants of both sexes (N = 30) were asked to report the average motion direction across two fields of oriented moving bars. Bars of different orientations flickered at different frequencies, thus inducing steady-state visual evoked potentials. Unbeknownst to participants, neurofeedback was implemented to implicitly reward attention to a specific "trained" orientation (rather than any particular motion direction). As attentional selectivity for this orientation increased, the motion coherence of both fields of bars increased, making the task easier without altering the relative reliability of the two sources of evidence. Critically, these neurofeedback trials were alternated with "test" trials in which motion coherence was not contingent on attentional selectivity, allowing us to assess the training efficacy. The protocol successfully biased sensory processing, resulting in earlier and stronger encoding of the trained evidence source. In turn, this evidence was weighted more heavily in behavioral and neural representations of the integrated average, although the two sources of evidence were always matched in reliability. These results demonstrate how biases in sensory processing can impact integrative decision-making processes.SIGNIFICANCE STATEMENT Many everyday decisions require active integration of different sources of sensory information, such as deciding when it is safe to cross a road, yet little is known about how the brain prioritizes sensory sources in the service of adaptive behavior, or whether such decisions can be altered through learning. Here we addressed these questions using a novel behavioral protocol that provided observers with real-time feedback of their own brain activity patterns in which sensory processing was implicitly biased toward a subset of the available information. We show that, while such biases are a normal and adaptive mechanism for humans to process complex visual information, they can also contribute to suboptimal decision-making.

Joint control of visually guided actions involves concordant increases in behavioural and neural coupling.

It is often necessary for individuals to coordinate their actions with others. In the real world, joint actions rely on the direct observation of co-actors and rhythmic cues. But how are joint actions coordinated when such cues are unavailable? To address this question, we recorded brain activity while pairs of participants guided a cursor to a target either individually (solo control) or together with a partner (joint control) from whom they were physically and visibly separated. Behavioural patterns revealed that joint action involved real-time coordination between co-actors and improved accuracy for the lower performing co-actor. Concurrent neural recordings and eye tracking revealed that joint control affected cognitive processing across multiple stages. Joint control involved increases in both behavioural and neural coupling - both quantified as interpersonal correlations - peaking at action completion. Correspondingly, a neural offset response acted as a mechanism for and marker of interpersonal neural coupling, underpinning successful joint actions.

Optimising non-invasive brain-computer interface systems for free communication between naïve human participants.

Free communication is one of the cornerstones of modern civilisation. While manual keyboards currently allow us to interface with computers and manifest our thoughts, a next frontier is communication without manual input. Brain-computer interface (BCI) spellers often achieve this by decoding patterns of neural activity as users attend to flickering keyboard displays. To date, the highest performing spellers report typing rates of ~10.00 words/minute. While impressive, these rates are typically calculated for experienced users repetitively typing single phrases. It is therefore not clear whether naïve users are able to achieve such high rates with the added cognitive load of genuine free communication, which involves continuously generating and spelling novel words and phrases. In two experiments, we developed an open-source, high-performance, non-invasive BCI speller and examined its feasibility for free communication. The BCI speller required users to focus their visual attention on a flickering keyboard display, thereby producing unique cortical activity patterns for each key, which were decoded using filter-bank canonical correlation analysis. In Experiment 1, we tested whether seventeen naïve users could maintain rapid typing during prompted free word association. We found that information transfer rates were indeed slower during this free communication task than during typing of a cued character sequence. In Experiment 2, we further evaluated the speller's efficacy for free communication by developing a messaging interface, allowing users to engage in free conversation. The results showed that free communication was possible, but that information transfer was reduced by voluntary textual corrections and turn-taking during conversation. We evaluated a number of factors affecting the suitability of BCI spellers for free communication, and make specific recommendations for improving classification accuracy and usability. Overall, we found that developing a BCI speller for free communication requires a focus on usability over reduced character selection time, and as such, future performance appraisals should be based on genuine free communication scenarios.

Differential Deployment of Visual Attention During Interactive Approach and Avoidance Behavior.

The ability to coordinate approach and avoidance actions in dynamic environments represents the boundary between extinction and the continued survival of many animal species. It is therefore crucial that sensory systems allocate limited attentional resources to the most relevant information to facilitate planning and execution of appropriate actions. Prominent theories of how attention regulates visual processing focus on the distinction between behaviorally relevant and irrelevant visual inputs. To date, however, no study has directly compared the deployment of attention to visual inputs relevant for approach and avoidance behaviors, which naturally occur in dynamic, interactive environments. In two experiments, we combined electroencephalography, frequency tagging, and eye gaze measures to investigate whether the deployment of visual selective attention differs for items relevant for approach and avoidance actions. Participants maneuvered a cursor to approach and avoid contact with moving items in a continuous interactive task. The results indicated that while the approach and avoidance tasks recruited equivalent attentional resources overall, attentional biases were directed toward task-relevant items during approach, and away from task-relevant items during avoidance. We conclude that the deployment of visual attention is guided not only by relevance to a behavioral goal, but also by the nature of that goal.

Stimulus-Driven Cortical Hyperexcitability in Individuals with Charles Bonnet Hallucinations.

Throughout the lifespan, the cerebral cortex adapts its structure and function in response to changing sensory input [1, 2]. Whilst such changes are typically adaptive, they can be maladaptive when they follow damage to the peripheral nervous system, including phantom limb pain and tinnitus [3, 4]. An intriguing example occurs in individuals with acquired ocular pathologies-most commonly age-related macular degeneration (MD) [5]-who lose their foveal vision but retain intact acuity in the peripheral visual field. Up to 40% of ocular pathology patients develop long-term hallucinations involving flashes of light, shapes, or geometric patterns and/or complex hallucinations, including faces, animals, or entire scenes, a condition known as Charles Bonnet Syndrome (CBS) [6, 7, 8]. Though CBS was first described over 250 years ago [9, 10], the neural basis for the hallucinations remains unclear, with no satisfactory explanation as to why some individuals develop hallucinations, while many do not. An influential but untested hypothesis for the visual hallucinations in CBS is that retinal deafferentation causes hyperexcitability in early visual cortex. To assess this, we investigated electrophysiological responses to peripheral visual field stimulation in MD patients with and without hallucinations and in matched controls without ocular pathology. Participants performed a concurrent attention task within intact portions of their peripheral visual field, while ignoring flickering checkerboards that drove periodic electrophysiological responses. CBS individuals showed strikingly elevated visual cortical responses to peripheral field stimulation compared with patients without hallucinations and controls, providing direct support for the hypothesis of visual cortical hyperexcitability in CBS.

Causal involvement of visual area MT in global feature-based enhancement but not contingent attentional capture.

When visual attention is set for a particular target feature, such as color or shape, neural responses to that feature are enhanced across the visual field. This global feature-based enhancement is hypothesized to underlie the contingent attentional capture effect, in which task-irrelevant items with the target feature capture spatial attention. In humans, however, different cortical regions have been implicated in global feature-based enhancement and contingent capture. Here, we applied intermittent theta-burst stimulation (iTBS) to assess the causal roles of two regions of extrastriate cortex - right area MT and the right temporoparietal junction (TPJ) - in both global feature-based enhancement and contingent capture. We recorded cortical activity using EEG while participants monitored centrally for targets defined by color and ignored peripheral checkerboards that matched the distractor or target color. In central vision, targets were preceded by colored cues designed to capture attention. Stimuli flickered at unique frequencies, evoking distinct cortical oscillations. Analyses of these oscillations and behavioral performance revealed contingent capture in central vision and global feature-based enhancement in the periphery. Stimulation of right area MT selectively increased global feature-based enhancement, but did not influence contingent attentional capture. By contrast, stimulation of the right TPJ left both processes unaffected. Our results reveal a causal role for the right area MT in feature-based attention, and suggest that global feature-based enhancement does not underlie the contingent capture effect.

Distinct roles of the intraparietal sulcus and temporoparietal junction in attentional capture from distractor features: An individual differences approach.

Setting attention for an elementary visual feature, such as color or motion, results in greater spatial attentional "capture" from items with target compared with distractor features. Thus, capture is contingent on feature-based control settings. Neuroimaging studies suggest that this contingent attentional capture involves interactions between dorsal and ventral frontoparietal networks. To examine the distinct causal influences of these networks on contingent capture, we applied continuous theta-burst stimulation (cTBS) to alter neural excitability within the dorsal intraparietal sulcus (IPS), the ventral temporoparietal junction (TPJ) and a control site, visual area MT. Participants undertook an attentional capture task before and after stimulation, in which they made speeded responses to color-defined targets that were preceded by spatial cues in the target or distractor color. Cues appeared either at the target location (valid) or at a non-target location (invalid). Reaction times were slower for targets preceded by invalid compared with valid cues, demonstrating spatial attentional capture. Cues with the target color captured attention to a greater extent than those with the distractor color, consistent with contingent capture. Effects of cTBS were not evident at the group level, but emerged instead from analyses of individual differences. Target capture magnitude was positively correlated pre- and post-stimulation for all three cortical sites, suggesting that cTBS did not influence target capture. Conversely, distractor capture was positively correlated pre- and post-stimulation of MT, but uncorrelated for IPS and TPJ, suggesting that stimulation of IPS and TPJ selectively disrupted distractor capture. Additionally, the effects of IPS stimulation were predicted by pre-stimulation attentional capture, whereas the effects of TPJ stimulation were predicted by pre-stimulation distractor suppression. The results are consistent with the existence of distinct neural circuits underlying target and distractor capture, as well as distinct roles for the IPS and TPJ.

Neural responses to target features outside a search array are enhanced during conjunction but not unique-feature search.

The visual world is typically too complex to permit full apprehension of its content from a single fixation. Humans therefore use visual search to direct attention and eye movements to locations or objects of interest in cluttered scenes. Psychophysical investigations have revealed that observers can select target elements from within an array of distractors on the basis of their spatial location or simple features, such as color. It remains unclear, however, how stimuli that lie outside the current search array are represented in the visual system. To investigate this, we recorded continuous neural activity using EEG while participants searched a foveal array of colored targets and distractors, and ignored irrelevant objects in the periphery. Search targets were defined either by a unique feature within the array or by a conjunction of features. Objects outside the array could match the target or distractor color within the array, or otherwise possessed a baseline (neutral) color present only in the periphery. The search array and irrelevant peripheral objects flickered at unique rates and thus evoked distinct frequency-tagged neural oscillations. During conjunction but not unique-feature search, target-colored objects outside the array evoked enhanced activity relative to distractor-colored and neutral objects. The results suggest that feature-based selection applies to stimuli at ignored peripheral locations, but only when central targets compete with distractors within the array. Distractor-colored and neutral objects evoked equivalent oscillatory responses, suggesting that feature-based selection at ignored locations during visual search arises exclusively from enhancement rather than suppression of neural activity.

Value learning modulates goal-directed actions.

With experience, particular objects can predict good or bad outcomes. This alters our perceptual response to them: Reliable predictors of salient outcomes are recognized faster and better than unreliable predictors, regardless of the value (gain, loss) of the outcome they predict. When attentional resources are constrained, learned value associations matter, causing recognition of gain-associated objects to be spared. Here, we ask how learned predictiveness and value change the way we interact with potentially rewarding objects. After associating virtual objects (drinking flutes) with monetary gains or losses, reaching for and grasping corresponding real objects depended on the object's learned value. Action was faster when directed at objects that previously predicted outcomes more rather than less consistently, regardless of value. Conversely, reaches were more direct for gain- than for loss-associated objects, regardless of their predictiveness. Action monitoring thus reveals how value learning components become accessible during action.

Event coding and motor priming: how attentional modulation may influence binding across action properties.

When we perceive an action it is internally transformed into a motor representation akin to the execution of that same action. Motor priming studies show that action observation facilitates the execution of physically similar actions, but interferes with the performance of dissimilar actions. Some evidence suggests, however, that once a specific motor plan is formed, perceiving an action with partially overlapping features (e.g. a congruent grip-type but and incongruent end-goal) can result in interference. In two experiments we investigate how modulating attention towards observed actions influences the binding that occurs between action features, and therefore the amount of partial-overlap interference to participants' performance. In the first study we directed attention towards a salient action feature (the grip-type). We found that perceiving partially overlapping (i.e. partially congruent) actions slowed participants' responses compared with observation of completely congruent or incongruent actions. In the second experiment attentional resources were taxed through the use of a secondary task. This resulted in the elimination of the partial-overlap interference effect. We discuss results in relation to feature binding and event codes.