Human brain dynamics dissociate early perceptual and late motor-related stages of affordance processing.

Affordances, the opportunities for action offered by the environment to an agent, are vital for meaningful behaviour and exist in every interaction with the environment. There is an ongoing debate in the field about whether the perception of affordances is an automated process. Some studies suggest that affordance perception is an automated process that is independent from the visual context and bodily interaction with the environment, whereas others argue that it is modulated by the visual and motor context in which affordances are perceived. The present paper aims to resolve this debate by examining affordance automaticity from the perspective of sensorimotor time windows. To investigate the impact of different forms of bodily interactions with an environment, that is, the movement context (physical vs. joystick movement), we replicated a previous study on affordance perception in which participants actively moved through differently wide doors in an immersive 3D virtual environment. In the present study, we displayed the same environment on a 2D screen with participants moving through doors of different widths using the keys on a standard keyboard. We compared components of the event-related potential (ERP) from the continuously recorded electroencephalogram (EEG) that were previously reported to be related to affordance perception of architectural transitions (passable and impassable doors). Comparing early sensory and later motor-related ERPs, our study replicated ERPs reflecting early affordance perception but found differences in later motor-related components. These results indicate a shift from automated perception of affordances during early sensorimotor time windows to movement context dependence of affordance perception at later stages, suggesting that affordance perception is a dynamic and flexible process that changes over sensorimotor stages.

Prediction of cognitive conflict during unexpected robot behavior under different mental workload conditions in a physical human-robot collaboration.

Objective. Brain-computer interface (BCI) technology is poised to play a prominent role in modern work environments, especially a collaborative environment where humans and machines work in close proximity, often with physical contact. In a physical human robot collaboration (pHRC), the robot performs complex motion sequences. Any unexpected robot behavior or faulty interaction might raise safety concerns. Error-related potentials, naturally generated by the brain when a human partner perceives an error, have been extensively employed in BCI as implicit human feedback to adapt robot behavior to facilitate a safe and intuitive interaction. However, the integration of BCI technology with error-related potential for robot control demands failure-free integration of highly uncertain electroencephalography (EEG) signals, particularly influenced by the physical and cognitive state of the user. As a higher workload on the user compromises their access to cognitive resources needed for error awareness, it is crucial to study how mental workload variations impact the error awareness as it might raise safety concerns in pHRC. In this study, we aim to study how cognitive workload affects the error awareness of a human user engaged in a pHRC.Approach. We designed a blasting task with an abrasive industrial robot and manipulated the mental workload with a secondary arithmetic task of varying difficulty. EEG data, perceived workload, task and physical performance were recorded from 24 participants moving the robot arm. The error condition was achieved by the unexpected stopping of the robot in 33% of trials.Main results. We observed a diminished amplitude for the prediction error negativity (PEN) and error positivity (Pe), indicating reduced error awareness with increasing mental workload. We further observed an increased frontal theta power and increasing trend in the central alpha and central beta power after the unexpected robot stopping compared to when the robot stopped correctly at the target. We also demonstrate that a popular convolution neural network model, EEGNet, could predict the amplitudes of PEN and Pe from the EEG data prior to the error.Significance. This prediction model could be instrumental in developing an online prediction model that could forewarn the system and operators of the diminished error awareness of the user, alluding to a potential safety breach in error-related potential-based BCI system for pHRC. Therefore, our work paves the way for embracing BCI technology in pHRC to optimally adapt the robot behavior for personalized user experience using real-time brain activity, enriching the quality of the interaction.

Multisensory input modulates memory-guided spatial navigation in humans.

Efficient navigation is supported by a cognitive map of space. The hippocampus plays a key role for this map by linking multimodal sensory information with spatial memory representations. However, in human navigation studies, the full range of sensory information is often unavailable due to the stationarity of experimental setups. We investigated the contribution of multisensory information to memory-guided spatial navigation by presenting a virtual version of the Morris water maze on a screen and in an immersive mobile virtual reality setup. Patients with hippocampal lesions and matched controls navigated to memorized object locations in relation to surrounding landmarks. Our results show that availability of multisensory input improves memory-guided spatial navigation in both groups. It has distinct effects on navigational behaviour, with greater improvement in spatial memory performance in patients. We conclude that congruent multisensory information shifts computations to extrahippocampal areas that support spatial navigation and compensates for spatial navigation deficits.

Neuromuscular assessment of force development, postural, and gait performance under cognitive-motor dual-tasking in healthy older adults and people with early Parkinson's disease: Study protocol for a cross-sectional Mobile Brain/Body Imaging (MoBI) study.

Background: Neuromuscular dysfunction is common in older adults and more pronounced in neurodegenerative diseases. In Parkinson's disease (PD), a complex set of factors often prevents the effective performance of activities of daily living that require intact and simultaneous performance of the motor and cognitive tasks. Methods: The cross-sectional study includes a multifactorial mixed-measure design. Between-subject factor grouping the sample will be Parkinson's Disease (early PD vs. healthy). The within-subject factors will be the task complexity (single- vs. dual-task) in each motor activity, i.e., overground walking, semi-tandem stance, and isometric knee extension, and a walking condition (wide vs. narrow lane) will be implemented for the overground walking activity only. To study dual-task (DT) effects, in each motor activity participants will be given a secondary cognitive task, i.e., a visual discrimination task for the overground walking, an attention task for the semi-tandem, and mental arithmetic for the isometric extension. Analyses of DT effects and underlying neuronal correlates will focus on both gait and cognitive performance where applicable. Based on an a priori sample size calculation, a total N = 42 older adults (55-75 years) will be recruited. Disease-specific changes such as laterality in motor unit behavior and cortical control of movement will be studied with high-density surface electromyography and electroencephalography during static and dynamic motor activities, together with whole-body kinematics. Discussion: This study will be one of the first to holistically address early PD neurophysiological and neuromuscular patterns in an ecologically valid environment under cognitive-motor DT conditions of different complexities. The outcomes of the study aim to identify the biomarker for early PD either at the electrophysiological, muscular or kinematic level or in the communication between these systems. Clinical Trial Registration: ClinicalTrials.Gov, NCT05477654. This study was approved by the Medical Ethical Committee (106/2021).

CLET: Computation of Latencies in Event-related potential Triggers using photodiode on virtual reality apparatuses.

To investigate event-related activity in human brain dynamics as measured with EEG, triggers must be incorporated to indicate the onset of events in the experimental protocol. Such triggers allow for the extraction of ERP, i.e., systematic electrophysiological responses to internal or external stimuli that must be extracted from the ongoing oscillatory activity by averaging several trials containing similar events. Due to the technical setup with separate hardware sending and recording triggers, the recorded data commonly involves latency differences between the transmitted and received triggers. The computation of these latencies is critical for shifting the epochs with respect to the triggers sent. Otherwise, timing differences can lead to a misinterpretation of the resulting ERPs. This study presents a methodical approach for the CLET using a photodiode on a non-immersive VR (i.e., LED screen) and an immersive VR (i.e., HMD). Two sets of algorithms are proposed to analyze the photodiode data. The experiment designed for this study involved the synchronization of EEG, EMG, PPG, photodiode sensors, and ten 3D MoCap cameras with a VR presentation platform (Unity). The average latency computed for LED screen data for a set of white and black stimuli was 121.98 ± 8.71 ms and 121.66 ± 8.80 ms, respectively. In contrast, the average latency computed for HMD data for the white and black stimuli sets was 82.80 ± 7.63 ms and 69.82 ± 5.52 ms. The codes for CLET and analysis, along with datasets, tables, and a tutorial video for using the codes, have been made publicly available.

Modeling the Intent to Interact with VR using Physiological Features.

OBJECTIVE: Mixed-Reality (XR) technologies promise a user experience (UX) that rivals the interactive experience with the real-world. The key facilitators in the design of such a natural UX are that the interaction has zero lag and that users experience no excess mental load. This is difficult to achieve due to technical constraints such as motion-to-photon latency as well as false-positives during gesture-based interaction.

Using spontaneous eye blink-related brain activity to investigate cognitive load during mobile map-assisted navigation.

The continuous assessment of pedestrians' cognitive load during a naturalistic mobile map-assisted navigation task is challenging because of limited experimental control over stimulus presentation, human-map-interactions, and other participant responses. To overcome this challenge, the present study takes advantage of navigators' spontaneous eye blinks during navigation to serve as event markers in continuously recorded electroencephalography (EEG) data to assess cognitive load in a mobile map-assisted navigation task. We examined if and how displaying different numbers of landmarks (3 vs. 5 vs. 7) on mobile maps along a given route would influence navigators' cognitive load during navigation in virtual urban environments. Cognitive load was assessed by the peak amplitudes of the blink-related fronto-central N2 and parieto-occipital P3. Our results show increased parieto-occipital P3 amplitude indicating higher cognitive load in the 7-landmark condition, compared to showing 3 or 5 landmarks. Our prior research already demonstrated that participants acquire more spatial knowledge in the 5- and 7-landmark conditions compared to the 3-landmark condition. Together with the current study, we find that showing 5 landmarks, compared to 3 or 7 landmarks, improved spatial learning without overtaxing cognitive load during navigation in different urban environments. Our findings also indicate a possible cognitive load spillover effect during map-assisted wayfinding whereby cognitive load during map viewing might have affected cognitive load during goal-directed locomotion in the environment or vice versa. Our research demonstrates that users' cognitive load and spatial learning should be considered together when designing the display of future navigation aids and that navigators' eye blinks can serve as useful event makers to parse continuous human brain dynamics reflecting cognitive load in naturalistic settings.

Stroop in motion: Neurodynamic modulation underlying interference control while sitting, standing, and walking.

There is conflicting evidence about how interference control in healthy adults is affected by walking as compared to standing or sitting. Although the Stroop paradigm is one of the best-studied paradigms to investigate interference control, the neurodynamics associated with the Stroop task during walking have never been studied. We investigated three Stroop tasks using variants with increasing interference levels - word-reading, ink-naming, and the switching of the two tasks, combined in a systematic dual-tasking fashion with three motor conditions - sitting, standing, and treadmill walking. Neurodynamics underlying interference control were recorded using the electroencephalogram. Worsened performance was observed for the incongruent compared to congruent trials and for the switching Stroop compared to the other two variants. The early frontocentral event-related potentials (ERPs) associated with executive functions (P2, N2) differentially signaled posture-related workloads, while the later stages of information processing indexed faster interference suppression and response selection in walking compared to static conditions. The early P2 and N2 components as well as frontocentral Theta and parietal Alpha power were sensitive to increasing workloads on the motor and cognitive systems. The distinction between the type of load (motor and cognitive) became evident only in the later posterior ERP components in which the amplitude non-uniformly reflected the relative attentional demand of a task. Our data suggest that walking might facilitate selective attention and interference control in healthy adults. Existing interpretations of ERP components recorded in stationary settings should be considered with care as they might not be directly transferable to mobile settings.

Does Standing Up Enhance Performance on the Stroop Task in Healthy Young Adults? A Systematic Review and Meta-Analysis.

Understanding the changes in cognitive processing that accompany changes in posture can expand our understanding of embodied cognition and open new avenues for applications in (neuro)ergonomics. Recent studies have challenged the question of whether standing up alters cognitive performance. An electronic database search for randomized controlled trials was performed using Academic Search Complete, CINAHL Ultimate, MEDLINE, PubMed, and Web of Science following PRISMA guidelines, PICOS framework, and standard quality assessment criteria (SQAC). We pooled data from a total of 603 healthy young adults for incongruent and 578 for congruent stimuli and Stroop effect (mean age = 24 years). Using random-effects results, no difference was found between sitting and standing for the Stroop effect (Hedges' g = 0.13, 95% CI = -0.04 to 0.29, p = 0.134), even when comparing congruent (Hedges' g = 0.10; 95% CI: -0.132 to 0.339; Z = 0.86; p = 0.389) and incongruent (Hedges' g = 0.18; 95% CI: -0.072 to 0.422; Z = 1.39; p = 0.164) stimuli separately. Importantly, these results imply that changing from a seated to a standing posture in healthy young adults is unlikely to have detrimental effects on selective attention and cognitive control. To gain a full understanding of this phenomenon, further research should examine this effect in a population of healthy older adults, as well as in a population with pathology.

Virtual Reality for Spatial Navigation.

Immersive virtual reality (VR) allows its users to experience physical space in a non-physical world. It has developed into a powerful research tool to investigate the neural basis of human spatial navigation as an embodied experience. The task of wayfinding can be carried out by using a wide range of strategies, leading to the recruitment of various sensory modalities and brain areas in real-life scenarios. While traditional desktop-based VR setups primarily focus on vision-based navigation, immersive VR setups, especially mobile variants, can efficiently account for motor processes that constitute locomotion in the physical world, such as head-turning and walking. When used in combination with mobile neuroimaging methods, immersive VR affords a natural mode of locomotion and high immersion in experimental settings, designing an embodied spatial experience. This in turn facilitates ecologically valid investigation of the neural underpinnings of spatial navigation.

HArtMuT-modeling eye and muscle contributors in neuroelectric imaging.

Objective.Magneto- and electroencephalography (M/EEG) measurements record a mix of signals from the brain, eyes, and muscles. These signals can be disentangled for artifact cleaning e.g. using spatial filtering techniques. However, correctly localizing and identifying these components relies on head models that so far only take brain sources into account.Approach.We thus developed the Head Artifact Model using Tripoles (HArtMuT). This volume conduction head model extends to the neck and includes brain sources as well as sources representing eyes and muscles that can be modeled as single dipoles, symmetrical dipoles, and tripoles. We compared a HArtMuT four-layer boundary element model (BEM) with the EEGLAB standard head model on their localization accuracy and residual variance (RV) using a HArtMuT finite element model (FEM) as ground truth. We also evaluated the RV on real-world data of mobile participants, comparing different HArtMuT BEM types with the EEGLAB standard head model.Main results.We found that HArtMuT improves localization for all sources, especially non-brain, and localization error and RV of non-brain sources were in the same range as those of brain sources. The best results were achieved by using cortical dipoles, muscular tripoles, and ocular symmetric dipoles, but dipolar sources alone can already lead to convincing results.Significance.We conclude that HArtMuT is well suited for modeling eye and muscle contributions to the M/EEG signal. It can be used to localize sources and to identify brain, eye, and muscle components. HArtMuT is freely available and can be integrated into standard software.

Successful wayfinding in age: A scoping review on spatial navigation training in healthy older adults.

Introduction: Spatial navigation is a complex cognitive function that declines in older age. Finding one's way around in familiar and new environments is crucial to live and function independently. However, the current literature illustrates the efficacy of spatial navigation interventions in rehabilitative contexts such as pathological aging and traumatic injury, but an overview of existing training studies for healthy older adults is missing. This scoping review aims to identify current evidence on existing spatial navigation interventions in healthy older adults and analyze their efficacy. Methods: To identify spatial navigation interventions and assessments and investigate their effectiveness, four electronic databases were searched (Pubmed, Web of Science, CINAHL and EMBASE). Two independent reviewers conducted a screening of title, abstract and full-texts and performed a quality assessment. Studies were eligible if (1) published in English, (2) the full text was accessible, (3) at least one group of healthy older adults was included with (4) mean age of 65 years or older, (5) three or more spatial navigation-related training sessions were conducted and (6) at least one spatial ability outcome was reported. Results: Ten studies were included (N = 1,003, age-range 20-95 years, 51.5% female), only healthy older adults (n = 368, mean age ≥ 65) were assessed further. Studies differed in sample size (n = 22-401), type of training, total intervention duration (100 min-50 h), and intervention period (1-16 weeks). Conclusion: The spatial navigation abilities addressed and the measures applied to elicit intervention effects varied in quantity and methodology. Significant improvements were found for at least one spatial ability-related outcome in six of 10 interventions. Two interventions achieved a non-significant positive trend, another revealed no measurable post-training improvement, and one study did not report pre-post-differences. The results indicate that different types of spatial navigation interventions improve components of spatial abilities in healthy older adults. The existing body of research does not allow conclusions on transferability of the trained components on everyday life spatial navigation performance. Future research should focus on reproducing and extending the promising approaches of available evidence. From this, valuable insights on healthy aging could emerge. Trial Registration: This scoping review was preregistered at Open Science Framework (https://osf.io/m9ab6).

Redesigning navigational aids using virtual global landmarks to improve spatial knowledge retrieval.

Although beacon- and map-based spatial strategies are the default strategies for navigation activities, today's navigational aids mostly follow a beacon-based design where one is provided with turn-by-turn instructions. Recent research, however, shows that our reliance on these navigational aids is causing a decline in our spatial skills. We are processing less of our surrounding environment and relying too heavily on the instructions given. To reverse this decline, we need to engage more in map-based learning, which encourages the user to process and integrate spatial knowledge into a cognitive map built to benefit flexible and independent spatial navigation behaviour. In an attempt to curb our loss of skills, we proposed a navigation assistant to support map-based learning during active navigation. Called the virtual global landmark (VGL) system, this augmented reality (AR) system is based on the kinds of techniques used in traditional orienteering. Specifically, a notable landmark is always present in the user's sight, allowing the user to continuously compute where they are in relation to that specific location. The efficacy of the unit as a navigational aid was tested in an experiment with 27 students from the University of Technology Sydney via a comparison of brain dynamics and behaviour. From an analysis of behaviour and event-related spectral perturbation, we found that participants were encouraged to process more spatial information with a map-based strategy where a silhouette of the compass-like landmark was perpetually in view. As a result of this technique, they consistently navigated with greater efficiency and better accuracy.

Neuroscience and architecture: Modulating behavior through sensorimotor responses to the built environment.

As we move through the world, natural and built environments implicitly guide behavior by appealing to certain sensory and motor dynamics. This process can be motivated by automatic attention to environmental features that resonate with specific sensorimotor responses. This review aims at providing a psychobiological framework describing how environmental features can lead to automated sensorimotor responses through defined neurophysiological mechanisms underlying attention. Through the use of automated processes in subsets of cortical structures, the goal of this framework is to describe on a neuronal level the functional link between the designed environment and sensorimotor responses. By distinguishing between environmental features and sensorimotor responses we elaborate on how automatic behavior employs the environment for sensorimotor adaptation. This is realized through a thalamo-cortical network integrating environmental features with motor aspects of behavior. We highlight the underlying transthalamic transmission from an Enactive and predictive perspective and review recent studies that effectively modulated behavior by systematically manipulating environmental features. We end by suggesting a promising combination of neuroimaging and computational analysis for future studies.

The Embodiment of Architectural Experience: A Methodological Perspective on Neuro-Architecture.

People spend a large portion of their time inside built environments. Research in neuro-architecture-the neural basis of human perception of and interaction with the surrounding architecture-promises to advance our understanding of the cognitive processes underlying this common human experience and also to inspire evidence-based architectural design principles. This article examines the current state of the field and offers a path for moving closer to fulfilling this promise. The paper is structured in three sections, beginning with an introduction to neuro-architecture, outlining its main objectives and giving an overview of experimental research in the field. Afterward, two methodological limitations attending current brain-imaging architectural research are discussed: the first concerns the limited focus of the research, which is often restricted to the aesthetic dimension of architectural experience; the second concerns practical limitations imposed by the typical experimental tools and methods, which often require participants to remain stationary and prevent naturalistic interaction with architectural surroundings. Next, we propose that the theoretical basis of ecological psychology provides a framework for addressing these limitations and motivates emphasizing the role of embodied exploration in architectural experience, which encompasses but is not limited to aesthetic contemplation. In this section, some basic concepts within ecological psychology and their convergences with architecture are described. Lastly, we introduce Mobile Brain/Body Imaging (MoBI) as one emerging brain imaging approach with the potential to improve the ecological validity of neuro-architecture research. Accordingly, we suggest that combining theoretical and conceptual resources from ecological psychology with state-of-the-art neuroscience methods (Mobile Brain/Body Imaging) is a promising way to bring neuro-architecture closer to accomplishing its scientific and practical goals.

Neural sources of prediction errors detect unrealistic VR interactions.

Objective. Neural interfaces hold significant promise to implicitly track user experience. Their application in virtual and augmented reality (VR/AR) simulations is especially favorable as it allows user assessment without breaking the immersive experience. In VR, designing immersion is one key challenge. Subjective questionnaires are the established metrics to assess the effectiveness of immersive VR simulations. However, administering such questionnaires requires breaking the immersive experience they are supposed to assess.Approach. We present a complimentary metric based on a event-related potentials. For the metric to be robust, the neural signal employed must be reliable. Hence, it is beneficial to target the neural signal's cortical origin directly, efficiently separating signal from noise. To test this new complementary metric, we designed a reach-to-tap paradigm in VR to probe electroencephalography (EEG) and movement adaptation to visuo-haptic glitches. Our working hypothesis was, that these glitches, or violations of the predicted action outcome, may indicate a disrupted user experience.Main results. Using prediction error negativity features, we classified VR glitches with 77% accuracy. We localized the EEG sources driving the classification and found midline cingulate EEG sources and a distributed network of parieto-occipital EEG sources to enable the classification success.Significance. Prediction error signatures from these sources reflect violations of user's predictions during interaction with AR/VR, promising a robust and targeted marker for adaptive user interfaces.

Removal of movement-induced EEG artifacts: current state of the art and guidelines.

Objective:Electroencephalography (EEG) is a non-invasive technique used to record cortical neurons' electrical activity using electrodes placed on the scalp. It has become a promising avenue for research beyond state-of-the-art EEG research that is conducted under static conditions. EEG signals are always contaminated by artifacts and other physiological signals. Artifact contamination increases with the intensity of movement.Approach:In the last decade (since 2010), researchers have started to implement EEG measurements in dynamic setups to increase the overall ecological validity of the studies. Many different methods are used to remove non-brain activity from the EEG signal, and there are no clear guidelines on which method should be used in dynamic setups and for specific movement intensities.Main results:Currently, the most common methods for removing artifacts in movement studies are methods based on independent component analysis. However, the choice of method for artifact removal depends on the type and intensity of movement, which affects the characteristics of the artifacts and the EEG parameters of interest. When dealing with EEG under non-static conditions, special care must be taken already in the designing period of an experiment. Software and hardware solutions must be combined to achieve sufficient removal of unwanted signals from EEG measurements.Significance:We have provided recommendations for the use of each method depending on the intensity of the movement and highlighted the advantages and disadvantages of the methods. However, due to the current gap in the literature, further development and evaluation of methods for artifact removal in EEG data during locomotion is needed.

Removal of movement-induced EEG artifacts: current state of the art and guidelines.

Electroencephalography (EEG) is a non-invasive technique used to record cortical neurons' electrical activity using electrodes placed on the scalp. It has become a promising avenue for research beyond state-of-the-art EEG research that is conducted under static conditions. EEG signals are always contaminated by artifacts and other physiological signals. Artifact contamination increases with the intensity of movement. In the last decade (since 2010), researchers have started to implement EEG measurements in dynamic setups to increase the overall ecological validity of the studies. Many different methods are used to remove non-brain activity from the EEG signal, and there are no clear guidelines on which method should be used in dynamic setups and for specific movement intensities. Currently, the most common methods for removing artifacts in movement studies are methods based on independent component analysis (ICA). However, the choice of method for artifact removal depends on the type and intensity of movement, which affects the characteristics of the artifacts and the EEG parameters of interest. When dealing with EEG under non-static conditions, special care must be taken already in the designing period of an experiment. Software and hardware solutions must be combined to achieve sufficient removal of unwanted signals from EEG measurements. We have provided recommendations for the use of each method depending on the intensity of the movement and highlighted the advantages and disadvantages of the methods. However, due to the current gap in the literature, further development and evaluation of methods for artifact removal in EEG data during locomotion is needed.