Neural correlates of visual and tactile path integration and their task related modulation.

Self-motion induces sensory signals that allow to determine travel distance (path integration). For veridical path integration, one must distinguish self-generated from externally induced sensory signals. Predictive coding has been suggested to attenuate self-induced sensory responses, while task relevance can reverse the attenuating effect of prediction. But how is self-motion processing affected by prediction and task demands, and do effects generalize across senses? In this fMRI study, we investigated visual and tactile self-motion processing and its modulation by task demands. Visual stimuli simulated forward self-motion across a ground plane. Tactile self-motion stimuli were delivered by airflow across the subjects' forehead. In one task, subjects replicated a previously observed distance (Reproduction/Active; high behavioral demand) of passive self-displacement (Reproduction/Passive). In a second task, subjects travelled a self-chosen distance (Self/Active; low behavioral demand) which was recorded and played back to them (Self/Passive). For both tasks and sensory modalities, Active as compared to Passive trials showed enhancement in early visual areas and suppression in higher order areas of the inferior parietal lobule (IPL). Contrasting high and low demanding active trials yielded supramodal enhancement in the anterior insula. Suppression in the IPL suggests this area to be a comparator of sensory self-motion signals and predictions thereof.

Perceiving your hand moving: BOLD suppression in sensory cortices and the role of the cerebellum in the detection of feedback delays.

Sensory consequences of self-generated as opposed to externally generated movements are perceived as less intense and lead to less neural activity in corresponding sensory cortices, presumably due to predictive mechanisms. Self-generated sensory inputs have been mostly studied in a single modality, using abstract feedback, with control conditions not differentiating efferent from reafferent feedback. Here we investigated the neural processing of (a) naturalistic action-feedback associations of (b) self-generated versus externally generated movements, and (c) how an additional (auditory) modality influences neural processing and detection of delays. Participants executed wrist movements using a passive movement device (PMD) as they watched their movements in real time or with variable delays (0-417 ms). The task was to judge whether there was a delay between the movement and its visual feedback. In the externally generated condition, movements were induced by the PMD to disentangle efferent from reafferent feedback. Half of the trials involved auditory beeps coupled to the onset of the visual feedback. We found reduced BOLD activity in visual, auditory, and somatosensory areas during self-generated compared with externally generated movements in unimodal and bimodal conditions. Anterior and posterior cerebellar areas were engaged for trials in which action-feedback delays were detected for self-generated movements. Specifically, the left cerebellar lobule IX was functionally connected with the right superior occipital gyrus. The results indicate efference copy-based predictive mechanisms specific to self-generated movements, leading to BOLD suppression in sensory areas. In addition, our results support the cerebellum's role in the detection of temporal prediction errors during our actions and their consequences.

Distinct Roles for the Cerebellum, Angular Gyrus, and Middle Temporal Gyrus in Action-Feedback Monitoring.

Action-feedback monitoring is essential to ensure meaningful interactions with the external world. This process involves generating efference copy-based sensory predictions and comparing these with the actual action-feedback. As neural correlates of comparator processes, previous fMRI studies have provided heterogeneous results, including the cerebellum, angular and middle temporal gyrus. However, these studies usually comprised only self-generated actions. Therefore, they might have induced not only action-based prediction errors, but also general sensory mismatch errors. Here, we aimed to disentangle these processes using a custom-made fMRI-compatible movement device, generating active and passive hand movements with identical sensory feedback. Online visual feedback of the hand was presented with a variable delay. Participants had to judge whether the feedback was delayed. Activity in the right cerebellum correlated more positively with delay in active than in passive trials. Interestingly, we also observed activation in the angular and middle temporal gyri, but across both active and passive conditions. This suggests that the cerebellum is a comparator area specific to voluntary action, whereas angular and middle temporal gyri seem to detect more general intersensory conflict. Correlations with behavior and cerebellar activity nevertheless suggest involvement of these temporoparietal areas in processing and awareness of temporal discrepancies in action-feedback monitoring.

Voluntary and Involuntary Movements Widen the Window of Subjective Simultaneity.

Forming a coherent percept of an event requires different sensory inputs originating from the event to be bound. Perceiving synchrony aids in binding of these inputs. In two experiments, we investigated how voluntary movements influence the perception of simultaneity, by measuring simultaneity judgments (SJs) for an audiovisual (AV) stimulus pair triggered by a voluntary button press. In Experiment 1, we manipulated contiguity between the action and its consequences by introducing delays between the button press and the AV stimulus pair. We found a widened window of subjective simultaneity (WSS) when the action-feedback relationship was time contiguous. Introducing a delay narrowed the WSS, suggesting that the wider WSS around the time of an action might facilitate perception of simultaneity. In Experiment 2, we introduced an involuntary condition using an externally controlled button to assess the influence of action-related predictive processes on SJs. We found a widened WSS around the action time, regardless of movement type, supporting the influence of causal relations in the perception of synchrony. Interestingly, the slopes of the psychometric functions in the voluntary condition were significantly steeper than the slopes in the involuntary condition, suggesting a role of action-related predictive mechanisms in making SJs more precise.

The angular gyrus is a supramodal comparator area in action-outcome monitoring.

Predicting and processing the sensory consequences of one's own actions is essential to enable successful interactions with the environment. Previous studies have suggested that the angular gyrus detects discrepancies between predicted and actual action consequences, at least for unimodal feedback. However, most actions lead to multisensory consequences, raising the question whether previous models can sufficiently explain action-outcome processing. Here, we investigated neural comparator processes during detection of delays between action and unimodal or bimodal consequences in human subjects with fMRI, using parametric and connectivity analyses. Participants had to perform button presses, which led to the presentation of either a dot on the screen, a tone, or both, presented with a variable delay after the button press. Participants were asked to judge whether there was a delay between action and feedback. Activity in the angular gyrus correlated positively with delay for both visual, auditory, and audio-visual action consequences. Furthermore, the angular gyrus was functionally connected with midline structures such as the posterior cingulate cortex and precuneus in all conditions. Our results show that the angular gyrus is (1) a supramodal area, sensitive to delays in multiple modalities, and (2) functionally connected with self-referential areas during delay detection of both unimodal and bimodal action consequences. Overall, our results suggest that the angular gyrus functions as a mediator between perception and interpretation, and that this process is remarkably similar for unimodal and bimodal action consequences.

Predicting the Multisensory Consequences of One's Own Action: BOLD Suppression in Auditory and Visual Cortices.

Predictive mechanisms are essential to successfully interact with the environment and to compensate for delays in the transmission of neural signals. However, whether and how we predict multisensory action outcomes remains largely unknown. Here we investigated the existence of multisensory predictive mechanisms in a context where actions have outcomes in different modalities. During fMRI data acquisition auditory, visual and auditory-visual stimuli were presented in active and passive conditions. In the active condition, a self-initiated button press elicited the stimuli with variable short delays (0-417ms) between action and outcome, and participants had to detect the presence of a delay for auditory or visual outcome (task modality). In the passive condition, stimuli appeared automatically, and participants had to detect the number of stimulus modalities (unimodal/bimodal). For action consequences compared to identical but unpredictable control stimuli we observed suppression of the blood oxygen level depended (BOLD) response in a broad network including bilateral auditory and visual cortices. This effect was independent of task modality or stimulus modality and strongest for trials where no delay was detected (undetected

Predicting the sensory consequences of one's own action: First evidence for multisensory facilitation.

Predicting the sensory consequences of our own actions contributes to efficient sensory processing and might help distinguish the consequences of self- versus externally generated actions. Previous research using unimodal stimuli has provided evidence for the existence of a forward model, which explains how such sensory predictions are generated and used to guide behavior. However, whether and how we predict multisensory action outcomes remains largely unknown. Here, we investigated this question in two behavioral experiments. In Experiment 1, we presented unimodal (visual or auditory) and bimodal (visual and auditory) sensory feedback with various delays after a self-initiated buttonpress. Participants had to report whether they detected a delay between their buttonpress and the stimulus in the predefined task modality. In Experiment 2, the sensory feedback and task were the same as in Experiment 1, but in half of the trials the action was externally generated. We observed enhanced delay detection for bimodal relative to unimodal trials, with better performance in general for actively generated actions. Furthermore, in the active condition, the bimodal advantage was largest when the stimulus in the task-irrelevant modality was not delayed-that is, when it was time-contiguous with the action-as compared to when both the task-relevant and task-irrelevant modalities were delayed. This specific enhancement for trials with a nondelayed task-irrelevant modality was absent in the passive condition. These results suggest that a forward model creates predictions for multiple modalities, and consequently contributes to multisensory interactions in the context of action.