The impact of joint attention on the sound-induced flash illusions.

Humans coordinate their focus of attention with others, either by gaze following or prior agreement. Though the effects of joint attention on perceptual and cognitive processing tend to be examined in purely visual environments, they should also show in multisensory settings. According to a prevalent hypothesis, joint attention enhances visual information encoding and processing, over and above individual attention. If two individuals jointly attend to the visual components of an audiovisual event, this should affect the weighing of visual information during multisensory integration. We tested this prediction in this preregistered study, using the well-documented sound-induced flash illusions, where the integration of an incongruent number of visual flashes and auditory beeps results in a single flash being seen as two (fission illusion) and two flashes as one (fusion illusion). Participants were asked to count flashes either alone or together, and expected to be less prone to both fission and fusion illusions when they jointly attended to the visual targets. However, illusions were as frequent when people attended to the flashes alone or with someone else, even though they responded faster during joint attention. Our results reveal the limitations of the theory that joint attention enhances visual processing as it does not affect temporal audiovisual integration.

Why There Is a Vestibular Sense, or How Metacognition Individuates the Senses.

Should the vestibular system be counted as a sense? This basic conceptual question remains surprisingly controversial. While it is possible to distinguish specific vestibular organs, it is not clear that this suffices to identify a genuine vestibular sense because of the supposed absence of a distinctive vestibular personal-level manifestation. The vestibular organs instead contribute to more general multisensory representations, whose name still suggest that they have a distinct 'sensory' contribution. The vestibular case shows a good example of the challenge of individuating the senses when multisensory interactions are the norm, neurally, representationally and phenomenally. Here, we propose that an additional metacognitive criterion can be used to single out a distinct sense, besides the existence of specific organs and despite the fact that the information coming from these organs is integrated with other sensory information. We argue that it is possible for human perceivers to monitor information coming from distinct organs, despite their integration, as exhibited and measured through metacognitive performance. Based on the vestibular case, we suggest that metacognitive awareness of the information coming from sensory organs constitutes a new criterion to individuate a sense through both physiological and personal criteria. This new way of individuating the senses accommodates both the specialised nature of sensory receptors as well as the intricate multisensory aspect of neural processes and experience, while maintaining the idea that each sense contributes something special to how we monitor the world and ourselves, at the subjective level.

Towards dynamic modeling of visual-vestibular conflict detection.

Visual-vestibular mismatch is a common occurrence, with causes ranging from vehicular travel, to vestibular dysfunction, to virtual reality displays. Behavioral and physiological consequences of this mismatch include adaptation of reflexive eye movements, oscillopsia, vertigo, and nausea. Despite this significance, we still do not have a good understanding of how the nervous system evaluates visual-vestibular conflict. Here we review research that quantifies perceptual sensitivity to visual-vestibular conflict and factors that mediate this sensitivity, such as noise on visual and vestibular sensory estimates. We emphasize that dynamic modeling methods are necessary to investigate how the nervous system monitors conflict between time-varying visual and vestibular signals, and we present a simple example of a drift-diffusion model for visual-vestibular conflict detection. The model makes predictions for detection of conflict arising from changes in both visual gain and latency. We conclude with discussion of topics for future research.

Sensitivity to Visual Speed Modulation in Head-Mounted Displays Depends on Fixation.

A primary cause of simulator sickness in head-mounted displays (HMDs) is conflict between the visual scene displayed to the user and the visual scene expected by the brain when the user's head is in motion. It is useful to measure perceptual sensitivity to visual speed modulation in HMDs because conditions that minimize this sensitivity may prove less likely to elicit simulator sickness. In prior research, we measured sensitivity to visual gain modulation during slow, passive, full-body yaw rotations and observed that sensitivity was reduced when subjects fixated a head-fixed target compared with when they fixated a scene-fixed target. In the current study, we investigated whether this pattern of results persists when (1) movements are faster, active head turns, and (2) visual stimuli are presented on an HMD rather than on a monitor. Subjects wore an Oculus Rift CV1 HMD and viewed a 3D scene of white points on a black background. On each trial, subjects moved their head from a central position to face a 15° eccentric target. During the head movement they fixated a point that was either head-fixed or scene-fixed, depending on condition. They then reported if the visual scene motion was too fast or too slow. Visual speed on subsequent trials was modulated according to a staircase procedure to find the speed increment that was just noticeable. Sensitivity to speed modulation during active head movement was reduced during head-fixed fixation, similar to what we observed during passive whole-body rotation. We conclude that fixation of a head-fixed target is an effective way to reduce sensitivity to visual speed modulation in HMDs, and may also be an effective strategy to reduce susceptibility to simulator sickness.

Insufficient compensation for self-motion during perception of object speed: The vestibular Aubert-Fleischl phenomenon.

To estimate object speed with respect to the self, retinal signals must be summed with extraretinal signals that encode the speed of eye and head movement. Prior work has shown that differences in perceptual estimates of object speed based on retinal and oculomotor signals lead to biased percepts such as the Aubert-Fleischl phenomenon (AF), in which moving targets appear slower when pursued. During whole-body movement, additional extraretinal signals, such as those from the vestibular system, may be used to transform object speed estimates from a head-centered to a world-centered reference frame. Here we demonstrate that whole-body pursuit in the form of passive yaw rotation, which stimulates the semicircular canals of the vestibular system, leads to a slowing of perceived object speed similar to the classic oculomotor AF. We find that the magnitude of the vestibular and oculomotor AF is comparable across a range of speeds, despite the different types of input signal involved. This covariation might hint at a common modality-independent mechanism underlying the AF in both cases.

The influence of obstacles on grasp planning.

When reaching to grasp for an object in the absence of obstacles, the choice of contact points is highly consistent within and between healthy humans, suggesting a preplanning of grasping movements (Gilster et al. in Exp Brain Res 217:137-151, 2012). In real life, objects may obstruct the favored contact points at a target object, requiring adjustments to avoid collision. In the present study, we investigated how an obstacle that directly obstructs the favored contact points for two-digit grasping changes the planning and execution of reach-to-grasp movements. Furthermore, we elucidated to what extent an obstacle placed at various angular positions around the target object (thereby not directly obstructing the favored contact points) still influences trajectories, contact points, and time-related parameters. When obstacles directly obstructed favored contact points participants either chose a completely new contact point or grasped the object only slightly away from the favored contact point. Obstacles located near the favored contact points but not directly obstructing them still resulted in a repulsive effect, meaning that contact points were shifted away from the obstacle to ensure sufficient distance to the obstacle. We found that the position of an obstacle even influences the direction in which the fingers set off. This leads to a deviation of the trajectory very early in the time course, yielding longer movement times if the main contact points are obstructed. Taken together, the early significant influence of obstacles on the grasping movement supports the assumption that grasping movements are preplanned.

Visual-Vestibular Conflict Detection Depends on Fixation.

Visual and vestibular signals are the primary sources of sensory information for self-motion. Conflict among these signals can be seriously debilitating, resulting in vertigo [1], inappropriate postural responses [2], and motion, simulator, or cyber sickness [3-8]. Despite this significance, the mechanisms mediating conflict detection are poorly understood. Here we model conflict detection simply as crossmodal discrimination with benchmark performance limited by variabilities of the signals being compared. In a series of psychophysical experiments conducted in a virtual reality motion simulator, we measure these variabilities and assess conflict detection relative to this benchmark. We also examine the impact of eye movements on visual-vestibular conflict detection. In one condition, observers fixate a point that is stationary in the simulated visual environment by rotating the eyes opposite head rotation, thereby nulling retinal image motion. In another condition, eye movement is artificially minimized via fixation of a head-fixed fixation point, thereby maximizing retinal image motion. Visual-vestibular integration performance is also measured, similar to previous studies [9-12]. We observe that there is a tradeoff between integration and conflict detection that is mediated by eye movements. Minimizing eye movements by fixating a head-fixed target leads to optimal integration but highly impaired conflict detection. Minimizing retinal motion by fixating a scene-fixed target improves conflict detection at the cost of impaired integration performance. The common tendency to fixate scene-fixed targets during self-motion [13] may indicate that conflict detection is typically a higher priority than the increase in precision of self-motion estimation that is obtained through integration.