Just-in-Time Encoding Into Visual Working Memory Is Contingent Upon Constant Availability of External Information.

Humans maintain an intricate balance between storing information in visual working memory (VWM) and just-in-time sampling of the external world, rooted in a trade-off between the cost of maintaining items in VWM versus retrieving information as it is needed. Previous studies have consistently shown that one prerequisite of just-in-time sampling is a high degree of availability of external information, and that introducing a delay before being able to access information led participants to rely less on the external world and more on VWM. However, these studies manipulated availability in such a manner that the cost of sampling was stable and predictable. It is yet unclear whether participants become less reliant on external information when it is more difficult to factor in the cost of sampling that information. In two experiments, participants copied an example layout from the left to the right side of the screen. In Experiment 1, intermittent occlusion of the example layout led participants to attempt to encode more items per inspection than when the layout was constantly available, but this did not consistently result in more correct placements. However, these findings could potentially be explained by inherent differences in how long the example layout could be viewed. Therefore in Experiment 2, the example layout only became available after a gaze-contingent delay, which could be constant or variable. Here, the introduction of any delay led to increased VWM load compared to no delay, although the degree of variability in the delay did not alter behaviour. These results reaffirm that the nature of when we engage VWM is dynamical, and suggest that any disruption to the continuous availability of external information is the main driver of increased VWM usage relative to whether availability is predictable or not.

The impact of acute asymmetric hearing loss on multisensory integration.

Humans have the remarkable ability to integrate information from different senses, which greatly facilitates the detection, localization and identification of events in the environment. About 466 million people worldwide suffer from hearing loss. Yet, the impact of hearing loss on how the senses work together is rarely investigated. Here, we investigate how a common sensory impairment, asymmetric conductive hearing loss (AHL), alters the way our senses interact by examining human orienting behaviour with normal hearing (NH) and acute AHL. This type of hearing loss disrupts auditory localization. We hypothesized that this creates a conflict between auditory and visual spatial estimates and alters how auditory and visual inputs are integrated to facilitate multisensory spatial perception. We analysed the spatial and temporal properties of saccades to auditory, visual and audiovisual stimuli before and after plugging the right ear of participants. Both spatial and temporal aspects of multisensory integration were affected by AHL. Compared with NH, AHL caused participants to make slow, inaccurate and unprecise saccades towards auditory targets. Surprisingly, increased weight on visual input resulted in accurate audiovisual localization with AHL. This came at a cost: saccade latencies for audiovisual targets increased significantly. The larger the auditory localization errors, the less participants were able to benefit from audiovisual integration in terms of saccade latency. Our results indicate that observers immediately change sensory weights to effectively deal with acute AHL and preserve audiovisual accuracy in a way that cannot be fully explained by statistical models of optimal cue integration.

More (corrective) consecutive saccades after a lesion to the posterior parietal cortex.

To reach a target, primary saccades (S1s) are often followed by (corrective) consecutive saccades (S2, and potentially S3, S4, S5), which are based on retinal and extraretinal feedback. Processing these extraretinal signals was found to be significantly impaired by lesions to the posterior parietal cortex (PPC). Recent studies, however, added a more nuanced view to the role of the PPC, where patients with PPC lesions still used extraretinal signals for S2s and perceptual judgements (Fabius et al., 2020; Rath-Wilson & Guitton, 2015). Hence, it seems that a PPC lesion is not disrupting extraretinal processing per se. Yet, a lesion might still result in less reliable processing of extraretinal signals. Here, we investigated whether this lower reliability manifests as decreased or delayed S2 initiation. Patients with PPC lesions (n = 7) and controls (n = 26) performed a prosaccade task where the target either remained visible or was removed after S1 onset. When S1 is removed, accurate S2s (corrections of S1 error) rely solely on extraretinal signals. We analysed S2 quantity and timing using linear mixed-effects modelling and additive hazards analyses. Patients demonstrated slower S1 execution and lower S1 amplitudes than controls, but their S2s still compensated the S1 undershoot, also when they only relied on extraretinal information. Surprisingly, patients showed an increased amount of S2s. This deviation from control behaviour can be seen as suboptimal, but given the decreased accuracy of the primary saccade, it could be optimal for patients to employ more (corrective) consecutive saccades to overcome this inaccuracy.

Eye Movements as Proxy for Visual Working Memory Usage: Increased Reliance on the External World in Korsakoff Syndrome.

In the assessment of visual working memory, estimating the maximum capacity is currently the gold standard. However, traditional tasks disregard that information generally remains available in the external world. Only when to-be-used information is not readily accessible, memory is taxed. Otherwise, people sample information from the environment as a form of cognitive offloading. To investigate how memory deficits impact the trade-off between sampling externally or storing internally, we compared gaze behaviour of individuals with Korsakoff amnesia (n = 24, age range 47-74 years) and healthy controls (n = 27, age range 40-81 years) on a copy task that provoked different strategies by having information freely accessible (facilitating sampling) or introducing a gaze-contingent waiting time (provoking storing). Indeed, patients sampled more often and longer, compared to controls. When sampling became time-consuming, controls reduced sampling and memorised more. Patients also showed reduced and longer sampling in this condition, suggesting an attempt at memorisation. Importantly, however, patients sampled disproportionately more often than controls, whilst accuracy dropped. This finding suggests that amnesia patients sample frequently and do not fully compensate for increased sampling costs by memorising more at once. In other words, Korsakoff amnesia resulted in a heavy reliance on the world as 'external memory'.