A computational account of transsaccadic attentional allocation based on visual gain fields.

Coordination of goal-directed behavior depends on the brain's ability to recover the locations of relevant objects in the world. In humans, the visual system encodes the spatial organization of sensory inputs, but neurons in early visual areas map objects according to their retinal positions, rather than where they are in the world. How the brain computes world-referenced spatial information across eye movements has been widely researched and debated. Here, we tested whether shifts of covert attention are sufficiently precise in space and time to track an object's real-world location across eye movements. We found that observers' attentional selectivity is remarkably precise and is barely perturbed by the execution of saccades. Inspired by recent neurophysiological discoveries, we developed an observer model that rapidly estimates the real-world locations of objects and allocates attention within this reference frame. The model recapitulates the human data and provides a parsimonious explanation for previously reported phenomena in which observers allocate attention to task-irrelevant locations across eye movements. Our findings reveal that visual attention operates in real-world coordinates, which can be computed rapidly at the earliest stages of cortical processing.

DeepGaze III: Modeling free-viewing human scanpaths with deep learning.

Humans typically move their eyes in "scanpaths" of fixations linked by saccades. Here we present DeepGaze III, a new model that predicts the spatial location of consecutive fixations in a free-viewing scanpath over static images. DeepGaze III is a deep learning-based model that combines image information with information about the previous fixation history to predict where a participant might fixate next. As a high-capacity and flexible model, DeepGaze III captures many relevant patterns in the human scanpath data, setting a new state of the art in the MIT300 dataset and thereby providing insight into how much information in scanpaths across observers exists in the first place. We use this insight to assess the importance of mechanisms implemented in simpler, interpretable models for fixation selection. Due to its architecture, DeepGaze III allows us to disentangle several factors that play an important role in fixation selection, such as the interplay of scene content and scanpath history. The modular nature of DeepGaze III allows us to conduct ablation studies, which show that scene content has a stronger effect on fixation selection than previous scanpath history in our main dataset. In addition, we can use the model to identify scenes for which the relative importance of these sources of information differs most. These data-driven insights would be difficult to accomplish with simpler models that do not have the computational capacity to capture such patterns, demonstrating an example of how deep learning advances can be used to contribute to scientific understanding.

Nanoscale Photoexcited Carrier Dynamics in Perovskites.

The optoelectronic properties of lead halide perovskite thin films can be tuned through compositional variations and strain, but the associated nanocrystalline structure makes it difficult to untangle the link between composition, processing conditions, and ultimately material properties and degradation. Here, we study the effect of processing conditions and degradation on the local photoconductivity dynamics in [(CsPbI3)0.05(FAPbI3)0.85(MAPbBr3)0.15] and (FA0.7Cs0.3PbI3) perovskite thin films using temporally and spectrally resolved microwave near-field microscopy with a temporal resolution as high as 5 ns and a spatial resolution better than 50 nm. For the latter FACs formulation, we find a clear effect of the process annealing temperature on film morphology, stability, and spatial photoconductivity distribution. After exposure of samples to ambient conditions and illumination, we find spectral evidence of halide segregation-induced degradation below the instrument resolution limit for the mixed halide formulation, while we find a clear spatially inhomogeneous increase in the carrier lifetime for the FACs formulation annealed at 180 °C.

Semantic object-scene inconsistencies affect eye movements, but not in the way predicted by contextualized meaning maps.

Semantic information is important in eye movement control. An important semantic influence on gaze guidance relates to object-scene relationships: objects that are semantically inconsistent with the scene attract more fixations than consistent objects. One interpretation of this effect is that fixations are driven toward inconsistent objects because they are semantically more informative. We tested this explanation using contextualized meaning maps, a method that is based on crowd-sourced ratings to quantify the spatial distribution of context-sensitive "meaning" in images. In Experiment 1, we compared gaze data and contextualized meaning maps for images, in which objects-scene consistency was manipulated. Observers fixated more on inconsistent versus consistent objects. However, contextualized meaning maps did not assign higher meaning to image regions that contained semantic inconsistencies. In Experiment 2, a large number of raters evaluated image-regions, which were deliberately selected for their content and expected meaningfulness. The results suggest that the same scene locations were experienced as slightly less meaningful when they contained inconsistent compared to consistent objects. In summary, we demonstrated that - in the context of our rating task - semantically inconsistent objects are experienced as less meaningful than their consistent counterparts and that contextualized meaning maps do not capture prototypical influences of image meaning on gaze guidance.

Spatial structure, phase, and the contrast of natural images.

The sensitivity of the human visual system is thought to be shaped by environmental statistics. A major endeavor in vision science, therefore, is to uncover the image statistics that predict perceptual and cognitive function. When searching for targets in natural images, for example, it has recently been proposed that target detection is inversely related to the spatial similarity of the target to its local background. We tested this hypothesis by measuring observers' sensitivity to targets that were blended with natural image backgrounds. Targets were designed to have a spatial structure that was either similar or dissimilar to the background. Contrary to masking from similarity, we found that observers were most sensitive to targets that were most similar to their backgrounds. We hypothesized that a coincidence of phase alignment between target and background results in a local contrast signal that facilitates detection when target-background similarity is high. We confirmed this prediction in a second experiment. Indeed, we show that, by solely manipulating the phase of a target relative to its background, the target can be rendered easily visible or undetectable. Our study thus reveals that, in addition to its structural similarity, the phase of the target relative to the background must be considered when predicting detection sensitivity in natural images.

Intracranial pressure waveform characteristics in idiopathic normal pressure hydrocephalus and late-onset idiopathic aqueductal stenosis.

BACKGROUND: Idiopathic normal pressure hydrocephalus (iNPH) and late-onset idiopathic aqueductal stenosis (LIAS) are two forms of chronic adult hydrocephalus of different aetiology. We analysed overnight intracranial pressure (ICP) monitoring to elucidate ICP waveform changes characteristic for iNPH and LIAS to better understand pathophysiological processes of both diseases. METHODS: 98 patients with iNPH and 14 patients with LIAS from two neurosurgical centres were included. All patients underwent diagnostic overnight computerised ICP monitoring with calculation of mean ICP, ICP heartbeat related pulse wave amplitude calculated in the frequency domain (AMP) and the time domain (MWA), index of cerebrospinal compensatory reserve (RAP) and power of slow vasogenic waves (SLOW). RESULTS: ICP was higher in LIAS than iNPH patients (9.3 ± 3.0 mmHg versus 5.4 ± 4.2 mmHg, p = 0.001). AMP and MWA were higher in iNPH versus LIAS (2.36 ± 0.91 mmHg versus 1.81 ± 0.59 mmHg for AMP, p = 0.012; 6.0 ± 2.0 mmHg versus 4.9 ± 1.2 mmHg for MWA, p = 0.049). RAP and SLOW indicated impaired reserve capacity and compliance in both diseases, but did not differ between groups. INPH patients were older than LIAS patients (77 ± 6 years versus 54 ± 14 years, p < 0.001). CONCLUSIONS: ICP is higher in LIAS than in iNPH patients, likely due to the chronically obstructed CSF flow through the aqueduct, but still in a range considered normal. Interestingly, AMP/MWA was higher in iNPH patients, suggesting a possible role of high ICP pulse pressure amplitudes in iNPH pathophysiology. Cerebrospinal reserve capacity and intracranial compliance is impaired in both groups and the pressure-volume relationship might be shifted towards lower ICP values in iNPH. The physiological influence of age on ICP and AMP/MWA requires further research.

There is no evidence that meaning maps capture semantic information relevant to gaze guidance: Reply to Henderson, Hayes, Peacock, and Rehrig (2021).

The concerns raised by Henderson, Hayes, Peacock, and Rehrig (2021) are based on misconceptions of our work. We show that Meaning Maps (MMs) do not predict gaze guidance better than a state-of-the-art saliency model that is based on semantically-neutral, high-level features. We argue that there is therefore no evidence to date that MMs index anything beyond these features. Furthermore, we show that although alterations in meaning cause changes in gaze guidance, MMs fail to capture these alterations. We agree that semantic information is important in the guidance of eye-movements, but the contribution of MMs for understanding its role remains elusive.

The Impact of Shape-Based Cue Discriminability on Attentional Performance.

With rapidly developing technology, visual cues became a powerful tool for deliberate guiding of attention and affecting human performance. Using cues to manipulate attention introduces a trade-off between increased performance in cued, and decreased in not cued, locations. For higher efficacy of visual cues designed to purposely direct user's attention, it is important to know how manipulation of cue properties affects attention. In this verification study, we addressed how varying cue complexity impacts the allocation of spatial endogenous covert attention in space and time. To gradually vary cue complexity, the discriminability of the cue was systematically modulated using a shape-based design. Performance was compared in attended and unattended locations in an orientation-discrimination task. We evaluated additional temporal costs due to processing of a more complex cue by comparing performance at two different inter-stimulus intervals. From preliminary data, attention scaled with cue discriminability, even for supra-threshold cue discriminability. Furthermore, individual cue processing times partly impacted performance for the most complex, but not simpler cues. We conclude that, first, cue complexity expressed by discriminability modulates endogenous covert attention at supra-threshold cue discriminability levels, with increasing benefits and decreasing costs; second, it is important to consider the temporal processing costs of complex visual cues.

Five points to check when comparing visual perception in humans and machines.

With the rise of machines to human-level performance in complex recognition tasks, a growing amount of work is directed toward comparing information processing in humans and machines. These studies are an exciting chance to learn about one system by studying the other. Here, we propose ideas on how to design, conduct, and interpret experiments such that they adequately support the investigation of mechanisms when comparing human and machine perception. We demonstrate and apply these ideas through three case studies. The first case study shows how human bias can affect the interpretation of results and that several analytic tools can help to overcome this human reference point. In the second case study, we highlight the difference between necessary and sufficient mechanisms in visual reasoning tasks. Thereby, we show that contrary to previous suggestions, feedback mechanisms might not be necessary for the tasks in question. The third case study highlights the importance of aligning experimental conditions. We find that a previously observed difference in object recognition does not hold when adapting the experiment to make conditions more equitable between humans and machines. In presenting a checklist for comparative studies of visual reasoning in humans and machines, we hope to highlight how to overcome potential pitfalls in design and inference.

Meaning maps and saliency models based on deep convolutional neural networks are insensitive to image meaning when predicting human fixations.

Eye movements are vital for human vision, and it is therefore important to understand how observers decide where to look. Meaning maps (MMs), a technique to capture the distribution of semantic information across an image, have recently been proposed to support the hypothesis that meaning rather than image features guides human gaze. MMs have the potential to be an important tool far beyond eye-movements research. Here, we examine central assumptions underlying MMs. First, we compared the performance of MMs in predicting fixations to saliency models, showing that DeepGaze II - a deep neural network trained to predict fixations based on high-level features rather than meaning - outperforms MMs. Second, we show that whereas human observers respond to changes in meaning induced by manipulating object-context relationships, MMs and DeepGaze II do not. Together, these findings challenge central assumptions underlying the use of MMs to measure the distribution of meaning in images.

Complications of elective intracranial pressure monitoring in adult hydrocephalus.

Continuous invasive monitoring of intracranial pressure (ICP) can be used in the diagnosis and management of various types of chronic cerebrospinal fluid (CSF) circulation disorders, such as hydrocephalus, shunt dysfunction and idiopathic intracranial hypertension. The risk profile and incidence of adverse events of this surgical procedure in this patient population is not well established. We aimed to investigate and describe the risks of ICP monitoring in adult patients with chronic CSF circulation disorders. We analysed 152 patients undergoing continuous ICP monitoring between 2010 and 2019, mainly for idiopathic normal pressure hydrocephalus. The average duration of ICP monitoring was 17 h 51 min. We observed no major adverse events, such as symptomatic intracranial haemorrhage, intracranial infection, or persistent neurological deficit. Minor complications were seen in 7% of patients and included accidental removal of the ICP probe in 4 patients, inability to remove the probe requiring surgical removal in 2 patients and single generalised seizures in 2 patients. In summary, the risk of serious adverse events and complications from invasive ICP monitoring in chronic CSF circulation disorders in adult patients appears to be low.

Spatially Resolved Persistent Photoconductivity in MoS2-WS2 Lateral Heterostructures.

The optical and electronic properties of 2D semiconductors are intrinsically linked via the strong interactions between optically excited bound species and free carriers. Here we use near-field scanning microwave microscopy (SMM) to image spatial variations in photoconductivity in MoS2-WS2 lateral multijunction heterostructures using photon energy-resolved narrowband illumination. We find that the onset of photoconductivity in individual domains corresponds to the optical absorption onset, confirming that the tightly bound excitons in transition metal dichalcogenides can nonetheless dissociate into free carriers. These photogenerated carriers are most likely n-type and are seen to persist for up to days. Informed by finite element modeling we reveal that they can increase the carrier density by up to 200 times. This persistent photoconductivity appears to be dominated by contributions from the multilayer MoS2 domains, and we attribute the flake-wide response in part to charge transfer across the heterointerface. Spatial correlation of our SMM imaging with photoluminescence (PL) mapping confirms the strong link between PL peak emission photon energy, PL intensity, and the local accumulated charge. This work reveals the spatially and temporally complex optoelectronic response of these systems and cautions that properties measured during or after illumination may not reflect the true dark state of these materials but rather a metastable charged state.

Correction to: Detecting distortions of peripherally presented letter stimuli under crowded conditions.

We discovered an error in the implementation of the function used to generate radial frequency (RF) distortions1 in our article (Wallis, Tobias, Bethge, & Wichmann, 2017).

Image content is more important than Bouma's Law for scene metamers.

We subjectively perceive our visual field with high fidelity, yet peripheral distortions can go unnoticed and peripheral objects can be difficult to identify (crowding). Prior work showed that humans could not discriminate images synthesised to match the responses of a mid-level ventral visual stream model when information was averaged in receptive fields with a scaling of about half their retinal eccentricity. This result implicated ventral visual area V2, approximated 'Bouma's Law' of crowding, and has subsequently been interpreted as a link between crowding zones, receptive field scaling, and our perceptual experience. However, this experiment never assessed natural images. We find that humans can easily discriminate real and model-generated images at V2 scaling, requiring scales at least as small as V1 receptive fields to generate metamers. We speculate that explaining why scenes look as they do may require incorporating segmentation and global organisational constraints in addition to local pooling.

What the FLOQ? A quality improvement project to reduce unnecessary paediatric respiratory viral swabs in a peripheral metropolitan hospital.

AIM: To reduce the number of paediatric respiratory viral swabs (locally referred to as a FLOQ) performed across the authors clinical centre from a baseline of over 800 ($38 000) per year by 25% over 4 months from 6 February 2017 to 31 May 2017. METHODS: A quality improvement project 'What the FLOQ?' (WTF) was instigated from 6 February 2017 to complement the Emergency Department (ED) 'Sensible Test Ordering Process' project from 1 April 2017. Stakeholder engagement across ED and general paediatric staff was sought. Alterations in practice included education of staff, targeted feedback to groups frequently ordering a FLOQ and rationalising patients appropriate for testing. Monthly requests were tallied on a run chart for FLOQs ordered in ED and the paediatric ward. A monthly audit of FLOQs performed on ED-discharged patients was conducted with feedback. RESULTS: Total FLOQ swabs decreased by 55% from 336 (February to May 2016) to 151 (February to May 2017). ED performed 66% less FLOQs from 237 (February to May 2016) to 82 (February to May 2017). There was no increase in the number of FLOQs performed on the paediatric ward February to May 2017. Monthly auditing of ED discharged patients under 2 years with a FLOQ went from 40 to 3%. CONCLUSION: Rationalising patient groups appropriate for testing with targeted feedback and broad stakeholder engagement successfully reduced FLOQs performed by 55%. This has projected savings of over $21 000 by 12 months. WTF has reduced the number of invasive patient procedures performed, benefitting staff and patients. Sustaining this change will be achieved through ongoing staff education on rationalisation criteria and consultant only requests outside of these parameters.

A parametric texture model based on deep convolutional features closely matches texture appearance for humans.

Our visual environment is full of texture-"stuff" like cloth, bark, or gravel as distinct from "things" like dresses, trees, or paths-and humans are adept at perceiving subtle variations in material properties. To investigate image features important for texture perception, we psychophysically compare a recent parametric model of texture appearance (convolutional neural network [CNN] model) that uses the features encoded by a deep CNN (VGG-19) with two other models: the venerable Portilla and Simoncelli model and an extension of the CNN model in which the power spectrum is additionally matched. Observers discriminated model-generated textures from original natural textures in a spatial three-alternative oddity paradigm under two viewing conditions: when test patches were briefly presented to the near-periphery ("parafoveal") and when observers were able to make eye movements to all three patches ("inspection"). Under parafoveal viewing, observers were unable to discriminate 10 of 12 original images from CNN model images, and remarkably, the simpler Portilla and Simoncelli model performed slightly better than the CNN model (11 textures). Under foveal inspection, matching CNN features captured appearance substantially better than the Portilla and Simoncelli model (nine compared to four textures), and including the power spectrum improved appearance matching for two of the three remaining textures. None of the models we test here could produce indiscriminable images for one of the 12 textures under the inspection condition. While deep CNN (VGG-19) features can often be used to synthesize textures that humans cannot discriminate from natural textures, there is currently no uniformly best model for all textures and viewing conditions.

Electronic and Morphological Inhomogeneities in Pristine and Deteriorated Perovskite Photovoltaic Films.

We perform scanning microwave microscopy (SMM) to study the spatially varying electronic properties and related morphology of pristine and degraded methylammonium lead-halide (MAPI) perovskite films fabricated under different ambient humidity. We find that higher processing humidity leads to the emergence of increased conductivity at the grain boundaries but also correlates with the appearance of resistive grains that contain PbI2. Deteriorated films show larger and increasingly insulating grain boundaries as well as spatially localized regions of reduced conductivity within grains. These results suggest that while humidity during film fabrication primarily benefits device properties due to the passivation of traps at the grain boundaries and self-doping, it also results in the emergence of PbI2-containing grains. We further establish that MAPI film deterioration under ambient conditions proceeds via the spatially localized breakdown of film conductivity, both at grain boundaries and within grains, due to local variations in susceptibility to deterioration. These results confirm that PbI2 has both beneficial and adverse effects on device performance and provide new means for device optimization by revealing spatial variations in sample conductivity as well as morphological differences in resistance to sample deterioration.

Detecting distortions of peripherally presented letter stimuli under crowded conditions.

When visual features in the periphery are close together they become difficult to recognize: something is present but it is unclear what. This is called "crowding". Here we investigated sensitivity to features in highly familiar shapes (letters) by applying spatial distortions. In Experiment 1, observers detected which of four peripherally presented (8 deg of retinal eccentricity) target letters was distorted (spatial 4AFC). The letters were presented either isolated or surrounded by four undistorted flanking letters, and distorted with one of two types of distortion at a range of distortion frequencies and amplitudes. The bandpass noise distortion ("BPN") technique causes spatial distortions in Cartesian space, whereas radial frequency distortion ("RF") causes shifts in polar coordinates. Detecting distortions in target letters was more difficult in the presence of flanking letters, consistent with the effect of crowding. The BPN distortion type showed evidence of tuning, with sensitivity to distortions peaking at approximately 6.5 c/deg for unflanked letters. The presence of flanking letters causes this peak to rise to approximately 8.5 c/deg. In contrast to the tuning observed for BPN distortions, RF distortion sensitivity increased as the radial frequency of distortion increased. In a series of follow-up experiments, we found that sensitivity to distortions is reduced when flanking letters were also distorted, that this held when observers were required to report which target letter was undistorted, and that this held when flanker distortions were always detectable. The perception of geometric distortions in letter stimuli is impaired by visual crowding.

Testing models of peripheral encoding using metamerism in an oddity paradigm.

Most of the visual field is peripheral, and the periphery encodes visual input with less fidelity compared to the fovea. What information is encoded, and what is lost in the visual periphery? A systematic way to answer this question is to determine how sensitive the visual system is to different kinds of lossy image changes compared to the unmodified natural scene. If modified images are indiscriminable from the original scene, then the information discarded by the modification is not important for perception under the experimental conditions used. We measured the detectability of modifications of natural image structure using a temporal three-alternative oddity task, in which observers compared modified images to original natural scenes. We consider two lossy image transformations, Gaussian blur and Portilla and Simoncelli texture synthesis. Although our paradigm demonstrates metamerism (physically different images that appear the same) under some conditions, in general we find that humans can be capable of impressive sensitivity to deviations from natural appearance. The representations we examine here do not preserve all the information necessary to match the appearance of natural scenes in the periphery.

Near-Field Control and Imaging of Free Charge Carrier Variations in GaN Nanowires.

Despite their uniform crystallinity, the shape and faceting of semiconducting nanowires (NWs) can give rise to variations in structure and associated electronic properties. Here we develop a hybrid scanning probe-based methodology to investigate local variations in electronic structure across individual n-doped GaN NWs integrated into a transistor device. We perform scanning microwave microscopy (SMM), which we combine with scanning gate microscopy (SGM) to determine the free-carrier SMM signal contribution and image local charge carrier density variations. In particular, we find significant variations in free carriers across NWs, with a higher carrier density at the wire facets. By increasing the local carrier density through tip-gating, we find that the tip injects current into the NW with strongly localized current when positioned over the wire vertices. These results suggest that the strong variations in electronic properties observed within NWs have significant implications for device design and may lead to new paths to optimization.