Value-driven effects on perceptual averaging.

Perceptual averaging refers to a strategy of encoding the statistical properties of entire sets of objects rather than encoding individual object properties, potentially circumventing the visual system's strict capacity limitations. Prior work has shown that such average representations of set properties, such as its mean size, can be modulated by top-down and bottom-up attention. However, it is unclear to what extent attentional biases through selection history, in the form of value-driven attentional capture, influences this type of summary statistical representation. To investigate, we conducted two experiments in which participants estimated the mean size of a set of heterogeneously sized circles while a previously rewarded color singleton was part of the set. In Experiment 1, all circles were gray, except either the smallest or the largest circle, which was presented in a color previously associated with a reward. When the largest circle in the set was associated with the highest value (as a proxy of selection history), we observed the largest biases, such that perceived mean size scaled linearly with the increasing value of the attended color singleton. In Experiment 2, we introduced a dual-task component in the form of an attentional search task to ensure that the observed bias of reward on perceptual averaging was not fully explained by focusing attention solely on the reward-signaling color singleton. Collectively, findings support the proposal that selection history, like bottom-up and top-down attention, influences perceptual averaging, and that this happens in a flexible manner proportional to the extent to which attention is captured.

Adaptation to average duration.

There has been a recent surge of research examining how the visual system compresses information by representing the average properties of sets of similar objects to circumvent strict capacity limitations. Efficient representation by perceptual averaging helps to maintain the balance between the needs to perceive salient events in the surrounding environment and sustain the illusion of stable and complete perception. Whereas there have been many demonstrations that the visual system encodes spatial average properties, such as average orientation, average size, and average numerosity along single dimensions, there has been no investigation of whether the fundamental nature of average representations extends to the temporal domain. Here, we used an adaptation paradigm to demonstrate that the average duration of a set of sequentially presented stimuli negatively biases the perceived duration of subsequently presented information. This negative adaptation aftereffect is indicative of a fundamental visual property, providing the first evidence that average duration is encoded along a single visual dimension. Our results not only have important implications for how the visual system efficiently encodes redundant information to evaluate salient events as they unfold within the dynamic context of the surrounding environment, but also contribute to the long-standing debate regarding the neural underpinnings of temporal encoding.

Statistical stability and set size exert distinct influences on visual search.

Despite continuous retinal chaos, we perceive the world as stable and complete. This illusion is sustained over consecutive glances by reliance on statistical redundancies inherent in the visual environment. For instance, repeating the average size of a collection of differently sized items speeds visual search for a randomly located target regardless of trial-to-trial changes in local element size (Corbett & Melcher, 2014b). Here, we manipulate set size to investigate the potential role attention may play in these facilitative effects of statistical stability on visual search. Observers discriminated the left or right tilt of a Gabor target defined by a unique conjunction of orientation and spatial frequency in displays of Gabors with a stable or unstable mean size over successive trials. When set size was manipulated over sequences of successive trials, but held constant within a given sequence in Experiment 1, we observed distinct effects of statistical stability and attention, such that participants made faster correct responses as a function of stability and slower correct responses as a function of increasing set size. Replicating these main effects in Experiment 2, when set size was always unstable, provided converging evidence for discrete influences of statistical stability and attentional contributions to visual search. Overall, results support the proposal that our stable impressions of the surrounding environment and our abilities to attend salient events within that environment are distinctively governed by inherent statistical context and attentional processing demands.

"It's Not a Tumor": A Framework for Capitalizing on Individual Diversity to Boost Target Detection.

Even experts routinely miss infrequent targets, such as weapons in baggage scans or tumors in mammograms, because the visual system is not equipped to notice the unusual. To date, limited progress has been made toward improving human factors that mediate such critical diagnostic tasks. Here, we present a novel framework for pairing individuals' estimates to increase target detection. Using a wisdom-of-crowds approach that capitalizes on the visual system's ability to efficiently combine information, we demonstrated how averaging two noninteracting individuals' continuous estimates of whether a briefly presented image contained a prespecified target can significantly boost detection across a range of tasks. Furthermore, we showed how pairing individuals' estimates to maximize decorrelated patterns of performance in one task can optimize performance on a separate task. These results make significant advances toward combating severe deficits in target detection using straightforward applications for maximizing performance within limited pools of observers.

Set similarity modulates object tracking in dynamic environments.

Based on the observation that sports teams rely on colored jerseys to define group membership, we examined how grouping by similarity affected observers' abilities to track a "ball" target passed between 20 colored circle "players" divided into two color "teams" of 10 players each, or five color teams of four players each. Observers were more accurate and exerted less effort (indexed by pupil diameter) when their task was to count the number of times any player gained possession of the ball versus when they had to count only the possessions by a given color team, especially when counting the possessions of one team when players were grouped into fewer teams of more individual members each. Overall, results confirm previous reports of costs for segregating a larger set into smaller subsets and suggest that grouping by similarity facilitates processing at the set level.

The Whole Warps the Sum of Its Parts.

The efficiency of averaging properties of sets without encoding redundant details is analogous to gestalt proposals that perception is parsimoniously organized as a function of recurrent order in the world. This similarity suggests that grouping and averaging are part of a broader set of strategies allowing the visual system to circumvent capacity limitations. To examine how gestalt grouping affects the manner in which information is averaged and remembered, I compared the error in observers' adjustments of remembered sizes of individual circles in two different mean-size sets defined by similarity, proximity, connectedness, or a common region. Overall, errors were more similar within the same gestalt-defined groups than between different gestalt-defined groups, such that the remembered sizes of individual circles were biased toward the mean size of their respective gestalt-defined groups. These results imply that gestalt grouping facilitates perceptual averaging to minimize the error with which individual items are encoded, thereby optimizing the efficiency of visual short-term memory.

Perceptual Averaging in Individuals with Autism Spectrum Disorder.

There is mounting evidence that observers rely on statistical summaries of visual information to maintain stable and coherent perception. Sensitivity to the mean (or other prototypical value) of a visual feature (e.g., mean size) appears to be a pervasive process in human visual perception. Previous studies in individuals diagnosed with Autism Spectrum Disorder (ASD) have uncovered characteristic patterns of visual processing that suggest they may rely more on enhanced local representations of individual objects instead of computing such perceptual averages. To further explore the fundamental nature of abstract statistical representation in visual perception, we investigated perceptual averaging of mean size in a group of 12 high-functioning individuals diagnosed with ASD using simplified versions of two identification and adaptation tasks that elicited characteristic perceptual averaging effects in a control group of neurotypical participants. In Experiment 1, participants performed with above chance accuracy in recalling the mean size of a set of circles (mean task) despite poor accuracy in recalling individual circle sizes (member task). In Experiment 2, their judgments of single circle size were biased by mean size adaptation. Overall, these results suggest that individuals with ASD perceptually average information about sets of objects in the surrounding environment. Our results underscore the fundamental nature of perceptual averaging in vision, and further our understanding of how autistic individuals make sense of the external environment.

Statistical extraction affects visually guided action.

The visual system summarizes average properties of ensembles of similar objects. We demonstrated an adaptation aftereffect of one such property, mean size, suggesting it is encoded along a single visual dimension (Corbett, et al., 2012), in a similar manner as basic stimulus properties like orientation and direction of motion. To further explore the fundamental nature of ensemble encoding, here we mapped the evolution of mean size adaptation over the course of visually guided grasping. Participants adapted to two sets of dots with different mean sizes. After adaptation, two test dots replaced the adapting sets. Participants first reached to one of these dots, and then judged whether it was larger or smaller than the opposite dot. Grip apertures were inversely dependent on the average dot size of the preceding adapting patch during the early phase of movements, and this aftereffect dissipated as reaches neared the target. Interestingly, perceptual judgments still showed a marked aftereffect, even though they were made after grasping was completed more-or-less veridically. This effect of mean size adaptation on early visually guided kinematics provides novel evidence that mean size is encoded fundamentally in both perception and action domains, and suggests that ensemble statistics not only influence our perceptions of individual objects but can also affect our physical interactions with the external environment.

Stable statistical representations facilitate visual search.

Observers represent the average properties of object ensembles even when they cannot identify individual elements. To investigate the functional role of ensemble statistics, we examined how modulating statistical stability affects visual search. We varied the mean and/or individual sizes of an array of Gabor patches while observers searched for a tilted target. In "stable" blocks, the mean and/or local sizes of the Gabors were constant over successive displays, whereas in "unstable" baseline blocks they changed from trial to trial. Although there was no relationship between the context and the spatial location of the target, observers found targets faster (as indexed by faster correct responses and fewer saccades) as the global mean size became stable over several displays. Building statistical stability also facilitated scanning the scene, as measured by larger saccadic amplitudes, faster saccadic reaction times, and shorter fixation durations. These findings suggest a central role for peripheral visual information, creating context to free resources for detailed processing of salient targets and maintaining the illusion of visual stability.

Global statistical regularities modulate the speed of visual search in patients with focal attentional deficits.

There is growing evidence that the statistical properties of ensembles of similar objects are processed in a qualitatively different manner than the characteristics of individual items. It has recently been proposed that these types of perceptual statistical representations are part of a strategy to complement focused attention in order to circumvent the visual system's limited capacity to represent more than a few individual objects in detail. Previous studies have demonstrated that patients with attentional deficits are nonetheless sensitive to these sorts of statistical representations. Here, we examined how such global representations may function to aid patients in overcoming focal attentional limitations by manipulating the statistical regularity of a visual scene while patients performed a search task. Three patients previously diagnosed with visual neglect searched for a target Gabor tilted to the left or right of vertical in displays of horizontal distractor Gabors. Although the local sizes of the distractors changed on every trial, the mean size remained stable for several trials. Patients made faster correct responses to targets in neglected regions of the visual field when global statistics remained constant over several trials, similar to age-matched controls. Given neglect patients' attentional deficits, these results suggest that stable perceptual representations of global statistics can establish a context to speed search without the need to represent individual elements in detail.

Characterizing ensemble statistics: mean size is represented across multiple frames of reference.

The visual system represents the overall statistical, not individual, properties of sets. Here we tested the spatial nature of ensemble statistics. We used a mean-size adaptation paradigm (Corbett et al. in Visual Cognition, 20, 211-231, 2012) to examine whether average size is encoded in multiple reference frames. We adapted observers to patches of small- and large-sized dots in opposite regions of the display (left/right or top/bottom) and then tested their perceptions of the sizes of single test dots presented in regions that corresponded to retinotopic, spatiotopic, and hemispheric coordinates within the adapting displays. We observed retinotopic, spatiotopic, and hemispheric adaptation aftereffects, such that participants perceived a test dot as being larger when it was presented in the area adapted to the patch of small dots than when it was presented in the area adapted to large dots. This aftereffect also transferred between eyes. Our results demonstrate that mean size is represented across multiple spatial frames of reference, supporting the proposal that ensemble statistics play a fundamental role in maintaining perceptual stability.

An aftereffect of adaptation to mean size.

The visual system rapidly represents the mean size of sets of objects. Here, we investigated whether mean size is explicitly encoded by the visual system, along a single dimension like texture, numerosity, and other visual dimensions susceptible to adaptation. Observers adapted to two sets of dots with different mean sizes, presented simultaneously in opposite visual fields. After adaptation, two test patches replaced the adapting dot sets, and participants judged which test appeared to have the larger average dot diameter. They generally perceived the test that replaced the smaller mean size adapting set as being larger than the test that replaced the larger adapting set. This differential aftereffect held for single test dots (Experiment 2) and high-pass filtered displays (Experiment 3), and changed systematically as a function of the variance of the adapting dot sets (Experiment 4), providing additional support that mean size is adaptable, and therefore explicitly encoded dimension of visual scenes.

Visual performance fields: frames of reference.

Performance in most visual discrimination tasks is better along the horizontal than the vertical meridian (Horizontal-Vertical Anisotropy, HVA), and along the lower than the upper vertical meridian (Vertical Meridian Asymmetry, VMA), with intermediate performance at intercardinal locations. As these inhomogeneities are prevalent throughout visual tasks, it is important to understand the perceptual consequences of dissociating spatial reference frames. In all studies of performance fields so far, allocentric environmental references and egocentric observer reference frames were aligned. Here we quantified the effects of manipulating head-centric and retinotopic coordinates on the shape of visual performance fields. When observers viewed briefly presented radial arrays of Gabors and discriminated the tilt of a target relative to homogeneously oriented distractors, performance fields shifted with head tilt (Experiment 1), and fixation (Experiment 2). These results show that performance fields shift in-line with egocentric referents, corresponding to the retinal location of the stimulus.

The whole is indeed more than the sum of its parts: perceptual averaging in the absence of individual item representation.

We tested Ariely's (2001) proposal that the visual system represents the overall statistical properties of sets of objects against alternative accounts of rapid averaging involving sub-sampling strategies. In four experiments, observers could rapidly extract the mean size of a set of circles presented in an RSVP sequence, but could not reliably identify individual members. Experiment 1 contrasted performance on a member identification task with performance on a mean judgment task, and showed that the tasks could be dissociated based on whether the test probe was presented before or after the sequence, suggesting that member identification and mean judgment are subserved by different mechanisms. In Experiment 2, we confirmed that when given a choice between a probe corresponding to the mean size of the set and a foil corresponding to the mean of the smallest and largest items only, the former is preferred to the latter, even when observers are explicitly instructed to average only the smallest and largest items. Experiment 3 showed that a test item corresponding to the mean size of the set could be reliably discriminated from a foil but the largest item in the set, differing by an equivalent amount, could not. In Experiment 4, observers rejected test items dissimilar to the mean size of the set in a member identification task, favoring test items that corresponded to the mean of the set over items that were actually shown. These findings suggest that mean representation is accomplished without explicitly encoding individual items.

Facilitating stable representations: serial dependence in vision.

We tested whether the intervening time between multiple glances influences the independence of the resulting visual percepts. Observers estimated how many dots were present in brief displays that repeated one, two, three, four, or a random number of trials later. Estimates made farther apart in time were more independent, and thus carried more information about the stimulus when combined. In addition, estimates from different visual field locations were more independent than estimates from the same location. Our results reveal a retinotopic serial dependence in visual numerosity estimates, which may be a mechanism for maintaining the continuity of visual perception in a noisy environment.

Electrophysiological evidence for a post-perceptual influence of global visual context on perceived orientation.

Visual orientation cues largely determine our perceptions of "vertical," as illustrated in the Rod and Frame Illusion (RFI): an upright rod appears slanted in the opposite direction of a surrounding tilted frame. We used event-related potentials (ERPs) to examine the cortical time course of the frame's illusory influence on the perceived orientation of the rod. ERPs to an orientation discrimination task revealed: 1) the amplitudes of the P1 and N1 (indices of early visual processing) were not affected by frame tilt, but the amplitude of the P3 (an index of later processing) was, and 2) increasing the perceptual difficulty of the rod tilt judgment did not affect the P1 or N1, but increased response time and attenuated P3 amplitude. Results suggest that global orientation context effects are manifest during later, post-perceptual stages of information processing.

When do we know which way is up? The time course of orientation perception.

There is evidence that global visual context affects orientation perception in later stages of processing than local context. We measured the relative time courses of two orientation illusions: The Tilt Illusion (TI) involves dense gratings in close proximity; the Rod and Frame Illusion (RFI) involves a solitary bar surrounded by a distant frame. We also varied whether the context was flashed briefly (Experiment 1) or remained visible (Experiment 2). Results showed that the TI (but not the RFI) occurs when the context is briefly flashed in advance of the test, that both illusions are strongest when the context and inducer appear simultaneously, and that the RFI frame must be visible for at least 800 ms to induce an illusion with asynchronous displays. Experiment 3 confirmed these patterns held for measures of illusion magnitude and discriminability. Results are consistent with an earlier effect of local spatial context and a later effect of global spatial context on orientation perception.

Observer pitch and roll influence: the rod and frame illusion.

Observers misperceive the orientation of a vertical rod when it is viewed in the context of a tilted frame (the rod and frame illusion, or RFI). The pitch and roll of the surrounding surfaces have independent influences on this illusion (Nelson and Prinzmetal, 2003). Experiment 1 measured the RFI when the pitch and roll of the floor that supported the observer was varied, and the observer was either seated in a chair or standing upright. There were additive influences of pitch and roll on the RFI of seated but not standing observers. Experiments 2 and 3 decoupled body roll and head roll in order to isolate the vestibular and proprioceptive contributions to these effects. The results are interpreted in support of a hierarchy of influence on the RFI: Visual input is given top priority, followed by vestibular input, and then proprioceptive input.

The rapid extraction of numeric meaning.

Several kinds of statistical properties can be rapidly extracted from visual displays (e.g., luminance and roughness, Olive, A., & Torralba, A. (2001). Modeling the shape of the scene: a holistic representation of the spatial envelope. International Journal of Computational Vision, 42, 145-175). Here, we investigate whether this phenomenon extends to meaning contained in Arabic numerals. Observers were shown brief displays containing two sets of numerals and asked to determine which contained the largest average value. Comparisons were made more quickly and accurately between displays of digits than between displays of letters and shapes; this effect could not be attributed to task instructions. When numeric meaning could be used in a classification task, performance was better for digits than for letters, but when numeric meaning could not be used as a basis of classification, performance was approximately equal across stimulus types, suggesting that numeric meaning is rapidly extracted only when it is task-relevant. The digit advantage was eliminated with unlimited viewing time, suggesting that this process is used when counting is not possible. Dual-task methodology revealed that this process requires limited-capacity attentional resources.