Comparing explicit and implicit ensemble perception: 3 stimulus variables and 3 presentation modes.

Visual scenes are too complex for one to immediately perceive all their details. As suggested by Gestalt psychologists, grouping similar scene elements and perceiving their summary statistics provides one shortcut for evaluating scene gist. Perceiving ensemble statistics overcomes processing, attention, and memory limits, facilitating higher-order scene understanding. Ensemble perception spans simple/complex dimensions (circle size, face emotion), including various statistics (mean, range), and inherently spans space and/or time, when sets are presented scattered across the visual scene, and/or sequentially in rapid series. Furthermore, ensemble perception occurs explicitly, when observers are asked to judge set mean, and also automatically/implicitly, when observers are engaged in an orthogonal task. We now study relationships among these ensemble-perception phenomena, testing explicit and implicit ensemble perception; for sets varying in circle size, line orientation, or disc brightness; and with spatial, temporal or spatio-temporal presentation. Following ensemble set presentation, observers were asked if a test image, or which of two test images, had been present in the set. Confirming previous results, responses reflected implicit mean perception, depending on test image distance from the mean, and on its being within or outside ensemble range. Subsequent experiments asked the same observers to explicitly judge whether test images were larger, more clockwise, or brighter than the set mean, or which of two test images was closer to the mean. Comparing implicit and explicit mean perception, we find that explicit ensemble averaging is more precise than implicit mean perception-for each ensemble variable and presentation mode. Implications are discussed regarding possible separate mechanisms for explicit versus implicit ensemble perception.

Perceptual history biases in serial ensemble representation.

Ensemble perception refers to the visual system's ability to efficiently represent groups of similar objects as a unified percept using their summary statistical information. Most studies focused on extraction of current trial averages, giving little attention to prior experience effects, although a few recent studies found that ensemble mean estimations contract toward previously presented stimuli, with most of these focusing on explicit perceptual averaging of simultaneously presented item ensembles. Yet, the time element is crucial in real dynamic environments, where we encounter ensemble items over time, aggregating information until reaching summary representations. Moreover, statistical information of objects and scenes is learned over time and often implicitly and then used for predictions that shape perception, promoting environmental stability. Therefore, we now focus on temporal aspects of ensemble statistics and test whether prior information, beyond the current trial, biases implicit perceptual decisions. We designed methods to separate current trial biases from those of previously seen trial ensembles. In each trial, six circles of different sizes were presented serially, followed by two test items. Participants were asked to choose which was present in the sequence. Participants unconsciously rely on ensemble statistics, choosing stimuli closer to the ensemble mean. To isolate the influence of earlier trials, the two test items were sometimes equidistant from the current trial mean. Results showed membership judgment biases toward current trial mean, when informative (largest effect). On equidistant trials, judgments were biased toward previously experienced stimulus statistics. Comparison of similar conditions with a shifted stimulus distribution ruled out a bias toward an earlier, presession, prototypical diameter. We conclude that ensemble perception, even for temporally experienced ensembles, is influenced not only by current trial mean but also by means of recently seen ensembles and that these influences are somewhat correlated on a participant-by-participant basis.

High-level visual search in children with autism.

Visual search has been classified as easy feature search, with rapid target detection and little set size dependence, versus slower difficult search with focused attention, with set size-dependent speed. Reverse hierarchy theory attributes these classes to rapid high cortical-level vision at a glance versus low-level vision with scrutiny, attributing easy search to high-level representations. Accordingly, faces "pop out" of heterogeneous object photographs. Individuals with autism have difficulties recognizing faces, and we now asked if this disability disturbs their search for faces. We compare search times and set size slopes for children with autism spectrum disorders (ASDs) and those with neurotypical development (NT) when searching for faces. Human face targets were found rapidly, with shallow set size slopes. The between-group difference between slopes (18.8 vs. 11.3 ms/item) is significant, suggesting that faces may not "pop out" as easily, but in our view does not warrant classifying ASD face search as categorically different from that of NT children. We also tested search for different target categories, dog and lion faces, and nonface basic categories, cars and houses. The ASD group was generally a bit slower than the NT group, and their slopes were somewhat steeper. Nevertheless, the overall dependencies on target category were similar: human face search fastest, nonface categories slowest, and dog and lion faces in between. We conclude that autism may spare vision at a glance, including face detection, despite its reported effects on face recognition, which may require vision with scrutiny. This dichotomy is consistent with the two perceptual modes suggested by reverse hierarchy theory.

Perceiving ensemble statistics of novel image sets.

Perception, representation, and memory of ensemble statistics has attracted growing interest. Studies found that, at different abstraction levels, the brain represents similar items as unified percepts. We found that global ensemble perception is automatic and unconscious, affecting later perceptual judgments regarding individual member items. Implicit effects of set mean and range for low-level feature ensembles (size, orientation, brightness) were replicated for high-level category objects. This similarity suggests that analogous mechanisms underlie these extreme levels of abstraction. Here, we bridge the span between visual features and semantic object categories using the identical implicit perception experimental paradigm for intermediate novel visual-shape categories, constructing ensemble exemplars by introducing systematic variations of a central category base or ancestor. In five experiments, with different item variability, we test automatic representation of ensemble category characteristics and its effect on a subsequent memory task. Results show that observer representation of ensembles includes the group's central shape, category ancestor (progenitor), or group mean. Observers also easily reject memory of shapes belonging to different categories, i.e. originating from different ancestors. We conclude that complex categories, like simple visual form ensembles, are represented in terms of statistics including a central object, as well as category boundaries. We refer to the model proposed by Benna and Fusi (bioRxiv 624239, 2019) that memory representation is compressed when related elements are represented by identifying their ancestor and each one's difference from it. We suggest that ensemble mean perception, like category prototype extraction, might reflect employment at different representation levels of an essential, general representation mechanism.

Perceptual learning of ensemble and outlier perception.

Previous studies have demonstrated a complex relationship between ensemble perception and outlier detection. We presented two array of heterogeneously oriented stimulus bars and different mean orientations and/or a bar with an outlier orientation, asking participants to discriminate the mean orientations or detect the outlier. Perceptual learning was found in every case, with improved performance accuracy and speeded responses. Testing for improved accuracy through cross-task transfer, we found considerable transfer from training outlier detection to mean discrimination performance, and none in the opposite direction. Implicit learning in terms of increased accuracy was not found in either direction when participants performed one task, and the second task's stimulus features were present. Reaction time improvement was found to transfer in all cases. This study adds to the already broad knowledge concerning perceptual learning and cross-task transfer of training effects.

The gist of Anne Treisman's revolution.

Anne Treisman investigated many aspects of perception, and in particular the roles of different forms of attention. Four aspects of her work are reviewed here, including visual search, set mean perception, perception in special populations, and binocular rivalry. The importance of the breakthrough in each case is demonstrated. Search is easy or slow depending on whether it depends on the application of global or focused attention. Mean perception depends on global attention and affords simultaneous representation of the means of at least two sets of elements, and then of comparing them. Deficits exhibited in Balint's or unilateral neglect patients identify basic sensory system mechanisms. And, the ability to integrate binocular information for stereopsis despite simultaneous binocular rivalry for color, demonstrates the division of labor underlying visual system computations. All these studies are related to an appreciation of the difference between perceiving the gist of a scene, its elements or objects, versus perceiving the details of the scene and its components. This relationship between Anne Treisman's revolutionary discoveries and the concept of gist perception is the core of the current review.

Relating categorization to set summary statistics perception.

Two cognitive processes have been explored that compensate for the limited information that can be perceived and remembered at any given moment. The first parsimonious cognitive process is object categorization. We naturally relate objects to their category, assume they share relevant category properties, often disregarding irrelevant characteristics. Another scene organizing mechanism is representing aspects of the visual world in terms of summary statistics. Spreading attention over a group of objects with some similarity, one perceives an ensemble representation of the group. Without encoding detailed information of individuals, observers process summary data concerning the group, including set mean for various features (from circle size to face expression). Just as categorization may include/depend on prototype and intercategory boundaries, so set perception includes property mean and range. We now explore common features of these processes. We previously investigated summary perception of low-level features with a rapid serial visual presentation (RSVP) paradigm and found that participants perceive both the mean and range extremes of stimulus sets, automatically, implicitly, and on-the-fly, for each RSVP sequence, independently. We now use the same experimental paradigm to test category representation of high-level objects. We find participants perceive categorical characteristics better than they code individual elements. We relate category prototype to set mean and same/different category to in/out-of-range elements, defining a direct parallel between low-level set perception and high-level categorization. The implicit effects of mean or prototype and set or category boundaries are very similar. We suggest that object categorization may share perceptual-computational mechanisms with set summary statistics perception.

Comparing set summary statistics and outlier pop out in vision.

Visual scenes are too complex to perceive immediately in all their details. Two strategies (among others) have been suggested as providing shortcuts for evaluating scene gist before its details: (a) Scene summary statistics provide average values that often suffice for judging sets of objects and acting in their environment. Set summary perception spans simple/complex dimensions (circle size, face emotion), various statistics (mean, variance, range), and separate statistics for discernible sets. (b) Related to set summary perception is detecting outliers from sets, called "pop out," which allows rapid perception of presence and properties of unusual, and thus, possibly salient items in the scene. To understand better visual system mechanisms underlying these two set-related perceptual phenomena, we now study their properties and the relationship between them. We present observers with two clouds of bars with distributed orientations and ask them to discriminate their mean orientations, reporting which cloud is oriented more clockwise, on average. In the second experiment, the two clouds had the same mean orientation, but one had a bar with an outlier orientation, which observers detected. We find that cloud mean orientation discrimination depends mainly on the difference in means, whereas outlier detection depends mainly on the distance of the outlier orientation from the edge of the cloud orientation range. Neither percept depends largely on the range of the set itself. A unified model of a population-code mechanism underlying these two phenomena is discussed.

Perceiving set mean and range: Automaticity and precision.

To compensate for the limited visual information that can be perceived and remembered at any given moment, many aspects of the visual world are represented as summary statistics. We acquire ensemble representations of element groups as a whole, spreading attention over objects, for which we encode no detailed information. Previous studies found that different features of items (from size/orientation to facial expression/biological motion) are summarized to their mean, over space or time. Summarizing is economical, saving time and energy when the environment is too rich and complex to encode each stimulus separately. We investigated set perception using rapid serial visual presentation sequences. Following each sequence, participants viewed two stimuli, member and nonmember, indicating the member. Sometimes, unbeknownst to participants, one stimulus was the set mean, and or the nonmember was outside the set range. Participants preferentially chose stimuli at/near the mean, a "mean effect," and more easily rejected out-of-range stimuli, a "range effect." Performance improved with member proximity to the mean and nonmember distance from set mean and edge, though they were instructed only to remember presented stimuli. We conclude that participants automatically encode both mean and range boundaries of stimulus sets, avoiding capacity limits and speeding perceptual decisions.

The Eyes Wide Shut Illusion.

The new "eyes wide shut" illusion uses a standard enlarging (shaving or makeup) mirror. Close one eye and look at the closed eye in the mirror; the eye should take up most of the mirror. Switch eyes to see the other closed eye. Switch back-and-forth a few times, then open both eyes. You see an open eye. Which eye is it? To find out, close one eye. Whichever you close, that's the eye you see. How can this be possible? The brain is fusing two images of the two eyes! The illusion depends on (a) binocular fusion: The brain combines two images to a single percept; (b) symmetry: Mirrors don't affect appearance of left-right symmetric objects and the eyes are sufficiently left-right symmetric for the brain to combine them. Why aren't the lingering asymmetries sufficient to prevent fusion? (c) Only vision with scrutiny affords conscious access to scene details. Consistent with reverse hierarchy theory, vision at a glance grants conscious perception of the gist of the scene, integrating images of nonperfectly symmetric eyes.

Global statistics are not neglected.

In the framework of Reverse Hierarchy Theory it was suggested that initial vision at a glance brings the gist of the scene to conscious perception using explicit high cortical level representations, which are initially built by implicit bottom-up processing (Hochstein & Ahissar, 2002). Only later return to lower cortical level representations introduces local details to conscious perception. Global statistics of similar elements are perceived rapidly and accurately, suggesting they are included in the initial perception of the gist of the scene, not depending on prior conscious perception of local details. Patients with unilateral spatial neglect have difficulty responding to elements in their contralesional hemifield. However, this deficit is especially pronounced for tasks that require focused attention, i.e., are dependent on the reverse-hierarchy return. We review recent studies that indicate that perception of global statistics is among the spread attention tasks that are somewhat spared from this deficit. Combining these results, we suggest that perhaps the function of global statistics perception might include serving as a basic percept required for finding salient deviants from the mean, as in rapid odd element feature search paradigms, and perhaps subsequently focusing attention to them.

Computing an average when part of the population is not perceived.

The syndrome of unilateral spatial neglect (USN) after right-hemisphere damage is characterized by failure of salient left-sided stimuli to activate an orienting response, attract attention, and gain access to conscious awareness. The explicit failure processing left-sided visual information is not uniform, however, and patients seem to be more successful performing certain visual tasks than others. The source of this difference is still not clear. We focus on processing of visual scene statistical properties, asking whether, in computing the average size of an array of objects, USN patients give appropriate weight to objects on the left; disregard left-side objects entirely; or assign them an intermediate, lower weight, in accord with their tendency to neglect these objects. The interest in testing this question stems from a series of studies in healthy individuals that led Chong and Treisman [Chong, S. C., & Treisman, A. Statistical processing: Computing the average size in perceptual groups. Vision Research, 45, 891-900, 2005a; Chong, S. C., & Treisman, A. Attentional spread in the statistical processing of visual displays. Perception & Psychophysics, 67, 1-13, 2005b] to propose that processing of statistical properties (like the average size of visual scene elements) is carried out in parallel, with no need for serial allocation of focal attention to the different scene elements. Our results corroborate this suggestion, showing that objects in the left ("neglected") hemispace contribute to average size computation, despite a marked imbalance in spatial distribution of attention, which leads to a reduced weight of left-side elements in the averaging computation. This finding sheds light on the nature of the impairment in USN and on basic mechanisms underlying statistical processing in vision. We confirm that statistical processing depends mainly on spread-attention mechanisms, which are largely spared in USN.

It's hard to forget: resetting memory in delay-match-to-multiple-image tasks.

The Delay-Match-to-Sample (DMS) task has been used in countless studies of memory, undergoing numerous modifications, making the task more and more challenging to participants. The physiological correlate of memory is modified neural activity during the cue-to-match delay period reflecting reverberating attractor activity in multiple interconnected cells. DMS tasks may use a fixed set of well-practiced stimulus images-allowing for creation of attractors-or unlimited novel images, for which no attractor exists. Using well-learned stimuli requires that participants determine if a remembered image was seen in the same or a preceding trial, only responding to the former. Thus, trial-to-trial transitions must include a "reset" mechanism to mark old images as such. We test two groups of monkeys on a delay-match-to-multiple-images task, one with well-trained and one with novel images. Only the first developed a reset mechanism. We then switched tasks between the groups. We find that introducing fixed images initiates development of reset, and once established, switching to novel images does not disable its use. Without reset, memory decays slowly, leaving ~40% recognizable after a minute. Here, presence of reward further enhances memory of previously-seen images.

Modeling behavior in different delay match to sample tasks in one simple network.

Delay match to sample (DMS) experiments provide an important link between the theory of recurrent network models and behavior and neural recordings. We define a simple recurrent network of binary neurons with stochastic neural dynamics and Hebbian synaptic learning. Most DMS experiments involve heavily learned images, and in this setting we propose a readout mechanism for match occurrence based on a smaller increment in overall network activity when the matched pattern is already in working memory, and a reset mechanism to clear memory from stimuli of previous trials using random network activity. Simulations show that this model accounts for a wide range of variations on the original DMS tasks, including ABBA tasks with distractors, and more general repetition detection tasks with both learned and novel images. The differences in network settings required for different tasks derive from easily defined changes in the levels of noise and inhibition. The same models can also explain experiments involving repetition detection with novel images, although in this case the readout mechanism for match is based on higher overall network activity. The models give rise to interesting predictions that may be tested in neural recordings.

High-level binocular rivalry effects.

Binocular rivalry (BR) occurs when the brain cannot fuse percepts from the two eyes because they are different. We review results relating to an ongoing controversy regarding the cortical site of the BR mechanism. Some BR qualities suggest it is low-level: (1) BR, as its name implies, is usually between eyes and only low-levels have access to utrocular information. (2) All input to one eye is suppressed: blurring doesn't stimulate accommodation; pupilary constrictions are reduced; probe detection is reduced. (3) Rivalry is affected by low-level attributes, contrast, spatial frequency, brightness, motion. (4) There is limited priming due to suppressed words or pictures. On the other hand, recent studies favor a high-level mechanism: (1) Rivalry occurs between patterns, not eyes, as in patchwork rivalry or a swapping paradigm. (2) Attention affects alternations. (3) Context affects dominance. There is conflicting evidence from physiological studies (single cell and fMRI) regarding cortical level(s) of conscious perception. We discuss the possibility of multiple BR sites and theoretical considerations that rule out this solution. We present new data regarding the locus of the BR switch by manipulating stimulus semantic content or high-level characteristics. Since these variations are represented at higher cortical levels, their affecting rivalry supports high-level BR intervention. In Experiment I, we measure rivalry when one eye views words and the other non-words and find significantly longer dominance durations for non-words. In Experiment II, we find longer dominance times for line drawings of simple, structurally impossible figures than for similar, possible objects. In Experiment III, we test the influence of idiomatic context on rivalry between words. Results show that generally words within their idiomatic context have longer mean dominance durations. We conclude that BR has high-level cortical influences, and may be controlled by a high-level mechanism.

Hearing while blinking: multisensory attentional blink revisited.

It is well established that cognitive system overload is reflected in the attentional blink (AB), the failure to report a second target when it closely follows detection of a first target within a rapid series of stimuli. However, there is intense controversy concerning the effect of first-target detection in one modality on subsequent dynamics of attentional resources in other modalities. Mixed results were found using an audiovisual AB paradigm: depletion of resources in one modality either impaired performance in the other modality or had no effect. Here, we circumvent the need for task switching by measuring an event-related potential, the mismatch negativity, which reflects implicit auditory change detection without requiring task engagement and is present even for background sounds that participants ignore. Surprisingly, we find that during the visual AB, auditory processing is enhanced rather than inhibited, as would be expected by system overload. We suggest that multimodal attentional resources may be freed rather than engaged during the visual AB. Suppression of irrelevant input may require active control by a central executive, which is preoccupied during the visual AB, and/or there may be no reason to suppress other-modal input since the visual system will miss its second target anyway.

Perceptual learning transfer between hemispheres and tasks for easy and hard feature search conditions.

Perceptual learning involves modification of cortical processes so that transfer to new task variants depends on neuronal representation overlap. Neuron selectivity varies with cortical level, so that the degree of transfer should depend on training-induced modification level. We ask how different can stimuli be, how far apart can their representations be, and still induce training transfer. We measure transfer across both long distances within the visual field, namely across cerebral hemispheres, as well as across perceptual dimensions, i.e., between detection of odd color and orientation. In Experiment 1, subjects learned feature search using eccentric arrays randomly presented in the right or left hemifield. Odd elements differed in color or orientation, depending on the presentation hemifield. Following training and performance improvement, the dimensions were switched between hemifields. There was little cross-hemifield or cross-task transfer for difficult cases, and the greater transfer found for easier cases could be across hemispheres and/or perceptual dimensions. Testing these two elements separately, Experiment 2 confirmed considerable transfer across task dimensions, training one dimension and testing another, and Experiment 3 confirmed such transfer across hemifields, training search on one side and testing on the other. Results support Reverse Hierarchy Theory (M. Ahissar & S. Hochstein, 1997, 2004) in that, for easier perceptual tasks involving and modifying higher cortical levels, considerable transfer occurs both across perceptual dimensions and across visual field location even across hemifields.

Measuring pictorial balance perception at first glance using Japanese calligraphy.

According to art theory, pictorial balance acts to unify picture elements into a cohesive composition. For asymmetrical compositions, balancing elements is thought to be similar to balancing mechanical weights in a framework of symmetry axes. Assessment of preference for balance (APB), based on the symmetry-axes framework suggested in Arnheim R, 1974 Art and Visual Perception: A Psychology of the Creative Eye (Berkeley, CA: University of California Press), successfully matched subject balance ratings of images of geometrical shapes over unlimited viewing time. We now examine pictorial balance perception of Japanese calligraphy during first fixation, isolated from later cognitive processes, comparing APB measures with results from balance-rating and comparison tasks. Results show high between-task correlation, but low correlation with APB. We repeated the rating task, expanding the image set to include five rotations of each image, comparing balance perception of artist and novice participant groups. Rotation has no effect on APB balance computation but dramatically affects balance rating, especially for art experts. We analyze the variety of rotation effects and suggest that, rather than depending on element size and position relative to symmetry axes, first fixation balance processing derives from global processes such as grouping of lines and shapes, object recognition, preference for horizontal and vertical elements, closure, and completion, enhanced by vertical symmetry.

The wide window of face detection.

Faces are detected more rapidly than other objects in visual scenes and search arrays, but the cause for this face advantage has been contested. In the present study, we found that under conditions of spatial uncertainty, faces were easier to detect than control targets (dog faces, clocks and cars) even in the absence of surrounding stimuli, making an explanation based only on low-level differences unlikely. This advantage improved with eccentricity in the visual field, enabling face detection in wider visual windows, and pointing to selective sparing of face detection at greater eccentricities. This face advantage might be due to perceptual factors favoring face detection. In addition, the relative face advantage is greater under flanked than non-flanked conditions, suggesting an additional, possibly attention-related benefit enabling face detection in groups of distracters.