Ensemble memory of a scene interacts with current perception regardless of attentional requirements.

How do we maintain a rich and stable perceptual experience across the entire visual scene, even when we are focusing on a subset of visual inputs? The current study explored this question by investigating whether the visual system processes summary statistics of multiple features regardless of task relevance, and how they interact with subsequent perception. To test the processing of multifeature summary statistics under different attentional requirements, we presented multiple Gabor patches with heterogeneous orientations/colors and asked participants to attend to a single feature dimension (Experiments 1 and 3) or a single item (Experiment 2) for the memory task. During the memory maintenance period (before memory response), we asked the participants to perform a discrimination task (Experiments 1 and 2) or a boundary localization task (Experiment 3) to test how the memory of the ensemble representation alters the subsequent perceptual experience. We found evidence for obligatory processing of scene summary statistics presented for the memory task, which interacted with the subsequent perceptual sensitivity. Specifically, not only summary statistics relevant but also those of task-irrelevant feature (Experiments 1 and 3) and outside the focus of attention (Experiment 2) were encoded and bidirectionally interacted with subsequent perception. These results suggest obligatory processing of summary statistics of a scene, which may allow rich and stable visual experience by integrating temporally adjacent visual inputs. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Outlier rejection in the process of pooling.

Ensemble perception allows our visual system to process large amounts of information efficiently by summarizing its statistical properties. A key aspect of ensemble perception is the devaluation of outlying elements, which leads to more informative summary statistics with reduced variance and a more representative mean. However, the mechanisms underlying this outlier rejection process are not well understood. One possibility is that outliers are selectively excluded before summarization. To test this, we investigated whether only weaker items were excluded from averaging. We manipulated the encoding strength of items in a display by changing the emotional intensities of faces, the spatial location of emotional outliers, and the spatial distribution of emotional faces. We found that the response to outliers varied depending on their location. Specifically, outliers were more likely to be excluded from averaging when presented in more peripheral regions, while their exclusion was partial in parafoveal regions. In other words, outlier rejection in ensemble processing is more flexible than the supposed rigid designation of weighting against outliers. Alternatively, the results fit well with hierarchically structured pooling, during which outliers are discounted more dynamically without positing any separate selective mechanism before summarization. We propose an explanation for outlier rejection in light of a recently proposed population response model of ensemble processing.

A population response model of ensemble perception.

Ensemble representations have been considered as one of the strategies that the visual system adopts to cope with its limited capacity. Thus, they include various statistical summaries such as mean, variance, and distributional properties and are formed over many stages of visual processing. The present study proposes a population-coding model of ensemble perception to provide a theoretical and computational framework for these various facets of ensemble perception. The proposed model consists of a simple feature layer and a pooling layer. We assumed ensemble representations as population responses in the pooling layer and decoded various statistical properties from population responses. Our model successfully predicted averaging performance in orientation, size, color, and motion direction across different tasks. Furthermore, it predicted variance discrimination performance and the priming effects of feature distributions. Finally, it explained the well-known variance and set-size effects and has a potential for explaining the adaptation and clustering effects. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Partial blindness: Visual experience is not rich, but not sparse.

Is our perceptual experience constrained by an information bottleneck (i.e., the limited capacity for cognitive access), and if so, how? To answer these questions, we investigated observers' perceptual resolution for an individual face when they saw either a single face or multiple faces simultaneously. We characterized perceptual resolution by measuring how accurately observers detected the degradation of a face. In two experiments, we found that the resolution of perception decreased when seeing multiple faces compared to when seeing a single face, which mainly resulted from the neglect of degradation. Importantly, this degradation neglect was based on partial blindness rather than complete blindness to a face, indicating that perceptual experience is limited by an information bottleneck as a form of partial blindness to an individual item. Together, our findings suggest that perceptual experience at a glance might be sparser than retinal resolution or perceptual resolution measured under conditions where the target stimulus is the sole focus of attention; however, it might be richer than a small handful of items.

Americans weigh an attended emotion more than Koreans in overall mood judgments.

Face ensemble coding is the perceptual ability to create a quick and overall impression of a group of faces, triggering social and behavioral motivations towards other people (approaching friendly people or avoiding an angry mob). Cultural differences in this ability have been reported, such that Easterners are better at face ensemble coding than Westerners are. The underlying mechanism has been attributed to differences in processing styles, with Easterners allocating attention globally, and Westerners focusing on local parts. However, the remaining question is how such default attention mode is influenced by salient information during ensemble perception. We created visual displays that resembled a real-world social setting in which one individual in a crowd of different faces drew the viewer's attention while the viewer judged the overall emotion of the crowd. In each trial, one face in the crowd was highlighted by a salient cue, capturing spatial attention before the participants viewed the entire group. American participants' judgment of group emotion more strongly weighed the attended individual face than Korean participants, suggesting a greater influence of local information on global perception. Our results showed that different attentional modes between cultural groups modulate social-emotional processing underlying people's perceptions and attributions.

The effect of masks on the emotion perception of a facial crowd.

The present study investigated the effect of facial masks on people's ability to perceive emotions in crowds. We presented faces with the bottom halves occluded by masks or full faces without occlusion. In two sequentially presented crowds, we varied the number of faces, emotional valence, and intensity of facial expressions, examining the impact of masks on the perception of crowd emotion. Participants reported which of the two crowds they would avoid based on the crowds' average emotions. The participants' ability to judge the average emotion of a crowd, especially a crowd expressing happiness, was impaired when the crowd wore masks. For faces covered by masks, crowd emotion judgments were more negatively biased than those without masks. However, participants could still distinguish the emotional intensities of a crowd wearing masks above chance. Additionally, participants responded more quickly to a crowd with more people without compromising accuracy, despite the perceptual challenges imposed by facial masks. Our results suggest that under ambiguous social situations in which individuals' emotions are partially hidden by masks, a large group may provide stronger social cues than a small group, thereby promoting communication and regulating social behaviors.

Similarity-based clustering of multifeature objects in visual working memory.

This study investigated the similarity-based clustering mechanism of multifeature stimuli, wherein items are separated or grouped based on their similarity in visual working memory (VWM). In particular, we investigated whether clustering occurred at an individual feature level or at an integrated object level when participants encoded objects with multiple features for VWM. To test this, we conducted two experiments in which participants remembered and reconstructed a randomly chosen feature (either color or orientation) from one of five presented stimuli. As a key manipulation, we kept the distributions of the two feature dimensions constant while controlling the conjunction between the two dimensions in two different conditions: congruent conjunction (CC) and incongruent conjunction (IC). With this manipulation, we expected to observe the same number of clusters regardless of the conjunction condition when clustering occurred at the feature level. However, we expected a different number of clusters for CC and IC conditions when clustering occurred at the object level. Across two experiments, we consistently observed evidence that favored feature-level clustering. Nevertheless, we found that the swap error rates increased in the IC condition only when two features had to be encoded in VWM. These results suggest that clustering occurs at the feature level in VWM and that feature-level clustering influences item-level feature binding. Therefore, our study demonstrated the flexibility of representational units in VWM.

The relationship between ensemble representations of facial information.

People accurately evaluate various types of facial information (gaze direction, facial expression, facial identity, and gender) of multiple faces. Considering such varieties, summarizing abilities of facial information might vary depending on its type because it is either changeable (e.g., gaze direction and expression) or invariant (e.g., identity and gender). The current study investigated the relationship between the averaging abilities of facial information using an individual difference approach and a dual-task paradigm to understand the effect of facial information type on the ensemble coding of facial information. We conducted two online experiments on the relationship between the averaging abilities of facial expressions and gaze direction (Experiment 1) and those of facial expressions and gender (Experiment 2). Participants were asked to estimate the average of each piece of facial information in the first and second blocks (single task), respectively, and both sequentially in the third and fourth blocks (dual task). We found that most of the error autocorrelations of facial information were high, indicating high measurement reliability. Participants' abilities to average facial expressions were correlated with those to average gaze directions, but not with those to average gender information. That is, the ensemble processing of facial expressions is related to gaze directions, but not genders. These results suggest that ensemble representations of facial information regarding changeable properties differ from those of invariant ones.

Effects of attention on visual discrimination and awareness: Evidence from eye-specific attention.

The role of attention in visual awareness is a foundational issue for understanding consciousness, but remains highly controversial. Therefore, in two experiments, we investigated whether and how attention modulates visual awareness using a monocular cuing paradigm. Although observers are not aware of which eye received a cue, a monocular cue can attract eye-specific attention to a cued eye. In Experiment 1, we found that eye-specific attention enhanced subjective visual awareness (i.e., awareness reports) as well as objective task performance (i.e., orientation discrimination) of a Gabor target. Importantly, attention enhanced visual awareness only when a decision for orientation discrimination was correct, suggesting that the effect of attention on visual awareness is closely associated with perceptual evidence for orientation discrimination. In Experiment 2, we showed that the modulatory effect of attention on visual awareness was based on contrast gain, which amplified the effective contrast of an attended stimulus, and did not differ from that on orientation discrimination performance qualitatively and quantitatively. Our findings suggest that attention influences visual awareness by amplifying perceptual evidence on which visual awareness and objective performance are commonly based. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Neural representations of ensemble coding in the occipital and parietal cortices.

The human visual system is able to extract summary statistics from sets of similar items, but the underlying neural mechanism remains poorly understood. Using functional magnetic resonance imaging (fMRI) and an encoding model, we examined how the neural representation of ensemble coding is constructed by manipulating the task-relevance of ensemble features. We found a gradual increase in orientation-selective responses to the mean orientation of multiple stimuli along the visual hierarchy only when these orientations were task-relevant. Such responses to the ensemble orientation were present in the extrastriate area, V3, even when the mean orientation was not task-relevant, indicating that the ensemble representation can co-exist with the task-relevant individual feature representation. Ensemble orientations were also represented in frontal regions, but those representations were robust only when each mean orientation was linked to a motor response dimension. Together, our findings suggest that the neural representation of the ensemble percept is formed by pooling signals at multiple levels of the visual processing stream.

Multivariate summary of a complex scene.

The current study investigated how people summarize and represent objects with multiple features to cope with the complexity due to the number of objects and feature dimensions. We presented a set of circles whose color and size were either correlated perfectly (r = 1) or not correlated at all (r = 0). Using a membership identification task, we found that participants formed a statistical representation that included information about conjunctions as well as each color and size dimensions. In addition, we found that participants represented different set boundaries depending on the correlation between features of a set. Lastly, a pair-matching task revealed that participants predicted one feature value from the other feature value based on the correlation between features of a set. Our findings suggest that people represent a multi-feature ensemble statistically as a multivariate feature distribution, which is an efficient strategy to cope with scene complexity.

Perceived variability reflects the reliability of individual items.

When confronted with many visual items, people can compute their variability accurately and rapidly, which facilitates efficient information processing and optimal decision making. However, how the visual system computes variability is still unclear. To investigate this, we implemented situations whereby estimates of variability based on several possible variability measures (e.g., range, standard deviation, and weighted standard deviation) could be differentiated, and then examined which best accounted for human variability perception. In three psychophysical experiments, participants watched two arrays of items with various orientations and judged which had more variable orientations. Results showed that perceived variability was most consistent with the weighted standard deviation based on the reliability of individual items. Specifically, participants gave less consideration to deviant orientations that were likely to be outliers, and greater consideration to salient orientations that were likely to be encoded precisely. This reliability-based weighted standard deviation suggests an efficient and flexible way of representing visual variability.

Partial awareness can be induced by independent cognitive access to different spatial frequencies.

Partial awareness-an intermediate state between complete consciousness and unconsciousness-has been explained by independent cognitive access to different levels of representation in hierarchical visual processing. This account, however, cannot explain graded visual experiences in low levels. We aimed to explain partial awareness in low levels of visual processing by independent cognitive access to different spatial frequencies. To observe partial awareness stably, we used a novel method. Stimuli were presented briefly (12 ms) and repeatedly with a specific inter-stimulus interval, ranging from 0 to 235 ms. By using various stimuli containing high and low spatial frequencies (superimposed sinusoidal gratings, Navon letters, and scenes), we found that conscious percept was degraded with increasing inter-stimulus intervals. However, the degree of degradation was smaller for low spatial frequency than for high spatial frequency information. Our results reveal that cognitive access to different spatial frequencies can occur independently and this can explain partial awareness in low levels of visual processing.

Quality of average representation can be enhanced by refined individual items.

Ensemble perception is efficient because it summarizes redundant and complex information. However, it loses the fine details of individual items during the averaging process. Such characteristics of ensemble perception are similar to those of coarse processing. Here, we tested whether extracting an average of a set was similar to coarse processing. To manipulate coarse processing, we used the fast flicker adaptation known as suppressing coarse information processed by the magnocellular pathway. We hypothesized that if computing the average of a set relied on coarse processing, the precision of an averaging task should decrease after adaptation compared to baseline (no-adaptation). Across experiments with various features (orientation in Experiment 1, size in Experiment 2, and facial expression in Experiment 3), we found that suppressing coarse information did not disrupt the performance of the averaging tasks. Rather, adaptation increased the precision of mean representation. The precision of mean representation might have increased because fine information was relatively enhanced after adaptation. Our results suggest that the quality of ensemble representation relies on that of individual items.

Effects of Selective Attention on Mean-Size Computation: Weighted Averaging and Perceptual Enlargement.

For efficient use of limited capacity, the visual system summarizes redundant information and prioritizes relevant information, strategies known respectively as ensemble perception and selective attention. Although previous studies showed a close relationship between these strategies, the specific mechanisms underlying the relationship have not been determined. We investigated how attention modulated mean-size computation. Fourteen people participated in this study. We hypothesized that attention biases mean-size computation by increasing the contribution (weighted averaging) and the apparent size (perceptual enlargement) of an attended item. Consistent with this hypothesis, our results showed that estimated mean sizes were biased toward the attended size and overestimated regardless of the attended size, supporting weighted averaging and perceptual enlargement, respectively. Taken together, the observed effects of selective attention on mean-size computation signify a close relationship between the two optimization mechanisms to achieve efficient management of the visual system's limited capacity.

Contextual cueing in target absent trials by distractor-distractor associations.

Contextual cueing refers to finding a target more efficiently in repeated displays than in novel displays. However, conflicting results have been reported regarding whether target absent judgments can also be efficient in repeated displays. To resolve this controversy, we first tested 3 factors that might influence the strength of distractor-distractor associations and then investigated how such associations produced faster responses on repeated target absent trials by measuring the patterns of eye movements. The factors were the number of distractors, number of repeated configurations, and diversity of the distractors' properties. In Experiments 1 and 2, we found that the diversity of the distractors was the only factor producing contextual cueing without a target. In Experiment 3, we recorded eye movements during a search task and found that the contextual cueing effect in the target absent condition was due to the lower number of fixations and larger mean saccadic amplitudes. Overall, these results suggest that the distractor-distractor associations, strengthened by the diversity of the distractors' properties, widened the attentional window. This enlarged window in turn helps people to reject more distractors at once and enables them to terminate a visual search faster in repeated target absent trials. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

The visual system does not compute a single mean but summarizes a distribution.

Ongoing discussions on perceptual averaging have the implicit assumption that individual representations are reduced into a single prototypical representation. However, some evidence suggests that the mean representation may be more complex. For example, studies that use a single item probe to estimate mean size often show biased estimations. To this end, we investigate whether the mean representation of size is reduced to a single mean or includes other properties of the set. Participants estimate the mean size of multiple circles in the display set by adjusting the mean size of the circles in the probe set that followed. Across 3 experiments, we vary the similarity of set-size, variance, and skewness between the display and probe sets and examine how property congruence affects mean estimation. Altogether, we find that keeping properties consistent between the 2 compared sets improves mean estimation accuracy. These results suggest that mean representation is not simply encoded as a single mean but includes properties such as numerosity, variance, and the shape of a distribution. Such multiplex nature of summary representation could be accounted for by a population summary that captures the distributional properties of a set rather than a single summary statistic. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Ensemble perception and focused attention: Two different modes of visual processing to cope with limited capacity.

The visual system has a limited capacity for dealing with complex and redundant information in a scene. Here, we propose that a distributed attention mode of processing is necessary for coping with this limit, together with a focused attention mode of processing. The distributed attention mode provides a statistical summary of a scene, whereas the focused attention mode provides relevant information for object recognition. In this paper, we claim that a distributed mode of processing is necessary because (1) averaging performance improves with increased set-sizes, (2) even unselected items are likely to contribute to averaging, and (3) the assumption of variable capacity limits in averaging over different set-sizes is not plausible. We then propose how the averaging process can access multiple items over the capacity limit of focused attention. The visual system can represent multiple items as population responses and read out relevant information using the two modes of attention. It can summarize population responses with a broad application of a Gaussian profile (i.e., distributed attention) and represent its peak as the mean. It can focus on relevant population responses with a narrow application of a Gaussian profile (i.e., focused attention) and select important information for object recognition. The two attention modes of processing provide a framework for incorporating two seemingly opposing fields of study (ensemble perception and selective attention) and a unified theory of a coping strategy with our limited capacity.

Adaptation to mean and variance: Interrelationships between mean and variance representations in orientation perception.

When there are many visual items, the visual system could represent their summary statistics (e.g., mean, variance) to process them efficiently. Although many previous studies have investigated the mean or variance representation itself, a relationship between these two ensemble representations has not been investigated much. In this study, we tested the potential interaction between mean and variance representations by using a visual adaptation method. We reasoned that if mean and variance representations interact with each other, an adaptation aftereffect to either mean or variance would influence the perception of the other. Participants watched a sequence of orientation arrays containing a specific statistical property during the adaptation period. To produce an adaptation aftereffect specific to variance or mean, one property of the adaptor arrays (variance or mean) had a fixed value while the other property was randomly varied. After the adaptation, participants were asked to discriminate the property of the test array that was randomly varied during the adaptation. We found that the adaptation aftereffect of orientation variance influenced the sensitivity of mean orientation discrimination (Experiment 1), and that the adaptation aftereffect of mean orientation influenced the bias of orientation variance discrimination (Experiment 2). These results suggest that mean and variance representations do closely interact with each other. Considering that mean and variance reflect the representative value and dispersion of multiple items respectively, the interactions between mean and variance representations may reflect their complementary roles to summarize complex visual information effectively.

Distributed attention model of perceptual averaging.

The visual system efficiently processes complex and redundant information in a scene despite its limited capacity. One strategy for coping with the complexity and redundancy of a scene is to summarize it by using average information. However, despite its importance, the mechanism of averaging is not well understood. Here, a distributed attention model of averaging is proposed. Human percept for an object can be disturbed by various sources of internal noise, which can occur either before (early noise) or after (late noise) forming an ensemble perception. The model assumes these noises and reflects noise cancellation by averaging multiple items. The model predicts increased precision for more items with decelerated increments for large set-sizes resulting from late noise. Importantly, the model incorporates mechanisms of attention, which modulate each item's contribution to the averaging process. The attention in the model also results in saturation of performance increments for small set-sizes because the amount of attention allocated to each item is greater for small set-sizes than for large set-sizes. To evaluate the proposed model, a psychophysical experiment was conducted in which observers' ability to discriminate average sizes of two displays was measured. The observers' averaging performance increased at a decreasing rate with small set-sizes and it approached an asymptote for large set-sizes. The model accurately predicted the observed pattern of data. It provides a theoretical framework for interpreting behavioral data and leads to an understanding of the characteristics of ensemble perception.