Attention in redundancy masking.

Peripheral vision is limited due to several factors, such as visual resolution, crowding, and attention. When attention is not directed towards a stimulus, detection, discrimination, and identification are often compromised. Recent studies have found a new phenomenon that strongly limits peripheral vision, "redundancy masking". In redundancy masking, the number of perceived items in repeating patterns is reduced. For example, when presenting three lines in the peripheral visual field and asking participants to report the number of lines, often only two lines are reported. Here, we investigated what role attention plays in redundancy masking. If redundancy masking was due to limited attention to the target, it should be stronger when less attention is allocated to the target, and absent when attention is maximally focused on the target. Participants were presented with line arrays and reported the number of lines in three cueing conditions (i.e., single cue, double cue, and no cue). Redundancy masking was observed in all cueing conditions, with observers reporting fewer lines than presented in the single, double, and no cue conditions. These results suggest that redundancy masking is not due to limited attention. The number of lines reported was closer to the correct number of lines in the single compared to the double and the no cue conditions, suggesting that reduced attention additionally compromised stimulus discrimination, and replicating typical effects of diminished attention. Taken together, our results suggest that the extent of attention to peripherally presented stimuli modulates discrimination performance, but does not account for redundancy masking.

The effect of processing partial information in dynamic face perception.

Face perception is a major topic in vision research. Most previous research has concentrated on (holistic) spatial representations of faces, often with static faces as stimuli. However, faces are highly dynamic stimuli containing important temporal information. How sensitive humans are regarding temporal information in dynamic faces is not well understood. Studies investigating temporal information in dynamic faces usually focus on the processing of emotional expressions. However, faces also contain relevant temporal information without any strong emotional expression. To investigate cues that modulate human sensitivity to temporal order, we utilized muted dynamic neutral face videos in two experiments. We varied the orientation of the faces (upright and inverted) and the presence/absence of eye blinks as partial dynamic cues. Participants viewed short, muted, monochromic videos of models vocalizing a widely known text (National Anthem). Videos were played either forward (in the correct temporal order) or backward. Participants were asked to determine the direction of the temporal order for each video, and (at the end of the experiment) whether they had understood the speech. We found that face orientation, and the presence/absence of an eye blink affected sensitivity, criterion (bias) and reaction time: Overall, sensitivity was higher for upright compared to inverted faces, and in the condition where an eye blink was present compared to the condition without an eye blink. Reaction times were mostly faster in the conditions with higher sensitivity. A bias to report inverted faces as 'backward' observed in Experiment I, where upright and inverted faces were presented randomly interleaved within each block, was absent when presenting upright and inverted faces in different blocks in Experiment II. Language comprehension results revealed that there was higher sensitivity when understanding the speech compared to not understanding the speech in both experiments. Taken together, our results showed higher sensitivity with upright compared to inverted faces, suggesting that the perception of dynamic, task-relevant information was superior with the canonical orientation of the faces. Furthermore, partial information coming from eye blinks, in addition to mouth movements, seemed to play a significant role in dynamic face perception, both when faces were presented upright and inverted. We suggest that studying the perception of facial dynamics beyond emotional expressions will help us to better understand the mechanisms underlying the temporal integration of facial information from different -partial and holistic- sources, and that our results show how different strategies, depending on the available information, are employed by human observers when judging the temporal order of faces.

Foveal vision determines the perceived emotion of face ensembles.

People can extract summary statistical information from groups of similar objects, an ability called ensemble perception. However, not every object in a group is weighted equally. For example, in ensemble emotion perception, faces far from fixation were weighted less than faces close to fixation. Yet the contribution of foveal input in ensemble emotion perception is still unclear. In two experiments, groups of faces with varying emotions were presented for 100 ms at three different eccentricities (0°, 3°, 8°). Observers reported the perceived average emotion of the group. In two conditions, stimuli consisted of a central face flanked by eight faces (flankers) (central-present condition) and eight faces without the central face (central-absent condition). In the central-present condition, the emotion of the central face was either congruent or incongruent with that of the flankers. In Experiment 1, flanker emotions were uniform (identical flankers); in Experiment 2 they were varied. In both experiments, performance in the central-present condition was superior at 3° compared to 0° and 8°. At 0°, performance was superior in the central-absent (i.e., no foveal input) compared to the central-present condition. Poor performance in the central-present condition was driven by the incongruent condition where the foveal face strongly biased responses. At 3° and 8°, performance was comparable between central-present and central-absent conditions. Our results showed how foveal input determined the perceived emotion of face ensembles, suggesting that ensemble perception fails when salient target information is available in central vision.

The Sabanci University Dynamic Face Database (SUDFace): Development and validation of an audiovisual stimulus set of recited and free speeches with neutral facial expressions.

Faces convey a wide range of information, including one's identity, and emotional and mental states. Face perception is a major research topic in many research fields, such as cognitive science, social psychology, and neuroscience. Frequently, stimuli are selected from a range of available face databases. However, even though faces are highly dynamic, most databases consist of static face stimuli. Here, we introduce the Sabanci University Dynamic Face (SUDFace) database. The SUDFace database consists of 150 high-resolution audiovisual videos acquired in a controlled lab environment and stored with a resolution of 1920 × 1080 pixels at a frame rate of 60 Hz. The multimodal database consists of three videos of each human model in frontal view in three different conditions: vocalizing two scripted texts (conditions 1 and 2) and one Free Speech (condition 3). The main focus of the SUDFace database is to provide a large set of dynamic faces with neutral facial expressions and natural speech articulation. Variables such as face orientation, illumination, and accessories (piercings, earrings, facial hair, etc.) were kept constant across all stimuli. We provide detailed stimulus information, including facial features (pixel-wise calculations of face length, eye width, etc.) and speeches (e.g., duration of speech and repetitions). In two validation experiments, a total number of 227 participants rated each video on several psychological dimensions (e.g., neutralness and naturalness of expressions, valence, and the perceived mental states of the models) using Likert scales. The database is freely accessible for research purposes.

Anisotropic representations of visual space modulate visual numerosity estimation.

Humans can estimate the number of visually displayed items without counting. This capacity of numerosity perception has often been attributed to a dedicated system to estimate numerosity, or alternatively to the exploitation of various stimulus features, such as density, convex hull, the size of items, and occupancy area. The distribution of the presented items is usually not varied with eccentricity in the visual field. However, our visual fields are highly asymmetric. To date, it is unclear how inhomogeneities of the visual field impact numerosity perception. Besides eccentricity, a pronounced asymmetry is the radial-tangential anisotropy. For example, in crowding, radially placed flankers interfere more strongly with target perception than tangentially placed flankers. Similarly, in redundancy masking, the number of perceived items in repeating patterns is reduced when the items are arranged radially but not when they are arranged tangentially. Here, we investigated whether numerosity perception is subject to the radial-tangential anisotropy of spatial vision to shed light on the underlying topology of numerosity perception. In Experiment 1, observers were presented with varying numbers of discs, predominantly arranged radially or tangentially, and asked to report their perceived number. In Experiment 2, observers were presented with the same displays as in Experiment 1, and were asked to encircle items that were perceived as a group. We found that numerosity estimation depended on the arrangement of discs, suggesting a radial-tangential anisotropy of numerosity perception. Grouping among discs did not seem to explain our results. We suggest that the topology of spatial vision modulates numerosity estimation and that asymmetries of visual space should be taken into account when investigating numerosity estimation.

High confidence and low accuracy in redundancy masking.

Visual scenes typically contain redundant information. One mechanism by which the visual system compresses such redundancies is 'redundancy masking' - the reduction of the perceived number of items in repeating patterns. For example, when presented with three lines in the periphery, observers frequently report only two lines. Redundancy masking is strong in radial arrangements and absent in tangential arrangements. Previous studies suggested that redundancy-masked percepts predominate in stimuli susceptible to redundancy masking. Here, we investigated whether strong redundancy masking is associated with high confidence in perceptual judgements. Observers viewed three to seven radially or tangentially arranged lines at 10° eccentricity. They first indicated the number of lines, and then rated their confidence in their responses. As expected, redundancy masking was strong in radial arrangements and weak in tangential arrangements. Importantly, with radial arrangements, observers were more confident in their responses when redundancy masking occurred (i.e., lower number of lines reported) than when it did not occur (i.e., correct number of lines reported). Hence, observers reported higher confidence for erroneous than for correct judgments. In contrast, with tangential arrangements, observers were similarly confident in their responses whether redundancy masking occurred or not. The inversion of confidence in the radial condition (higher confidence when accuracy was low and lower confidence when accuracy was high) suggests that redundancy-masked appearance trumps 'veridical' perception. The often-reported richness of visual consciousness may partly be due to overconfidence in erroneous judgments in visual scenes that are subject to redundancy masking.

Multidimensional feature interactions in visual crowding: When configural cues eliminate the polarity advantage.

Crowding occurs when surrounding objects (flankers) impair target perception. A key property of crowding is the weaker interference when target and flankers strongly differ on a given dimension. For instance, identification of a target letter is usually superior with flankers of opposite versus the same contrast polarity as the target (the "polarity advantage"). High performance when target-flanker similarity is low has been attributed to the ungrouping of target and flankers. Here, we show that configural cues can override the usual advantage of low target-flanker similarity, and strong target-flanker grouping can reduce - instead of exacerbate - crowding. In Experiment 1, observers were presented with line triplets in the periphery and reported the tilt (left or right) of the central line. Target and flankers had the same (uniform condition) or opposite contrast polarity (alternating condition). Flanker configurations were either upright (||), unidirectionally tilted (\\ or //), or bidirectionally tilted (\/ or /\). Upright flankers yielded stronger crowding than unidirectional flankers, and weaker crowding than bidirectional flankers. Importantly, our results revealed a clear interaction between contrast polarity and flanker configuration. Triplets with upright and bidirectional flankers, but not unidirectional flankers, showed the polarity advantage. In Experiments 2 and 3, we showed that emergent features and redundancy masking (i.e. the reduction of the number of perceived items in repeating configurations) made it easier to discriminate between uniform triplets when flanker tilts were unidirectional (but not when bidirectional). We propose that the spatial configurations of uniform triplets with unidirectional flankers provided sufficient task-relevant information to enable a similar performance as with alternating triplets: strong-target flanker grouping alleviated crowding. We suggest that features which modulate crowding strength can interact non-additively, limiting the validity of typical crowding rules to contexts where only single, independent dimensions determine the effects of target-flanker similarity.

Atypical visual field asymmetries in redundancy masking.

Redundancy masking is the reduction of the perceived number of items in repeating patterns. It shares a number of characteristics with crowding, the impairment of target identification in visual clutter. Crowding strongly depends on the location of the target in the visual field. For example, it is stronger in the upper compared to the lower visual field and is usually weakest on the horizontal meridian. This pattern of visual field asymmetries is common in spatial vision, as revealed by tasks measuring, for example, spatial resolution and contrast sensitivity. Here, to characterize redundancy masking and reveal its similarities to and differences from other spatial tasks, we investigated whether redundancy masking shows the same typical visual field asymmetries. Observers were presented with three to six radially arranged lines at 10° eccentricity at one of eight locations around fixation and were asked to report the number of lines. We found asymmetries that differed pronouncedly from those found in crowding. Redundancy masking did not differ between upper and lower visual fields. Importantly, redundancy masking was stronger on the horizontal meridian than on the vertical meridian, the opposite of what is usually found in crowding. These results show that redundancy masking diverges from crowding in regard to visual field asymmetries, suggesting different underlying mechanisms of redundancy masking and crowding. We suggest that the observed atypical visual field asymmetries in redundancy masking are due to the superior extraction of regularity and a more pronounced compression of visual space on the horizontal compared to the vertical meridian.

Broad attention uncovers benefits of stimulus uniformity in visual crowding.

Crowding is the interference by surrounding objects (flankers) with target perception. Low target-flanker similarity usually yields weaker crowding than high similarity ('similarity rule') with less interference, e.g., by opposite- than same-contrast polarity flankers. The advantage of low target-flanker similarity has typically been shown with attentional selection of a single target object. Here, we investigated the validity of the similarity rule when broadening attention to multiple objects. In three experiments, we measured identification for crowded letters (Experiment 1), tumbling Ts (Experiment 2), and tilted lines (Experiment 3). Stimuli consisted of three items that were uniform or alternating in contrast polarity and were briefly presented at ten degrees eccentricity. Observers reported all items (full report) or only the left, central, or right item (single-item report). In Experiments 1 and 2, consistent with the similarity rule, single central item performance was superior with opposite- compared to same-contrast polarity flankers. With full report, the similarity rule was inverted: performance was better for uniform compared to alternating stimuli. In Experiment 3, contrast polarity did not affect performance. We demonstrated a reversal of the similarity rule under broadened attention, suggesting that stimulus uniformity benefits crowded object recognition when intentionally directing attention towards all stimulus elements. We propose that key properties of crowding have only limited validity as they may require a-priori differentiation of target and context.

Geometrically restricted image descriptors: A method to capture the appearance of shape.

Shape perception varies depending on many factors. For example, presenting a stimulus in the periphery often yields a different appearance compared with its foveal presentation. However, how exactly shape appearance is altered under different conditions remains elusive. One reason for this is that studies typically measure identification performance, leaving details about target appearance unknown. The lack of appearance-based methods and general challenges to quantify appearance complicate the investigation of shape appearance. Here, we introduce Geometrically Restricted Image Descriptors (GRIDs), a method to investigate the appearance of shapes. Stimuli in the GRID paradigm are shapes consisting of distinct line elements placed on a grid by connecting grid nodes. Each line is treated as a discrete target. Observers are asked to capture target appearance by placing lines on a freely viewed response grid. We used GRIDs to investigate the appearance of letters and letter-like shapes. Targets were presented at 10° eccentricity in the right visual field. Gaze-contingent stimulus presentation was used to prevent eye movements to the target. The data were analyzed by quantifying the differences between targets and response in regard to overall accuracy, element discriminability, and several distinct error types. Our results show how shape appearance can be captured by GRIDs, and how a fine-grained analysis of stimulus parts provides quantifications of appearance typically not available in standard measures of performance. We propose that GRIDs are an effective tool to investigate the appearance of shapes.

Hidden by bias: how standard psychophysical procedures conceal crucial aspects of peripheral visual appearance.

The perception of a target depends on other stimuli surrounding it in time and space. This contextual modulation is ubiquitous in visual perception, and is usually quantified by measuring performance on sets of highly similar stimuli. Implicit or explicit comparisons among the stimuli may, however, inadvertently bias responses and conceal strong variability of target appearance. Here, we investigated the influence of contextual stimuli on the perception of a repeating pattern (a line triplet), presented in the visual periphery. In the neutral condition, the triplet was presented a single time to capture its minimally biased perception. In the similar and dissimilar conditions, it was presented within stimulus sets composed of lines similar to the triplet, and distinct shapes, respectively. The majority of observers reported perceiving a line pair in the neutral and dissimilar conditions, revealing 'redundancy masking', the reduction of the perceived number of repeating items. In the similar condition, by contrast, the number of lines was overestimated. Our results show that the similar context did not reveal redundancy masking which was only observed in the neutral and dissimilar context. We suggest that the influence of contextual stimuli has inadvertently concealed this crucial aspect of peripheral appearance.

Capsule networks as recurrent models of grouping and segmentation.

Classically, visual processing is described as a cascade of local feedforward computations. Feedforward Convolutional Neural Networks (ffCNNs) have shown how powerful such models can be. However, using visual crowding as a well-controlled challenge, we previously showed that no classic model of vision, including ffCNNs, can explain human global shape processing. Here, we show that Capsule Neural Networks (CapsNets), combining ffCNNs with recurrent grouping and segmentation, solve this challenge. We also show that ffCNNs and standard recurrent CNNs do not, suggesting that the grouping and segmentation capabilities of CapsNets are crucial. Furthermore, we provide psychophysical evidence that grouping and segmentation are implemented recurrently in humans, and show that CapsNets reproduce these results well. We discuss why recurrence seems needed to implement grouping and segmentation efficiently. Together, we provide mutually reinforcing psychophysical and computational evidence that a recurrent grouping and segmentation process is essential to understand the visual system and create better models that harness global shape computations.

Redundancy masking: The loss of repeated items in crowded peripheral vision.

Crowding is the deterioration of target identification in the presence of neighboring objects. Recent studies using appearance-based methods showed that the perceived number of target elements is often diminished in crowding. Here we introduce a related type of diminishment in repeating patterns (sets of parallel lines), which we term "redundancy masking." In four experiments, observers were presented with arrays of small numbers of lines centered at 10° eccentricity. The task was to indicate the number of lines. In Experiment 1, spatial characteristics of redundancy masking were examined by varying the inter-line spacing. We found that redundancy masking decreased with increasing inter-line spacing and ceased at spacings of approximately 0.25 times the eccentricity. In Experiment 2, we assessed whether the strength of redundancy masking differed between radial and tangential arrangements of elements as it does in crowding. Redundancy masking was strong with radially arranged lines (horizontally arranged vertical lines), and absent with tangentially arranged lines (vertically arranged horizontal lines). In Experiment 3, we investigated whether target size (line width and length) modulated redundancy masking. There was an effect of width: Thinner lines yielded stronger redundancy masking. We did not find any differences between the tested line lengths. In Experiment 4, we varied the regularity of the line arrays by vertically or horizontally jittering the positions of the lines. Redundancy masking was strongest with regular spacings and weakened with decreasing regularity. Our experiments show under which conditions whole items are lost in crowded displays, and how this redundancy masking resembles-and partly diverges from-crowded identification. We suggest that redundancy masking is a contributor to the deterioration of performance in crowded displays with redundant patterns.

Emergent features break the rules of crowding.

Crowding is the deleterious influence of surrounding objects (flankers) on target identification. A central rule of crowding is that it is stronger when the target and the flankers are similar. Here, we show in three experiments how emergent features break this rule. Observers identified targets with various emergent features consisting of a pair of adjacent chevrons either pointing in opposite ('Diamonds' and 'Xs'), or the same (both up or down) directions. Targets were flanked by Diamonds or Xs, resulting in conditions with different levels of target-flanker similarity. Despite high target-flanker similarity, Diamonds were identified better than Xs when flanked by Diamonds. Participants' judgments of target conspicuity, however, showed that Diamonds were not perceived to stand out more strongly from X than Diamond flankers. Next, we asked observers to indicate whether all presented items were identical. We found superior performance with all Diamonds compared to all Xs, indicating that display uniformity judgments benefitted from the emergent features of Diamonds. Our results showed that emergent features and the information content of the entire display strongly modulated crowding. We suggest that conventional crowding rules only hold when target and flankers are artificially constrained to be mutually independent.

Letters Lost: Capturing Appearance in Crowded Peripheral Vision Reveals a New Kind of Masking.

Peripheral vision is strongly limited by crowding, the deleterious influence of flanking items on target perception. Distinguishing what is seen from what is merely inferred in crowding is difficult because task demands and prior knowledge may influence observers' reports. Here, we used a standard identification task in which participants were susceptible to these influences, and to minimize them, we used a free-report-and-drawing paradigm. Three letters were presented in the periphery. In Experiment 1, 10 participants were asked to identify the central target letter. In Experiment 2, 25 participants freely named and drew what they saw. When three identical letters were presented, performance was almost perfect in Experiment 1, but it was very poor in Experiment 2, in which most participants reported only two letters. Our study reveals limitations of standard crowding paradigms and uncovers a hitherto unrecognized effect that we call redundancy masking.

Disrupting uniformity: Feature contrasts that reduce crowding interfere with peripheral word recognition.

Peripheral word recognition is impaired by crowding, the harmful influence of surrounding objects (flankers) on target identification. Crowding is usually weaker when the target and the flankers differ (for example in color). Here, we investigated whether reducing crowding at syllable boundaries improved peripheral word recognition. In Experiment 1, a target letter was flanked by single letters to the left and right and presented at 8° in the lower visual field. Target and flankers were either the same or different in regard to contrast polarity, color, luminance, and combined color/luminance. Crowding was reduced when the target differed from the flankers in contrast polarity, but not in any of the other conditions. Using the same color and luminance values as in Experiment 1, we measured recognition performance (speed and accuracy) for uniform (e.g., all letters black), congruent (e.g., alternating black and white syllables), and incongruent (e.g., alternating black and white non-syllables) words in Experiment 2. Participants verbally reported the target word, briefly displayed at 8° in the lower visual field. Congruent and incongruent words were recognized slower compared to uniform words in the opposite contrast polarity condition, but not in the other conditions. Our results show that the same feature contrast between the target and the flankers that yielded reduced crowding, deteriorated peripheral word recognition when applied to syllables and non-syllabic word parts. We suggest that a potential advantage of reduced crowding at syllable boundaries in word recognition is counteracted by the disruption of word uniformity.

Emergent features in the crowding zone: When target-flanker grouping surmounts crowding.

Crowding is the impairment of target identification when the target is surrounded by nearby flankers. Two hallmarks of crowding are that it is stronger when the flankers are close to the target and when the target strongly groups with the flankers. Here we show the opposite of both. A chevron target (pointing up or down) was presented at 8° eccentricity in the right visual field. It was surrounded by four flankers. Three of the flankers varied (pointing left or right). The fourth, the critical flanker (CF), was fixed in one orientation (left, right, up, down), yielding different configurations with the target. The CF's distance to the target was varied. Target identification depended strongly on the distance and the orientation of the CF. Remarkably, when the target and the CF grouped into a good configuration and elicited an emergent feature, performance was high if the CF was close to the target. This effect was particularly strong when participants were informed about the different CF-target configurations before the experiment. Reducing crowding and grouping by asynchronous presentation of the CF and the other items abolished the effect. When participants reported the entire configuration of the CF and the target, performance rapidly decreased with increasing spacing when the CF and the target were different but not when they were the same, indicating different spatial extents of the corresponding grouping processes. Our results show that the features emerging from the configurations of the target and a flanker strongly modulate crowding. Strong target-flanker grouping can benefit performance.

Appearance changes and error characteristics in crowding revealed by drawings.

Peripheral vision is strongly limited by crowding: Targets that are easily recognized in isolation are unrecognizable when flanked by close-by objects. Crowding does not only impair target recognition but also changes appearance. Here we investigated appearance changes and errors in crowding by letting observers draw crowded stimuli. Observers drew stimuli presented at 6° and 12° eccentricity. Stimuli consisted of characters and letter-like symbols. Targets were presented with either a flanker on each side or in isolation. To characterize appearance changes and errors in crowding, we developed a scoring system that captured differences between the drawings and the stimuli. The resulting drawings revealed strong appearance changes under crowding. Importantly, our results reveal crowding errors that are usually not shown in standard crowding paradigms. We found high rates of element Omissions and element Truncations, indicating a central role of target "diminishment" in crowding. Furthermore, we show that a subset of the observed element Omissions and Additions was possibly caused by feature migration. Relatively high rates of position errors, in particular element Translations, reflected the often reported location uncertainty in crowding. Virtually no complete target-flanker substitutions were observed. We suggest a new classification system for errors in crowding, and propose drawing as a useful appearance-based method to investigate crowding.