A deep new look at color.

Bowers et al. counter deep neural networks (DNNs) as good models of human visual perception. From our color perspective we feel their view is based on three misconceptions: A misrepresentation of the state-of-the-art of color perception; the type of model required to move the field forward; and the attribution of shortcomings to DNN research that are already being resolved.

Emergent color categorization in a neural network trained for object recognition.

Color is a prime example of categorical perception, yet it is unclear why and how color categories emerge. On the one hand, prelinguistic infants and several animals treat color categorically. On the other hand, recent modeling endeavors have successfully utilized communicative concepts as the driving force for color categories. Rather than modeling categories directly, we investigate the potential emergence of color categories as a result of acquiring visual skills. Specifically, we asked whether color is represented categorically in a convolutional neural network (CNN) trained to recognize objects in natural images. We systematically trained new output layers to the CNN for a color classification task and, probing novel colors, found borders that are largely invariant to the training colors. The border locations were confirmed using an evolutionary algorithm that relies on the principle of categorical perception. A psychophysical experiment on human observers, analogous to our primary CNN experiment, shows that the borders agree to a large degree with human category boundaries. These results provide evidence that the development of basic visual skills can contribute to the emergence of a categorical representation of color.

Inhibition of return in the oculomotor decision process: Dissociating visual target discrimination from saccade readiness delays.

Saccades toward previously cued or fixated locations typically have longer latencies than those toward novel locations, a phenomenon known as inhibition of return (IOR). Despite extensive debate on its potential function, it remains unclear what the role of IOR in the oculomotor decision process is. Here, we ask whether the effect on eye movement planning is best characterized as a delay in visual target discrimination or as a reduction in readiness to execute the movement (saccade readiness). To evaluate this question, we use target-distractor tasks with clear speed-accuracy trade-offs. Simultaneously cueing both the target and distractor (or neither) we find longer latencies at the cued locations. Despite this delay in latencies, accuracy improves in line with the speed-accuracy trade-off curve (Experiment 1). This suggests that while visual target discrimination can progress unimpeded, saccade readiness is reduced. Based on this reduction in readiness we predict that the more saccades rely on visual target discrimination, the less their destination will be affected by inducing IOR. Indeed, after cueing either the target or an onset distractor (Experiment 2), short-latency, stimulus-driven, saccades are strongly biased away from the cued location, while the destinations of longer latency goal-driven saccades are affected only minimally. The fact that primarily stimulus-driven saccades are affected by inducing IOR is interesting as it can explain why the spatial bias associated with IOR is not consistently found. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

The Lifetime of Salience Extends Beyond the Initial Saccade.

Several models of selection in search predict that saccades are biased toward conspicuous objects (also referred to as salient objects). Indeed, it has been demonstrated that initial saccades are biased toward the most conspicuous candidate. However, in a recent study, no such bias was found for the second saccade, and it was concluded that the attraction of conspicuous elements is limited to only short-latency initial saccades. This conclusion is based on only a single feature manipulation (orientation contrast) and conflicts with the prediction of influential salience models. Here, we investigate whether this result can be generalized beyond the domain of orientation. In displays containing three luminance annuli (Experiment 1), we find a considerable bias toward the most conspicuous candidate for the second saccade. In Experiment 1, the target could not be discriminated peripherally. When we made the target peripherally discriminable, the second saccade was no longer biased toward the more conspicuous candidate (Experiment 2). Thus, conspicuity plays a role in saccadic selection beyond the initial saccade. Whether second saccades are biased toward conspicuous objects appears to depend on the type of feature contrast underlying the conspicuity and the peripheral discriminability of target properties.

Revisiting the global effect and inhibition of return.

Saccades toward previously cued locations have longer latencies than saccades toward other locations, a phenomenon known as inhibition of return (IOR). Watanabe (Exp Brain Res 138:330-342. doi: 10.1007/s002210100709 , 2001) combined IOR with the global effect (where saccade landing points fall in between neighboring objects) to investigate whether IOR can also have a spatial component. When one of two neighboring targets was cued, there was a clear bias away from the cued location. In a condition where both targets were cued, it appeared that the global effect magnitude was similar to the condition without any cues. However, as the latencies in the double cue condition were shorter compared to the no cue condition, it is still an open question whether these results are representative for IOR. Considering the double cue condition can provide valuable insight into the interaction of the mechanisms underlying the two phenomena, here, we revisit this condition in an adapted paradigm. Our paradigm does result in longer latencies for the cued locations, and we find that the magnitude of the global effect is reduced significantly. Unexpectedly, this holds even when only including saccades with the same latencies for both conditions. Thus, the increased latencies associated with IOR cannot directly explain the reduction in global effect. The global effect reduction can likely best be seen as either a result of short-term depression of exogenous visual signals or a result of IOR established at the center of gravity of cues.

Saccades toward the target are planned as sequences rather than as single steps.

To find a target during visual search, observers often need to make multiple eye movements, which results in a scan path. It is an open question whether the saccade destinations in scan paths are planned ahead. In the two experiments reported here, we investigated this question by focusing on the observer's ability to deviate from potentially planned paths. In the first experiment, the stimulus configuration could change during the initial saccade. We found that the observer's ability to deviate from potentially planned paths crucially depended on whether altered configurations could be processed with sufficient rapidity. In a follow-up experiment, we asked whether planned paths can include more than two saccade destinations. Investigating the influence of potentially planned paths on a secondary task demonstrated that planned paths can include at least three saccade destinations. Together, these experiments provide the first evidence of scan-path planning in visual search.

Background, an important factor in visual search.

The ability to detect an object depends on the contrast between the object and its background. Despite this, many models of visual search rely solely on the properties of target and distractors, and do not take the background into account. Yet, both target and distractors have their individual contrasts with the background. These contrasts generally differ, because the target and distractors are different in at least one feature. Therefore, background is likely to play an important role in visual search. In three experiments we manipulated the properties of the background (luminance, orientation and spatial frequency, respectively) while keeping the target and distractors constant. In the first experiment, in which target and distractors had a different luminance, changing the background luminance had an extensive effect on search times. When background luminance was in between that of the target and distractors, search times were always short. Interestingly, when the background was darker than both the target and the distractors, search times were much longer than when the background was lighter. Manipulating orientation and spatial frequency of the background, on the other hand, resulted in search times that were longest for small target-background differences. Thus, background plays an important role in search. This role depends on the individual contrast of both target and distractors with the background and the type of feature contrast (luminance, orientation or spatial frequency).

How longer saccade latencies lead to a competition for salience.

It has been suggested that independent bottom-up and top-down processes govern saccadic selection. However, recent findings are hard to explain in such terms. We hypothesized that differences in visual-processing time can explain these findings, and we tested this using search displays containing two deviating elements, one requiring a short processing time and one requiring a long processing time. Following short saccade latencies, the deviation requiring less processing time was selected most frequently. This bias disappeared following long saccade latencies. Our results suggest that an element that attracts eye movements following short saccade latencies does so because it is the only element processed at that time. The temporal constraints of processing visual information therefore seem to be a determining factor in saccadic selection. Thus, relative saliency is a time-dependent phenomenon.

Saccadic selection and crowding in visual search: stronger lateral masking leads to shorter search times.

We investigated the role of crowding in saccadic selection during visual search. To guide eye movements, often information from the visual periphery is used. Crowding is known to deteriorate the quality of peripheral information. In four search experiments, we studied the role of crowding, by accompanying individual search elements by flankers. Varying the difference between target and flankers allowed us to manipulate crowding strength throughout the stimulus. We found that eye movements are biased toward areas with little crowding for conditions where a target could be discriminated peripherally. Interestingly, for conditions in which the target could not be discriminated peripherally, this bias reversed to areas with strong crowding. This led to shorter search times for a target presented in areas with stronger crowding, compared to a target presented in areas with less crowding. These findings suggest a dual role for crowding in visual search. The presence of flankers similar to the target deteriorates the quality of the peripheral target signal but can also attract eye movements, as more potential targets are present over the area.