Optimizing colour for camouflage and visibility using deep learning: the effects of the environment and the observer's visual system.

Avoiding detection can provide significant survival advantages for prey, predators, or the military; conversely, maximizing visibility would be useful for signalling. One simple determinant of detectability is an animal's colour relative to its environment. But identifying the optimal colour to minimize (or maximize) detectability in a given natural environment is complex, partly because of the nature of the perceptual space. Here for the first time, using image processing techniques to embed targets into realistic environments together with psychophysics to estimate detectability and deep neural networks to interpolate between sampled colours, we propose a method to identify the optimal colour that either minimizes or maximizes visibility. We apply our approach in two natural environments (temperate forest and semi-arid desert) and show how a comparatively small number of samples can be used to predict robustly the most and least effective colours for camouflage. To illustrate how our approach can be generalized to other non-human visual systems, we also identify the optimum colours for concealment and visibility when viewed by simulated red-green colour-blind dichromats, typical for non-human mammals. Contrasting the results from these visual systems sheds light on why some predators seem, at least to humans, to have colouring that would appear detrimental to ambush hunting. We found that for simulated dichromatic observers, colour strongly affected detection time for both environments. In contrast, trichromatic observers were more effective at breaking camouflage.

Fever in the returning traveller: the importance of sensitivity.

We report the case of a 28-year-old man, presenting with episodes of fever and rigours, having recently returned from Cameroon and Uganda. Initial investigations for malaria were negative, and the patient was sent home without a clear diagnosis. Subsequent review of the blood film revealed the presence of Plasmodium ovale. This case highlights the importance of repeated and careful inspection of blood films, given the relatively low sensitivity of rapid diagnostic tests in P. ovale infection. It also illustrates the importance of the travel history in the diagnosis of malaria.

A general rule for sensory cue summation: evidence from photographic, musical, phonetic and cross-modal stimuli.

The Euclidean and MAX metrics have been widely used to model cue summation psychophysically and computationally. Both rules happen to be special cases of a more general Minkowski summation rule , where m = 2 and ∞, respectively. In vision research, Minkowski summation with power m = 3-4 has been shown to be a superior model of how subthreshold components sum to give an overall detection threshold. Recently, we have previously reported that Minkowski summation with power m = 2.84 accurately models summation of suprathreshold visual cues in photographs. In four suprathreshold discrimination experiments, we confirm the previous findings with new visual stimuli and extend the applicability of this rule to cue combination in auditory stimuli (musical sequences and phonetic utterances, where m = 2.95 and 2.54, respectively) and cross-modal stimuli (m = 2.56). In all cases, Minkowski summation with power m = 2.5-3 outperforms the Euclidean and MAX operator models. We propose that this reflects the summation of neuronal responses that are not entirely independent but which show some correlation in their magnitudes. Our findings are consistent with electrophysiological research that demonstrates signal correlations (r = 0.1-0.2) between sensory neurons when these are presented with natural stimuli.

Perception of visual texture and the expression of disruptive camouflage by the cuttlefish, Sepia officinalis.

Juvenile cuttlefish (Sepia officinalis) camouflage themselves by changing their body pattern according to the background. This behaviour can be used to investigate visual perception in these molluscs and may also give insight into camouflage design. Edge detection is an important aspect of vision, and here we compare the body patterns that cuttlefish produced in response to checkerboard backgrounds with responses to backgrounds that have the same spatial frequency power spectrum as the checkerboards, but randomized spatial phase. For humans, phase randomization removes visual edges. To describe the cuttlefish body patterns, we scored the level of expression of 20 separate pattern 'components', and then derived principal components (PCs) from these scores. After varimax rotation, the first component (PC1) corresponded closely to the so-called disruptive body pattern, and the second (PC2) to the mottle pattern. PC1 was predominantly expressed on checkerboards, and PC2 on phase-randomized backgrounds. Thus, cuttlefish probably have edge detectors that control the expression of disruptive pattern. Although the experiments used unnatural backgrounds, it seems probable that cuttlefish display disruptive camouflage when there are edges in the visual background caused by discrete objects such as pebbles. We discuss the implications of these findings for our understanding of disruptive camouflage.

Generalization of color by chickens: experimental observations and a Bayesian model.

Sensory generalization influences animals' responses to novel stimuli. Because color forms a perceptual continuum, it is a good subject for studying generalization. Moreover, because different causes of variation in spectral signals, such as pigmentation, gloss, and illumination, have differing behavioral significance, it may be beneficial to have adaptable generalization. We report on generalization by poultry chicks following differential training to rewarded (T(+)) and unrewarded (T(-)) colors, in particular on the phenomenon of peak shift, which leads to subjects preferring stimuli displaced away from T(-). The first three experiments test effects of learning either a fine or a coarse discrimination. In experiments 1 and 2, peak shift occurs, but contrary to some predictions, the shift is smaller after the animal learned a fine discrimination than after it learned a coarse discrimination. Experiment 3 finds a similar effect for generalization on a color axis orthogonal to that separating T(+) from T(-). Experiment 4 shows that generalization is rapidly modified by experience. These results imply that the scale of a "perceptual ruler" is set by experience. We show that the observations are consistent with generalization following principles of Bayesian inference, which forms a powerful framework for understanding this type of behavior.

Cuttlefish responses to visual orientation of substrates, water flow and a model of motion camouflage.

Low-level mechanisms in vertebrate vision are sensitive to line orientation. Here we investigate orientation sensitivity in the cuttlefish Sepia pharaonis, by allowing animals to settle on stripe patterns. When camouflaging themselves cuttlefish are known to be sensitive to image parameters such as contrast and spatial scale, but we find no effect of background orientation on the patterns displayed. It is nonetheless clear that the animals see orientation, because they prefer to rest with the body-axis perpendicular to the stripes. We consider three possible mechanisms to account for this behaviour. Firstly, that the body patterns are themselves oriented, and that the cuttlefish align themselves to aid static camouflage. This is unlikely, as the patterns displayed have no dominant orientation at any spatial scale. A second possibility is that motion camouflage favours alignment of the body orthogonal to background stripes, and we suggest how this alignment can minimise motion signals produced by occlusion. Thirdly we show that cuttlefish prefer to rest with their body-axis parallel to the water flow, and it is possible that they use visual patterns such as sand ripples to determine water flow.

Stability of the color-opponent signals under changes of illuminant in natural scenes.

Illumination varies greatly both across parts of a natural scene and as a function of time, whereas the spectral reflectance function of surfaces remains more stable and is of much greater relevance when searching for specific targets. This study investigates the functional properties of postreceptoral opponent-channel responses, in particular regarding their stability against spatial and temporal variation in illumination. We studied images of natural scenes obtained in UK and Uganda with digital cameras calibrated to produce estimated L-, M-, and S-cone responses of trichromatic primates (human) and birds (starling). For both primates and birds we calculated luminance and red-green opponent (RG) responses. We also calculated a primate blue-yellow-opponent (BY) response. The BY response varies with changes in illumination, both across time and across the image, rendering this factor less invariant. The RG response is much more stable than the BY response across such changes in illumination for primates, less so for birds. These differences between species are due to the greater separation of bird L and M cones in wavelength and the narrower bandwidth of the cone action spectra. This greater separation also produces a larger chromatic signal for a given change in spectral reflectance. Thus bird vision seems to suffer a greater degree of spatiotemporal "clutter" than primate vision, but also enhances differences between targets and background. Therefore, there may be a trade-off between the degree of chromatic clutter in a visual system versus the degree of chromatic difference between a target and its background. Primate and bird visual systems have found different solutions to this trade-off.

Modular organization of adaptive colouration in flounder and cuttlefish revealed by independent component analysis.

Flounders and cuttlefish have an impressive ability to change colouration, for camouflage and, in the case of cuttlefish, for communication. We pursue the hypothesis that these diverse patterns are created by combining a small number of distinct pattern modules. Independent component analysis (ICA) is a powerful tool for identifying independent sources of variation in linear mixtures of signals. Two versions of ICA are used, one assuming that sources have independence over time, and the other over space. These reveal the modularity of the skin colouration system, and suggest how the pattern modules are combined in specific behavioural contexts. ICA may therefore be a useful tool for studying animal camouflage and communication.

System identification applied to a visuomotor task: near-optimal human performance in a noisy changing task.

Sensory-motor integration has frequently been studied using a single-step change in a control variable such as prismatic lens angle and has revealed human visuomotor adaptation to often be partial and inefficient. We propose that the changes occurring in everyday life are better represented as the accumulation of many smaller perturbations contaminated by measurement noise. We have therefore tested human performance to random walk variations in the visual feedback of hand movements during a pointing task. Subjects made discrete targeted pointing movements to a visual target and received terminal feedback via a cursor the position of which was offset from the actual movement endpoint by a random walk element and a random observation element. By applying ideal observer analysis, which for this task compares human performance against that of a Kalman filter, we show that the subjects' performance was highly efficient with Fisher efficiencies reaching 73%. We then used system identification techniques to characterize the control strategy used. A "modified" delta-rule algorithm best modeled the human data, which suggests that they estimated the random walk perturbation of feedback in this task using an exponential weighting of recent errors. The time constant of the exponential weighting of the best-fitting model varied with the rate of random walk drift. Because human efficiency levels were high and did not vary greatly across three levels of observation noise, these results suggest that the algorithm the subjects used exponentially weighted recent errors with a weighting that varied with the level of drift in the task to maintain efficient performance.

Colour categorization by domestic chicks.

Spectral stimuli form a physical continuum, which humans divide into discrete non-overlapping regions or categories that are designated by colour names. Little is known about whether non-verbal animals form categories on stimulus continua, but work in psychology and artificial intelligence provides models for stimulus generalization and categorization. We compare predictions of such models to the way poultry chicks (Gallus gallus) generalize to novel stimuli following appetitive training to either one or two colours. If the two training colours are (to human eyes) red and greenish-yellow or green and blue, chicks prefer intermediates, i.e. orange rather than red or yellow and turquoise rather than green or blue. The level of preference for intermediate colours implies that the chicks interpolate between the training stimuli. However, they do not extrapolate beyond the limits set by the training stimuli, at least for red and yellow training colours. Similarly, chicks trained to red and blue generalize to purple, but they do not generalize across grey after training to the complementary colours yellow and blue. These results are consistent with a modified version of a Bayesian model of generalization from multiple examples that was proposed by Shepard and show similarities to human colour categorization.

On variability in the density of corticocortical and thalamocortical connections.

Variability is an important but neglected aspect of connectional neuroanatomy. The quantitative density of the 'same' corticocortical or thalamocortical connection may vary by over two orders of magnitude between different injections of the same tracer. At present, however, the frequency distribution of connection densities is unknown. Therefore, it is unclear what kind of sampling strategies or statistical methods are appropriate for quantitative studies of connectivity. Nor is it clear if the measured variability represents differences between subjects, or if it is simply a consequence of intra-individual differences resulting from experimental technique and the exact placement of tracers relative to local spatial and laminar variation in connectivity. We used quantitative measurements of the density of a large number of corticocortical and thalamocortical connections from our own laboratories and from the literature. Variability in the density of given corticocortical and thalamocortical connections is high, with the standard deviation of density proportional to the mean. The frequency distribution is close to exponential. Therefore, analysis methods relying on the normal distribution are not appropriate. We provide an appendix that gives simple statistical guidance for samples drawn from exponentially distributed data. For a given corticocortical or thalamocortical connection density, between-individual standard deviation is 0.85 to 1.25 times the within-individual standard deviation. Therefore, much of the variability reported in conventional neuroanatomical studies (with one tracer deposited per animal) is due to within-individual factors. We also find that strong, but not weak, corticocortical connections are substantially more variable than thalamocortical connections. We propose that the near exponential distribution of connection densities is a simple consequence of 'patchy' connectivity. We anticipate that connection data will be well described by the negative binomial, a class of distribution that applies to events occurring in clumped or patchy substrates. Local patchiness may be a feature of all corticocortical connections and could explain why strong corticocortical connections are more variable than strong thalamocortical connections. This idea is supported by the columnar patterns of many corticocortical but few thalamocortical connections in the literature.

Different mechanisms underlie three inhibitory phenomena in cat area 17.

Recently, it has been proposed that all suppressive phenomena observed in the primary visual cortex (V1) are mediated by a single mechanism, involving inhibition by pools of neurons, which, between them, represent a wide range of stimulus specificities. The strength of such inhibition would depend on the stimulus that produces it (particularly its contrast) rather than on the firing rate of the inhibited cell. We tested this hypothesis by measuring contrast-response functions (CRFs) of neurons in cat V1 for stimulation of the classical receptive field of the dominant eye with an optimal grating alone, and in the presence of inhibition caused by (1) a superimposed orthogonal grating (cross-orientation inhibition); (2) a surrounding iso-oriented grating (surround inhibition); and (3) an orthogonal grating in the other eye (interocular suppression). We fitted hyperbolic ratio functions and found that the effect of cross-orientation inhibition was best described as a rightward shift of the CRF ('contrast-gain control'), while surround inhibition and interocular suppression were primarily characterised as downward shifts of the CRF ('response-gain control'). However, the latter also showed a component of contrast-gain control. The two modes of suppression were differently distributed between the layers of cortex. Response-gain control prevailed in layer 4, whereas cells in layers 2/3, 5 and 6 mainly showed contrast-gain control. As in human observers, surround gratings caused suppression when the central grating was of high contrast, but in over a third of the cells tested, enhanced responses for low-contrast central stimuli, hence actually decreasing threshold contrast.

Insights into motion perception by observer modeling.

The statistical efficiency of human observers performing a simplified version of the motion detection task of Salzman and Newsome [Science 264, 231 (1994)] is high but not perfect. This reduced efficiency may be caused by noise internal to the observers or by the observers' using strategies that are different from that used by an ideal machine. We therefore investigated which of three simple models best accounts for the observers' performance. The models compared were a motion detector that uses the proportion of dots in the first frame that move coherently (as would an ideal machine), a model that bases its decision on the number of dots that move, and a model that differentially weights motions that occur at different locations in the visual field (for instance, differentially weights the point of fixation and the periphery). We compared these models by explicitly modeling the human observers' performance. We recorded the exact stimulus configuration on each trial together with the observer's response, and, for the different models, we found the parameters that best predicted the observer's performance in a least-squares sense. We then used N-fold cross validation to compare the models and hence the associated hypotheses. Our results show that the performance of observers is based on the proportion, not the absolute number, of dots that are moving and that there was no evidence of any differential spatial weighting. Whereas this method of modeling the observers' response is demonstrated only for one simple psychophysical paradigm, it is general and can be applied to any psychophysical framework in which the entire stimulus can be recorded.

Nonlinear principal components analysis of neuronal spike train data.

Many recent approaches to decoding neural spike trains depend critically on the assumption that for low-pass filtered spike trains, the temporal structure is optimally represented by a small number of linear projections onto the data. We therefore tested this assumption of linearity by comparing a linear factor analysis technique (principal components analysis) with a nonlinear neural network based method. It is first shown that the nonlinear technique can reliably identify a neuronally plausible nonlinearity in synthetic spike trains. However, when applied to the outputs from primary visual cortical neurons, this method shows no evidence for significant temporal nonlinearities. The implications of this are discussed.

Optimal, unsupervised learning in invariant object recognition.

A means for establishing transformation-invariant representations of objects is proposed and analyzed, in which different views are associated on the basis of the temporal order of the presentation of these views, as well as their spatial similarity. Assuming knowledge of the distribution of presentation times, an optimal linear learning rule is derived. Simulations of a competitive network trained on a character recognition task are then used t highlight the success of this learning rule in relation to simple Hebbian learning and to show that the theory can give accurate quantitative predictions for the optimal parameters for such networks.

Responses of neurons in primary and inferior temporal visual cortices to natural scenes.

The primary visual cortex (V1) is the first cortical area to receive visual input, and inferior temporal (IT) areas are among the last along the ventral visual pathway. We recorded, in area V1 of anaesthetized cats and area IT of awake macaque monkeys, responses of neurons to videos of natural scenes. Responses were analysed to test various hypotheses concerning the nature of neural coding in these two regions. A variety of spike-train statistics were measured including spike-count distributions, interspike interval distributions, coefficients of variation, power spectra, Fano factors and different sparseness measures. All statistics showed non-Poisson characteristics and several revealed self-similarity of the spike trains. Spike-count distributions were approximately exponential in both visual areas for eight different videos and for counting windows ranging from 50 ms to 5 seconds. The results suggest that the neurons maximize their information carrying capacity while maintaining a fixed long-term-average firing rate, or equivalently, minimize their average firing rate for a fixed information carrying capacity.

Searching for filters with 'interesting' output distributions: an uninteresting direction to explore?

It has been independently proposed, by Barlow, Field, Intrator and co-workers, that the receptive fields of neurons in V1 are optimized to generate 'sparse', Kurtotic, or 'interesting' output probability distributions. We investigate the empirical evidence for this further and argue that filters can produce 'interesting' output distributions simply because natural images have variable local intensity variance. If the proposed filters have zero DC, then the probability distribution of filter outputs (and hence the output Kurtosis) is well predicted simply from these effects of variable local variance. This suggests that finding filters with high output Kurtosis does not necessarily signal interesting image structure. It is then argued that finding filters that maximize output Kurtosis generates filters that are incompatible with observed physiology. In particular the optimal difference-of-Gaussian (DOG) filter should have the smallest possible scale, an on-centre off-surround cell should have a negative DC, and that the ratio of centre width to surround width should approach unity. This is incompatible with the physiology. Further, it is also predicted that oriented filters should always be oriented in the vertical direction, and of all the filters tested, the filter with the highest output Kurtosis has the lowest signal-to-noise ratio (the filter is simply the difference of two neighbouring pixels). Whilst these observations are not incompatible with the brain using a sparse representation, it does argue that little significance should be placed on finding filters with highly Kurtotic output distributions. It is therefore argued that other constraints are required in order to understand the development of visual receptive fields.

A statistical analysis of natural images matches psychophysically derived orientation tuning curves.

A neural net method is used to extract principal components from real-world images. The initial components are a Gaussian followed by horizontal and vertical operators, starting with the first derivative and moving to successively higher orders. Two of the components are 'bar-detectors'. Their measured orientation selectivity is similar to that suggested by Foster & Ward (Proc. R. Soc. Lond. B 243, 75 (1991] to account for brief-exposure psychophysical data. In tests with noise images, the ratio of sensitivity between the two components is controlled by the degree of anisotropy in the image.