Fixation-pattern similarity analysis reveals adaptive changes in face-viewing strategies following aversive learning.

Animals can effortlessly adapt their behavior by generalizing from past aversive experiences, allowing to avoid harm in novel situations. We studied how visual information was sampled by eye-movements during this process called fear generalization, using faces organized along a circular two-dimensional perceptual continuum. During learning, one face was conditioned to predict a harmful event, whereas the most dissimilar face stayed neutral. This introduced an adversity gradient along one specific dimension, while the other, unspecific dimension was defined solely by perceptual similarity. Aversive learning changed scanning patterns selectively along the adversity-related dimension, but not the orthogonal dimension. This effect was mainly located within the eye region of faces. Our results provide evidence for adaptive changes in viewing strategies of faces following aversive learning. This is compatible with the view that these changes serve to sample information in a way that allows discriminating between safe and adverse for a better threat prediction.

The periaqueductal gray and Bayesian integration in placebo analgesia.

In placebo hypoalgesia research, the strength of treatment expectations and experiences are key components. However, the reliability or precision of expectations had been mostly ignored although being a likely source for interindividual differences. In the present study, we adopted a Bayesian framework, naturally combining expectation magnitudes and precisions. This postulates that expectations (prior) are integrated with incoming nociceptive information (likelihood) and both are weighted by their relative precision to form the pain percept and placebo effect. Sixty-two healthy subjects received heat pain during fMRI. Placebo effects were more pronounced in subjects with more precise treatment expectations and correlated positively with the relative precision of the prior expectation. Neural correlates of this precision were observed in the periaqueductal gray and the rostral ventromedial medulla, indicating that already at the level of the brainstem the precision of an expectation can influence pain perception presenting strong evidence for Bayesian integration in placebo hypoalgesia.

Suppression of Striatal Prediction Errors by the Prefrontal Cortex in Placebo Hypoalgesia.

Classical learning theories predict extinction after the discontinuation of reinforcement through prediction errors. However, placebo hypoalgesia, although mediated by associative learning, has been shown to be resistant to extinction. We tested the hypothesis that this is mediated by the suppression of prediction error processing through the prefrontal cortex (PFC). We compared pain modulation through treatment cues (placebo hypoalgesia, treatment context) with pain modulation through stimulus intensity cues (stimulus context) during functional magnetic resonance imaging in 48 male and female healthy volunteers. During acquisition, our data show that expectations are correctly learned and that this is associated with prediction error signals in the ventral striatum (VS) in both contexts. However, in the nonreinforced test phase, pain modulation and expectations of pain relief persisted to a larger degree in the treatment context, indicating that the expectations were not correctly updated in the treatment context. Consistently, we observed significantly stronger neural prediction error signals in the VS in the stimulus context compared with the treatment context. A connectivity analysis revealed negative coupling between the anterior PFC and the VS in the treatment context, suggesting that the PFC can suppress the expression of prediction errors in the VS. Consistent with this, a participant's conceptual views and beliefs about treatments influenced the pain modulation only in the treatment context. Our results indicate that in placebo hypoalgesia contextual treatment information engages prefrontal conceptual processes, which can suppress prediction error processing in the VS and lead to reduced updating of treatment expectancies, resulting in less extinction of placebo hypoalgesia.SIGNIFICANCE STATEMENT In aversive and appetitive reinforcement learning, learned effects show extinction when reinforcement is discontinued. This is thought to be mediated by prediction errors (i.e., the difference between expectations and outcome). Although reinforcement learning has been central in explaining placebo hypoalgesia, placebo hypoalgesic effects show little extinction and persist after the discontinuation of reinforcement. Our results support the idea that conceptual treatment beliefs bias the neural processing of expectations in a treatment context compared with a more stimulus-driven processing of expectations with stimulus intensity cues. We provide evidence that this is associated with the suppression of prediction error processing in the ventral striatum by the prefrontal cortex. This provides a neural basis for persisting effects in reinforcement learning and placebo hypoalgesia.

An extensive dataset of eye movements during viewing of complex images.

We present a dataset of free-viewing eye-movement recordings that contains more than 2.7 million fixation locations from 949 observers on more than 1000 images from different categories. This dataset aggregates and harmonizes data from 23 different studies conducted at the Institute of Cognitive Science at Osnabrück University and the University Medical Center in Hamburg-Eppendorf. Trained personnel recorded all studies under standard conditions with homogeneous equipment and parameter settings. All studies allowed for free eye-movements, and differed in the age range of participants (~7-80 years), stimulus sizes, stimulus modifications (phase scrambled, spatial filtering, mirrored), and stimuli categories (natural and urban scenes, web sites, fractal, pink-noise, and ambiguous artistic figures). The size and variability of viewing behavior within this dataset presents a strong opportunity for evaluating and comparing computational models of overt attention, and furthermore, for thoroughly quantifying strategies of viewing behavior. This also makes the dataset a good starting point for investigating whether viewing strategies change in patient groups.

The neuronal basis of fear generalization in humans.

Organisms tend to respond similarly to stimuli that are perceptually close to an event that predicts adversity, a phenomenon known as fear generalization. Greater dissimilarity yields weaker behavioral responses, forming a fear-tuning profile. The perceptual model of fear generalization assumes that behavioral fear tuning results from perceptual similarities, suggesting that brain responses should also exhibit the same fear-tuning profile. Using fMRI and a circular fear-generalization procedure, we tested this prediction. In contrast with the perceptual model, insula responses showed less generalization than behavioral responses and encoded the aversive quality of the conditioned stimulus, as shown by high pattern similarity between the conditioned stimulus and the shock. Also inconsistent with the perceptual model, object-sensitive visual areas responded to ambiguity-related outcome uncertainty. Together these results indicate that fear generalization is not passively driven by perception, but is an active process integrating threat identification and ambiguity-based uncertainty to orchestrate a flexible, adaptive fear response.

Primary visual cortex represents the difference between past and present.

The visual system is confronted with rapidly changing stimuli in everyday life. It is not well understood how information in such a stream of input is updated within the brain. We performed voltage-sensitive dye imaging across the primary visual cortex (V1) to capture responses to sequences of natural scene contours. We presented vertically and horizontally filtered natural images, and their superpositions, at 10 or 33 Hz. At low frequency, the encoding was found to represent not the currently presented images, but differences in orientation between consecutive images. This was in sharp contrast to more rapid sequences for which we found an ongoing representation of current input, consistent with earlier studies. Our finding that for slower image sequences, V1 does no longer report actual features but represents their relative difference in time counteracts the view that the first cortical processing stage must always transfer complete information. Instead, we show its capacities for change detection with a new emphasis on the role of automatic computation evolving in the 100-ms range, inevitably affecting information transmission further downstream.

The contributions of image content and behavioral relevancy to overt attention.

During free-viewing of natural scenes, eye movements are guided by bottom-up factors inherent to the stimulus, as well as top-down factors inherent to the observer. The question of how these two different sources of information interact and contribute to fixation behavior has recently received a lot of attention. Here, a battery of 15 visual stimulus features was used to quantify the contribution of stimulus properties during free-viewing of 4 different categories of images (Natural, Urban, Fractal and Pink Noise). Behaviorally relevant information was estimated in the form of topographical interestingness maps by asking an independent set of subjects to click at image regions that they subjectively found most interesting. Using a Bayesian scheme, we computed saliency functions that described the probability of a given feature to be fixated. In the case of stimulus features, the precise shape of the saliency functions was strongly dependent upon image category and overall the saliency associated with these features was generally weak. When testing multiple features jointly, a linear additive integration model of individual saliencies performed satisfactorily. We found that the saliency associated with interesting locations was much higher than any low-level image feature and any pair-wise combination thereof. Furthermore, the low-level image features were found to be maximally salient at those locations that had already high interestingness ratings. Temporal analysis showed that regions with high interestingness ratings were fixated as early as the third fixation following stimulus onset. Paralleling these findings, fixation durations were found to be dependent mainly on interestingness ratings and to a lesser extent on the low-level image features. Our results suggest that both low- and high-level sources of information play a significant role during exploration of complex scenes with behaviorally relevant information being more effective compared to stimulus features.

Spatial biases in viewing behavior.

Viewing behavior exhibits temporal and spatial structure that is independent of stimulus content and task goals. One example of such structure is horizontal biases, which are likely rooted in left-right asymmetries of the visual and attentional systems. Here, we studied the existence, extent, and mechanisms of this bias. Left- and right-handed subjects explored scenes from different image categories, presented in original and mirrored versions. We also varied the spatial spectral content of the images and the timing of stimulus onset. We found a marked leftward bias at the start of exploration that was independent of image category. This left bias was followed by a weak bias to the right that persisted for several seconds. This asymmetry was found in the majority of right-handers but not in left-handers. Neither low- nor high-pass filtering of the stimuli influenced the bias. This argues against mechanisms related to the hemispheric segregation of global versus local visual processing. Introducing a delay in stimulus onset after offset of a central fixation spot also had no influence. The bias was present even when stimuli were presented continuously and without any requirement to fixate, associated to both fixation- and saccade-contingent image changes. This suggests the bias is not caused by structural asymmetries in fixation control. Instead the pervasive horizontal bias is compatible with known asymmetries of higher-level attentional areas related to the detection of novel events.

Parametric trial-by-trial prediction of pain by easily available physiological measures.

Pain is commonly assessed by subjective reports on rating scales. However, in many experimental and clinical settings, an additional, objective indicator of pain is desirable. In order to identify an objective, parametric signature of pain intensity that is predictive at the individual stimulus level across subjects, we recorded skin conductance and pupil diameter responses to heat pain stimuli of different durations and temperatures in 34 healthy subjects. The temporal profiles of trial-wise physiological responses were characterized by component scores obtained from principal component analysis. These component scores were then used as predictors in a linear regression analysis, resulting in accurate pain predictions for individual trials. Using the temporal information encoded in the principal component scores explained the data better than prediction by a single summary statistic (i.e., maximum amplitude). These results indicate that perceived pain is best reflected by the temporal dynamics of autonomic responses. Application of the regression model to an independent data set of 20 subjects resulted in a very good prediction of the pain ratings demonstrating the generalizability of the identified temporal pattern. Utilizing the readily available temporal information from skin conductance and pupil diameter responses thus allows parametric prediction of pain in human subjects.

Predictions in the light of your own action repertoire as a general computational principle.

We argue that brains generate predictions only within the constraints of the action repertoire. This makes the computational complexity tractable and fosters a step-by-step parallel development of sensory and motor systems. Hence, it is more of a benefit than a literal constraint and may serve as a universal normative principle to understand sensorimotor coupling and interactions with the world.

Cortical long-range interactions embed statistical knowledge of natural sensory input: a voltage-sensitive dye imaging study.

How is contextual processing as demonstrated with simplified stimuli, cortically enacted in response to ecologically relevant complex and dynamic stimuli? Using voltage-sensitive dye imaging, we captured mesoscopic population dynamics across several square millimeters of cat primary visual cortex. By presenting natural movies locally through either one or two adjacent apertures, we show that simultaneous presentation leads to mutual facilitation of activity. These synergistic effects were most effective when both movie patches originated from the same natural movie, thus forming a coherent stimulus in which the inherent spatio-temporal structure of natural movies were preserved in accord with Gestalt principles of perceptual organization. These results suggest that natural sensory input triggers cooperative mechanisms that are imprinted into the cortical functional architecture as early as in primary visual cortex.

Unmasking the contribution of low-level features to the guidance of attention.

The role of low-level stimulus-driven control in the guidance of overt visual attention has been difficult to establish because low- and high-level visual content are spatially correlated within natural visual stimuli. Here we show that impairment of parietal cortical areas, either permanently by a lesion or reversibly by repetitive transcranial magnetic stimulation (rTMS), leads to fixation of locations with higher values of low-level features as compared to control subjects or in a no-rTMS condition. Moreover, this unmasking of stimulus-driven control crucially depends on the intrahemispheric balance between top-down and bottom-up cortical areas. This result suggests that although in normal behavior high-level features might exert a strong influence, low-level features do contribute to guide visual selection during the exploration of complex natural stimuli.

Natural scene evoked population dynamics across cat primary visual cortex captured with voltage-sensitive dye imaging.

Neurons in primary visual cortex have been characterized by their selectivity to orientation, spatiotemporal frequencies, and motion direction, among others all essential parameters to decompose complex image structure. However, their concerted functioning upon real-world visual dynamics remained unobserved since most studies tested these parameters in isolation rather than in rich mixture. We used voltage-sensitive dye imaging to characterize population responses to natural scene movies, and for comparison, to well-established moving gratings. For the latter, we confirm previous observations of a deceleration/acceleration notch. Upon stimulation with natural movies, however, a subsequent acceleration component was almost absent. Furthermore, we found that natural stimuli revealed sparsely distributed nonseparable space-time dynamics, continuously modulated by movie motion. Net excitation levels detected with gratings were reached only rarely with natural movies. Emphasizing this observation, across the entire time course, both average and peak amplitudes were lower than nonspecific, that is, minimum, activity obtained for gratings. We estimated a necessary increase of ∼30% of movie contrast to match high grating activity levels. Our results suggest that in contrast to gratings, processing of complex natural input is based on a balanced and stationary interplay between excitation and inhibition and point to the importance of suppressive mechanisms in shaping the operating regime of cortical dynamics.

Independent encoding of grating motion across stationary feature maps in primary visual cortex visualized with voltage-sensitive dye imaging.

In early visual cortex different stimulus parameters are represented in overlaid feature maps. Such functioning was extensively explored by the use of drifting gratings characterized by orientation, spatial-temporal frequency, and direction of motion. However surprisingly, the direct cortical visuotopic drift of the gratings' stripy pattern has never been detected simultaneously to these stationary feature maps. It therefore remains to be demonstrated how physical signals of grating motion across the cortex are represented independently of other parametric maps and thus, how multi-dimensional input is processed independently to enable effective read-out further downstream. Taking advantage of the high spatial and temporal resolution of voltage-sensitive dye imaging, we here show the real-time encoding of position and orientation. By decomposing the cortical responses to drifting gratings we visualize the typical emergence of stationary orientation maps in which specific domains exhibited highest amplitudes. Simultaneously to these patchy maps, we demonstrate coherently propagating waves of activity that precisely matched the actual movement of the gratings in space and time, most dominantly for spatial frequencies lower than the preferred range. Thus, the primary visual cortex multiplexes information about retinotopic motion by additional temporal modulation of stationary orientation signals. These signals may be used to variably extract coarse-grained object motion and form information at higher visual processing stages.

Developmental Changes in Natural Viewing Behavior: Bottom-Up and Top-Down Differences between Children, Young Adults and Older Adults.

Despite the growing interest in fixation selection under natural conditions, there is a major gap in the literature concerning its developmental aspects. Early in life, bottom-up processes, such as local image feature - color, luminance contrast etc. - guided viewing, might be prominent but later overshadowed by more top-down processing. Moreover, with decline in visual functioning in old age, bottom-up processing is known to suffer. Here we recorded eye movements of 7- to 9-year-old children, 19- to 27-year-old adults, and older adults above 72 years of age while they viewed natural and complex images before performing a patch-recognition task. Task performance displayed the classical inverted U-shape, with young adults outperforming the other age groups. Fixation discrimination performance of local feature values dropped with age. Whereas children displayed the highest feature values at fixated points, suggesting a bottom-up mechanism, older adult viewing behavior was less feature-dependent, reminiscent of a top-down strategy. Importantly, we observed a double dissociation between children and elderly regarding the effects of active viewing on feature-related viewing: Explorativeness correlated with feature-related viewing negatively in young age, and positively in older adults. The results indicate that, with age, bottom-up fixation selection loses strength and/or the role of top-down processes becomes more important. Older adults who increase their feature-related viewing by being more explorative make use of this low-level information and perform better in the task. The present study thus reveals an important developmental change in natural and task-guided viewing.

Saliency on a natural scene background: effects of color and luminance contrast add linearly.

In natural vision, shifts in spatial attention are associated with shifts of gaze. Computational models of such overt attention typically use the concept of a saliency map: Normalized maps of center-surround differences are computed for individual stimulus features and added linearly to obtain the saliency map. Although the predictions of such models correlate with fixated locations better than chance, their mechanistic assumptions are less well investigated. Here, we tested one key assumption: Do the effects of different features add linearly or according to a max-type of interaction? We measured the eye position of observers viewing natural stimuli whose luminance contrast and/or color contrast (saturation) increased gradually toward one side. We found that these feature gradients biased fixations toward regions of high contrasts. When two contrast gradients (color and luminance) were superimposed, linear summation of their individual effects predicted their combined effect. This demonstrated that the interaction of color and luminance contrast with respect to human overt attention is--irrespective of the precise model--consistent with the assumption of linearity, but not with a max-type interaction of these features.

Visual stimulus locking of EEG is modulated by temporal congruency of auditory stimuli.

Disparate sensory streams originating from a common underlying event share similar dynamics, and this plays an important part in multisensory integration. Here we investigate audiovisual binding by presenting continuously changing, temporally congruent and incongruent stimuli. Recorded EEG signals are used to quantify spectrotemporal and waveform locking of neural activity to stimulus dynamics. Spectrotemporal analysis reveals locking to visual stimulus dynamics in both a broad alpha and the beta band. The properties of these effects suggest they are a correlate of bottom-up processing in the visual system. Waveform locking reveals two cortically distinct processes that lock to visual stimulus dynamics with differing topographies and time lags relative to the stimuli. Most importantly, these are modulated in strength by the congruency of an accompanying auditory stream. In addition, the waveform locking found at occipital electrodes shows an increase over stimulus duration for visual and congruent audiovisual stimuli. Hence we argue that these effects reflect audiovisual interaction. We thus propose that spectrotemporal and waveform locking reflect different mechanisms involved in the processing of dynamic audiovisual stimuli.

Effects of luminance contrast and its modifications on fixation behavior during free viewing of images from different categories.

During viewing of natural scenes, do low-level features guide attention, and if so, does this depend on higher-level features? To answer these questions, we studied the image category dependence of low-level feature modification effects. Subjects fixated contrast-modified regions often in natural scene images, while smaller but significant effects were observed for urban scenes and faces. Surprisingly, modifications in fractal images did not influence fixations. Further analysis revealed an inverse relationship between modification effects and higher-level, phase-dependent image features. We suggest that high- and mid-level features--such as edges, symmetries, and recursive patterns--guide attention if present. However, if the scene lacks such diagnostic properties, low-level features prevail. We posit a hierarchical framework, which combines aspects of bottom-up and top-down theories and is compatible with our data.