Multimodal semantic revision during inferential processing: The role of inhibitory control in text and picture comprehension.

Although language comprehension usually requires multimodal information, no study to date has investigated how comprehenders deal with the revision of a text's interpretation when different modalities are involved. Twenty-four young adults listened to a story prompting an inference (e.g., polar bear), and then saw a picture that was either consistent (polar bear) or inconsistent but still plausible (penguin). Larger negativity (N400) in the inconsistent picture indicated successful inferential monitoring. Subsequently, a sentence carried the disambiguating word which was either expected ("bear") or unexpected ("penguin") in relation to the auditory-verbal information. Larger negativity in the unexpected word coming from the consistent picture suggested that comprehenders had difficulties selecting the unexpected concept when previous information was contradictory. More importantly, this effect was modulated by inhibitory control, where a higher resistance to distractor interference (flanker task) was associated with a better ability to suppress pictorial information, therefore preventing semantic competition. Similarly, accuracy measured in a final comprehension question demonstrated that higher inhibitory control was related to a more efficient ability to revise the situation model across modalities. Our findings speak to a relationship between story comprehension and mental flexibility during multimodal processing.

How structure sculpts function: Unveiling the contribution of anatomical connectivity to the brain's spontaneous correlation structure.

Intrinsic brain activity is characterized by highly organized co-activations between different regions, forming clustered spatial patterns referred to as resting-state networks. The observed co-activation patterns are sustained by the intricate fabric of millions of interconnected neurons constituting the brain's wiring diagram. However, as for other real networks, the relationship between the connectional structure and the emergent collective dynamics still evades complete understanding. Here, we show that it is possible to estimate the expected pair-wise correlations that a network tends to generate thanks to the underlying path structure. We start from the assumption that in order for two nodes to exhibit correlated activity, they must be exposed to similar input patterns from the entire network. We then acknowledge that information rarely spreads only along a unique route but rather travels along all possible paths. In real networks, the strength of local perturbations tends to decay as they propagate away from the sources, leading to a progressive attenuation of the original information content and, thus, of their influence. Accordingly, we define a novel graph measure, topological similarity, which quantifies the propensity of two nodes to dynamically correlate as a function of the resemblance of the overall influences they are expected to receive due to the underlying structure of the network. Applied to the human brain, we find that the similarity of whole-network inputs, estimated from the topology of the anatomical connectome, plays an important role in sculpting the backbone pattern of time-average correlations observed at rest.

A binocular rivalry study of motion perception in the human brain.

The relationship between brain activity and conscious visual experience is central to our understanding of the neural mechanisms underlying perception. Binocular rivalry, where monocular stimuli compete for perceptual dominance, has been previously used to dissociate the constant stimulus from the varying percept. We report here fMRI results from humans experiencing binocular rivalry under a dichoptic stimulation paradigm that consisted of two drifting random dot patterns with different motion coherence. Each pattern had also a different color, which both enhanced rivalry and was used for reporting which of the two patterns was visible at each time. As the perception of the subjects alternated between coherent motion and motion noise, we examined the effect that these alternations had on the strength of the MR signal throughout the brain. Our results demonstrate that motion perception is able to modulate the activity of several of the visual areas which are known to be involved in motion processing. More specifically, in addition to area V5 which showed the strongest modulation, a higher activity during the perception of motion than during the perception of noise was also clearly observed in areas V3A and LOC, and less so in area V3. In previous studies, these areas had been selectively activated by motion stimuli but whether their activity reflects motion perception or not remained unclear; here we show that they are involved in motion perception as well. The present findings therefore suggest a lack of a clear distinction between 'processing' versus 'perceptual' areas in the brain, but rather that the areas involved in the processing of a specific visual attribute are also part of the neuronal network that is collectively responsible for its perceptual representation.

Representation of perceived object shape by the human lateral occipital complex.

The human lateral occipital complex (LOC) has been implicated in object recognition, but it is unknown whether this region represents low-level image features or perceived object shape. We used an event-related functional magnetic resonance imaging adaptation paradigm in which the response to pairs of successively presented stimuli is lower when they are identical than when they are different. Adaptation across a change between the two stimuli in a pair provides evidence for a common neural representation invariant to that change. We found adaptation in the LOC when perceived shape was identical but contours differed, but not when contours were identical but perceived shape differed. These data indicate that the LOC represents not simple image features, but rather higher level shape information.

Visual representation of malleable and rigid objects that deform as they rotate.

Most studies and theories of object recognition have addressed the perception of rigid objects. Yet, physical objects may also move in a nonrigid manner. A series of priming studies examined the conditions under which observers can recognize novel views of objects moving nonrigidly. Observers were primed with 2 views of a rotating object that were linked by apparent motion or presented statically. The apparent malleability of the rotating prime object varied such that the object appeared to be either malleable or rigid. Novel deformed views of malleable objects were primed when falling within the object's motion path. Priming patterns were significantly more restricted for deformed views of rigid objects. These results suggest that moving malleable objects may be represented as continuous events, whereas rigid objects may not. That is, object representations may be "dynamically remapped" during the analysis of the object's motion.

The lateral occipital complex and its role in object recognition.

Here we review recent findings that reveal the functional properties of extra-striate regions in the human visual cortex that are involved in the representation and perception of objects. We characterize both the invariant and non-invariant properties of these regions and we discuss the correlation between activation of these regions and recognition. Overall, these results indicate that the lateral occipital complex plays an important role in human object recognition.

Surface segmentation cues influence negative priming for novel and familiar shapes.

In a series of experiments, a negative priming paradigm was used to determine how the visual system represents novel shapes under conditions of inattention. Observers in a shape-matching task viewed overlapping shapes with or without surface segmentation cues. Positive priming occurred with opaque and transparent surface-like shapes, whereas negative priming was found with outlined and transparent shapes that lacked surface segmentation cues. This effect generalized to familiar shapes. These results support the importance of segmentation cues in negative priming and suggest that, under otherwise identical conditions, surface segmentation processes can determine whether positive or negative priming occurs in an implicit memory task. Thus, selective attention for overlapping shapes may be best understood in relation to surface segmentation processes.

Activation in human MT/MST by static images with implied motion.

A still photograph of an object in motion may convey dynamic information about the position of the object immediately before and after the photograph was taken (implied motion). Medial temporal/medial superior temporal cortex (MT/MST) is one of the main brain regions engaged in the perceptual analysis of visual motion. In two experiments we examined whether MT/MST is also involved in representing implied motion from static images. We found stronger functional magnetic resonance imaging (fMRI) activation within MT/MST during viewing of static photographs with implied motion compared to viewing of photographs without implied motion. These results suggest that brain regions involved in the visual analysis of motion are also engaged in processing implied dynamic information from static images.

Cortical regions involved in perceiving object shape.

The studies described here use functional magnetic resonance imaging to test whether common or distinct cognitive and/or neural mechanisms are involved in extracting object structure from the different image cues defining an object's shape, such as contours, shading, and monocular depth cues. We found overlapping activations in the lateral and ventral occipital cortex [known as the lateral occipital complex (LOC)] for objects defined by different visual cues (e.g., grayscale photographs and line drawings) when each was compared with its own scrambled-object control. In a second experiment we found a reduced response when objects were repeated, independent of whether they appeared in the same or a different format (i.e., grayscale images vs line drawings). A third experiment showed that activation in the LOC was no stronger for three-dimensional shapes defined by contours or monocular depth cues, such as occlusion, than for two-dimensional shapes, suggesting that these regions are not selectively involved in processing three-dimensional shape information. These results suggest that common regions in the LOC are involved in extracting and/or representing information about object structure from different image cues.

The visual representation of three-dimensional, rotating objects.

Depth rotations can reveal new object parts and result in poor recognition of "static" objects (Biederman & Gerhardstein, 1993). Recent studies have suggested that multiple object views can be associated through temporal contiguity and similarity (Edelman & Weinshall, 1991; Lawson, Humphreys & Watson, 1994; Wallis, 1996). Motion may also play an important role in object recognition since observers recognize novel views of objects rotating in the picture plane more readily than novel views of statically re-oriented objects (Kourtzi & Shiffrar, 1997). The series of experiments presented here investigated how different views of a depth-rotated object might be linked together even when these views do not share the same parts. The results suggest that depth rotated object views can be linked more readily with motion than with temporal sequence alone to yield priming of novel views of 3D objects that fall in between "known" views. Motion can also enhance path specific view linkage when visible object parts differ across views. Such results suggest that object representations depend on motion processes.

Dynamic representations of human body movement.

Psychophysical and neurophysiological studies suggest that human body motions can be readily recognized. Human bodies are highly articulated and can move in a nonrigid manner. As a result, we perceive highly dissimilar views of the human form in motion. How does the visual system integrate multiple views of a human body in motion so that we can perceive human movement as a continuous event? The results of a set of priming experiments suggest that motion can readily facilitate the linkage of different views of a moving human. Positive priming was found for novel views of a human body that fell within the path of human movement. However, no priming was observed for novel views outside the path of motion. Furthermore, priming was restricted to those views that satisfied the biomechanical constraints of human movement. These results suggest that visual representation of human movement may be based upon the movement limitations of the human body and may reflect a dynamic interaction of motion and object-recognition processes.

The integration of object properties.

Is a complex object learned as a whole from the onset of learning, or is property recall fragmented and only integrated with repeated experience? The authors studied the recall of 3-property visual objects and found that even when property recall was low, recall coherence was high. Properties not organized as visual objects did not show high recall coherence. In addition, the authors failed to find evidence that spatial location, spatial proximity, spatial distinctiveness, distinctive motion, or perceptual grouping induced by common motion affected the integration of properties. Unitariness, temporal contiguity, and intention, however, were factors that influenced memory coherence. The authors concluded that integration occurs when properties are encoded as the constituents of a cognitive structure.