Neural activity underlying the detection of an object movement by an observer during forward self-motion: Dynamic decoding and temporal evolution of directional cortical connectivity.

Relatively little is known about how the human brain identifies movement of objects while the observer is also moving in the environment. This is, ecologically, one of the most fundamental motion processing problems, critical for survival. To study this problem, we used a task which involved nine textured spheres moving in depth, eight simulating the observer's forward motion while the ninth, the target, moved independently with a different speed towards or away from the observer. Capitalizing on the high temporal resolution of magnetoencephalography (MEG) we trained a Support Vector Classifier (SVC) using the sensor-level data to identify correct and incorrect responses. Using the same MEG data, we addressed the dynamics of cortical processes involved in the detection of the independently moving object and investigated whether we could obtain confirmatory evidence for the brain activity patterns used by the classifier. Our findings indicate that response correctness could be reliably predicted by the SVC, with the highest accuracy during the blank period after motion and preceding the response. The spatial distribution of the areas critical for the correct prediction was similar but not exclusive to areas underlying the evoked activity. Importantly, SVC identified frontal areas otherwise not detected with evoked activity that seem to be important for the successful performance in the task. Dynamic connectivity further supported the involvement of frontal and occipital-temporal areas during the task periods. This is the first study to dynamically map cortical areas using a fully data-driven approach in order to investigate the neural mechanisms involved in the detection of moving objects during observer's self-motion.

Cross-Modal Cue Effects in Motion Processing.

The everyday environment brings to our sensory systems competing inputs from different modalities. The ability to filter these multisensory inputs in order to identify and efficiently utilize useful spatial cues is necessary to detect and process the relevant information. In the present study, we investigate how feature-based attention affects the detection of motion across sensory modalities. We were interested to determine how subjects use intramodal, cross-modal auditory, and combined audiovisual motion cues to attend to specific visual motion signals. The results showed that in most cases, both the visual and the auditory cues enhance feature-based orienting to a transparent visual motion pattern presented among distractor motion patterns. Whereas previous studies have shown cross-modal effects of spatial attention, our results demonstrate a spread of cross-modal feature-based attention cues, which have been matched for the detection threshold of the visual target. These effects were very robust in comparisons of the effects of valid vs. invalid cues, as well as in comparisons between cued and uncued valid trials. The effect of intramodal visual, cross-modal auditory, and bimodal cues also increased as a function of motion-cue salience. Our results suggest that orienting to visual motion patterns among distracters can be facilitated not only by intramodal priors, but also by feature-based cross-modal information from the auditory system.

Using action understanding to understand the left inferior parietal cortex in the human brain.

Humans have a sophisticated knowledge of the actions that can be performed with objects. In an fMRI study we tried to establish whether this depends on areas that are homologous with the inferior parietal cortex (area PFG) in macaque monkeys. Cells have been described in area PFG that discharge differentially depending upon whether the observer sees an object being brought to the mouth or put in a container. In our study the observers saw videos in which the use of different objects was demonstrated in pantomime; and after viewing the videos, the subject had to pick the object that was appropriate to the pantomime. We found a cluster of activated voxels in parietal areas PFop and PFt and this cluster was greater in the left hemisphere than in the right. We suggest a mechanism that could account for this asymmetry, relate our results to handedness and suggest that they shed light on the human syndrome of apraxia. Finally, we suggest that during the evolution of the hominids, this same pantomime mechanism could have been used to 'name' or request objects.

Interaction of cortical networks mediating object motion detection by moving observers.

The task of parceling perceived visual motion into self- and object motion components is critical to safe and accurate visually guided navigation. In this paper, we used functional magnetic resonance imaging to determine the cortical areas functionally active in this task and the pattern connectivity among them to investigate the cortical regions of interest and networks that allow subjects to detect object motion separately from induced self-motion. Subjects were presented with nine textured objects during simulated forward self-motion and were asked to identify the target object, which had an additional, independent motion component toward or away from the observer. Cortical activation was distributed among occipital, intra-parietal and fronto-parietal areas. We performed a network analysis of connectivity data derived from partial correlation and multivariate Granger causality analyses among functionally active areas. This revealed four coarsely separated network clusters: bilateral V1 and V2; visually responsive occipito-temporal areas, including bilateral LO, V3A, KO (V3B) and hMT; bilateral VIP, DIPSM and right precuneus; and a cluster of higher, primarily left hemispheric regions, including the central sulcus, post-, pre- and sub-central sulci, pre-central gyrus, and FEF. We suggest that the visually responsive networks are involved in forming the representation of the visual stimulus, while the higher, left hemisphere cluster is involved in mediating the interpretation of the stimulus for action. Our main focus was on the relationships of activations during our task among the visually responsive areas. To determine the properties of the mechanism corresponding to the visual processing networks, we compared subjects' psychophysical performance to a model of object motion detection based solely on relative motion among objects and found that it was inconsistent with observer performance. Our results support the use of scene context (e.g., eccentricity, depth) in the detection of object motion. We suggest that the cortical activation and visually responsive networks provide a potential substrate for this computation.

Acoustic facilitation of object movement detection during self-motion.

In humans, as well as most animal species, perception of object motion is critical to successful interaction with the surrounding environment. Yet, as the observer also moves, the retinal projections of the various motion components add to each other and extracting accurate object motion becomes computationally challenging. Recent psychophysical studies have demonstrated that observers use a flow-parsing mechanism to estimate and subtract self-motion from the optic flow field. We investigated whether concurrent acoustic cues for motion can facilitate visual flow parsing, thereby enhancing the detection of moving objects during simulated self-motion. Participants identified an object (the target) that moved either forward or backward within a visual scene containing nine identical textured objects simulating forward observer translation. We found that spatially co-localized, directionally congruent, moving auditory stimuli enhanced object motion detection. Interestingly, subjects who performed poorly on the visual-only task benefited more from the addition of moving auditory stimuli. When auditory stimuli were not co-localized to the visual target, improvements in detection rates were weak. Taken together, these results suggest that parsing object motion from self-motion-induced optic flow can operate on multisensory object representations.

Visual deficits in a patient with 'kaleidoscopic disintegration of the visual world'.

We describe psychophysical, neuropsychological and neuro-ophthalmological studies of visual abilities in a patient who, following a right hemisphere stroke, had difficulty in combining parts of objects into a whole and in reading. Strikingly, her perceptual problems were accentuated when the objects moved or when she moved. Formal testing showed that her main deficits were in depth perception, various tasks of motion and object recognition of degraded stimuli. But low-level detection and discrimination of form and color were normal. Despite her deficits in visual motion and degraded static-object recognition, her visual recognition of 'biological motion' stimuli was normal. Structural magnetic resonance imaging revealed an infarct in the ventro-medial occipito-temporal region, extending ventro-laterally and leading to a 'kaleidoscopic disintegration of visible objects'.

Functional neuroanatomy of biological motion perception in humans.

We used whole brain functional MRI to investigate the neural network specifically engaged in the recognition of "biological motion" defined by point-lights attached to the major joints and head of a human walker. To examine the specificity of brain regions responsive to biological motion, brain activations obtained during a "walker vs. non-walker" discrimination task were compared with those elicited by two other tasks: (i) non-rigid motion (NRM), involving the discrimination of overall motion direction in the same "point-lights" display, and (ii) face-gender discrimination, involving the discrimination of gender in briefly presented photographs of men and women. Brain activity specific to "biological motion" recognition arose in the lateral cerebellum and in a region in the lateral occipital cortex presumably corresponding to the area KO previously shown to be particularly sensitive to kinetic contours. Additional areas significantly activated during the biological motion recognition task involved both, dorsal and ventral extrastriate cortical regions. In the ventral regions both face-gender discrimination and biological motion recognition elicited activation in the lingual and fusiform gyri and in the Brodmann areas 22 and 38 in superior temporal sulcus (STS). Along the dorsal pathway, both biological motion recognition and non-rigid direction discrimination gave rise to strong responses in several known motion sensitive areas. These included Brodmann areas 19/37, the inferior (Brodmann Area 39), and superior parietal lobule (Brodmann Area 7). Thus, we conjecture that, whereas face (and form) stimuli activate primarily the ventral system and motion stimuli primarily the dorsal system, recognition of biological motion stimuli may activate both systems as well as their confluence in STS. This hypothesis is consistent with our findings in stroke patients, with unilateral brain lesions involving at least one of these areas, who, although correctly reporting the direction of the point-light walker, fail on the biological motion task.

A laterally interconnected neural architecture in MST accounts for psychophysical discrimination of complex motion patterns.

The complex patterns of visual motion formed across the retina during self-motion, often referred to as optic flow, provide a rich source of information describing our dynamic relationship within the environment. Psychophysical studies indicate the existence of specialized detectors for component motion patterns (radial, circular, planar) that are consistent with the visual motion properties of cells in the medial superior temporal area (MST) of nonhuman primates. Here we use computational modeling and psychophysics to investigate the structural and functional role of these specialized detectors in performing a graded motion pattern (GMP) discrimination task. In the psychophysical task perceptual discrimination varied significantly with the type of motion pattern presented, suggesting perceptual correlates to the preferred motion bias reported in MST. Simulated perceptual discrimination in a population of independent MST-like neural responses showed inconsistent psychophysical performance that varied as a function of the visual motion properties within the population code. Robust psychophysical performance was achieved by fully interconnecting neural populations such that they inhibited nonpreferred units. Taken together, these results suggest that robust processing of the complex motion patterns associated with self-motion and optic flow may be mediated by an inhibitory structure of neural interactions in MST.

Regional cerebral correlates of global motion perception: evidence from unilateral cerebral brain damage.

We used a psychophysical task to measure sensitivity to motion direction in 50 stroke patients with unilateral brain lesions and 85 control subjects. Subjects were asked to discriminate the overall direction of motion in dynamic stochastic random dot displays in which only a variable proportion of the spots moved in a single direction while the remainder moved randomly. Behavioural and neurophysiological evidence shows that the middle temporal (MT/V5) and middle superior temporal (MST) areas in the macaque monkey are indispensably involved in the perception of this type of motion. In human subjects too, lesions in the same region disrupt performance on this task. Here we assessed more extensively the correlation between direction sensitivity for global motion and the anatomical locus of the lesion. Thresholds for perceiving the direction of global motion were impaired in the visual field contralateral to the lesion in patients with lesions in the occipitoparietal and parietotemporal areas involving the human analogue of areas MT/V5 and MST, but not by lesions in the occipito-temporal or anterior frontal areas. Patients with lesions involving the anterior temporal or parietal lobes displayed poor performance for stimuli presented in either visual field, which is consistent with the large and bilateral receptive fields in these areas in monkeys. The perception of global motion was also more impaired in the centripetal than the centrifugal direction in the hemifield contralateral to the MT/V5 lesion. Surprisingly, thresholds were normal in all patients when the displays contained static but not dynamic visual noise, suggesting that their deficit reflects an inability to filter out dynamic noise. Although frequent repeated testing of some patients whose lesion involved the human homologue of MT was accompanied by an improvement in performance, this was no greater than in other patients who received training on different motion tasks.

A lesion of cortical area V2 selectively impairs the perception of the direction of first-order visual motion.

Lesions of area MT/V5 in monkeys and its presumed homologue, the motion area, in humans impair motion perception, including the discrimination of the direction of global motion in random dot kinematograms. Here we report the results of similar tests on patient TF, who has a discrete and very small, unilateral infarct in the medial superior part of the right occipital cortex. Structural MRI, co-registered in software with a standardized human brain atlas, reveals that the lesion involves area V2. The patient was impaired in his retinotopically corresponding left lower quadrant on several motion tasks including discrimination in random dot kinematograms of direction, speed and motion-defined discontinuity. He was also impaired on tasks selectively involving first-order motion based on luminance contrast but not on second-order motion based on texture contrast. The results show that even though area MT/V5 is intact, motion perception is abnormal and, in particular, his perception of first-order motion is impaired.

Blindness to form from motion despite intact static form perception and motion detection.

We studied the motion perception, including form and meaning generated by motion, in a hemianopic patient who also had visual perceptual impairments in her seeing hemifield as a result of a lesion in ventral extrastriate cortex. She was unable to recognise 2- or 3-dimensional forms, and even borders, generated by motion alone, failed to recognise mimed actions or the Johannson 'biological motion' display, and ceased to recognise people well-known to her when they moved. Her performance with static displays, although impaired, could not explain her inability to perceive shape or derive meaning from moving displays. Unlike a motion-blind patient, she can still see and describe the motion, with the exception of second-order motion, but not what it creates or represents.

Anomalous perception of coherence and transparency in moving plaid patterns.

While the low-level processes mediating the detection of primary visual attributes are well understood, much less is known about the way in which these attributes are assigned to objects in the visual world. For example, when a region of the retinal image contains multiple motion signals at a range of spatial scales, how do we know whether these signals come from a single object or multiple objects? Here, we present data from four neurological patients on a psychophysical task requiring them to report whether the two components of a plaid pattern appear to move coherently or transparently. The spatial frequency of one component of the plaid is held constant while that of the other is manipulated. While some of the patients perceive coherent motion over a much smaller range of spatial frequencies than normal controls, others report coherence over almost the entire range tested. We discuss the implications of these findings for computational theories of motion perception and higher-level visual processing.

The perception and discrimination of speed in complex motion.

Random dot kinematograms were used to simulate radial, rotational and spiral optic flow. The stimuli were designed so that, while dot speed increased linearly with distance from the centre of the display, the density of dots remained uniform throughout their presentation. In two experiments, subjects were required to perform a temporal 2AFC speed discrimination task. Experiment 1 measured the perceived speed of a range of optic flow patterns against a rotational comparison stimulus. Radial motions were found to appear faster than rotations by approximately 10%, with a smaller but significant effect for spirals. Experiment 2 measured discrimination thresholds for pairs of similar optic flow stimuli identical in all respects except mean speed. No consistent differences were observed between the speed discrimination thresholds of radial, rotational and spiral motions and a control stimulus with the same speed profile in which motion followed fixed random trajectories. The perceived speed results are interpreted in terms of a model satisfying constraints on motion-in-depth and object rigidity, while speed discrimination appears to be based upon the pooled responses of elementary motion detectors.

A computational model of selective deficits in first and second-order motion processing.

Recent neurological studies of selective impairments in first and second-order motion processing are of considerable relevance in elucidating the mechanisms of motion perception in normal human observers. We examine the stimuli which have been used to assess first and second-order motion processing capabilities in clinical subjects, and discuss the nature of the computations necessary to extract their motion. We find that a simple computational model of first and second-order motion processing is able to account for the data. The model consists of a first-order channel computing motion at coarse and fine scales, and a coarse scale second-order channel. The second-order channel is sensitive to motion information defined by variations in luminance, contrast, spatial frequency and flicker. When elements of the model are disabled, its performance on either first or second-order motion can be selectively impaired in line with the neurological data.

Perception of first- and second-order motion: separable neurological mechanisms?

An unresolved issue in visual motion perception is how distinct are the processes underlying "first-order" and "second-order" motion. The former is defined by spatiotemporal variations of luminance and the latter by spatiotemporal variations in other image attributes, such as contrast or depth. Here we describe two neurological patients with focal unilateral lesions whose contrasting perceptual deficits on psychophysical tasks of "first-order" and "second-order" motion are related to the maps of the human brain established by functional neuroimaging and gross anatomical features. We used a relatively fine-grained neocortical parcellation method applied to high-resolution MRI scans of the patients' brains to illustrate a subtle, yet highly specific dissociation in the visual motion system in humans. Our results suggest that the two motion systems are mediated by regionally separate mechanisms from an early stage of cortical processing.

Large receptive fields for optic flow detection in humans.

We used a psychophysical summation technique to study the properties of detectors tuned to radial, circular and translational motion, and to determine the spatial extent of their receptive fields. Signal-to-noise motion thresholds were measured for patterns curtailed spatially in various ways. Sensitivity for radial, circular and translational motion increased with stimulus area at a rate predicted by an ideal integrator. When sectors of noise were added to the stimulus, sensitivity decreased at a rate consistent with an ideal integrator. Summation was tested for large annular stimuli, and shown to hold up to 70 degrees in some cases, suggesting very large receptive fields for this type of motion (consistent with the physiology of neurones in the dorsal region of the medial superior temporal area (MSTd)). This is a far greater area than observed for summation of contrast sensitivity to gratings (Anderson SJ and Burr DC, Vis Res 1987;29:621-635, and to this type of stimuli (Morrone MC, Burr DC and Vaina LM, Nature 1995;376:507-509, consistent with the suggestion that the two techniques examine different levels of motion analysis.

Neural systems underlying learning and representation of global motion.

We demonstrate performance-related changes in cortical and cerebellar activity. The largest learning-dependent changes were observed in the anterior lateral cerebellum, where the extent and intensity of activation correlated inversely with psychophysical performance. After learning had occurred (a few minutes), the cerebellar activation almost disappeared; however, it was restored when the subjects were presented with a novel, untrained direction of motion for which psychophysical performance also reverted to chance level. Similar reductions in the extent and intensity of brain activations in relation to learning occurred in the superior colliculus, anterior cingulate, and parts of the extrastriate cortex. The motion direction-sensitive middle temporal visual complex was a notable exception, where there was an expansion of the cortical territory activated by the trained stimulus. Together, these results indicate that the learning and representation of visual motion discrimination are mediated by different, but probably interacting, neuronal subsystems.

Complex motion perception and its deficits.

Within the hierarchy of motion perception, the dorsolateral middle superior temporal area (MSTd) is optimally suited for the analysis of the complex motion patterns that are directly useful for visually guided behaviour (e.g. computation of heading). Recent electrophysiological and psychophysical evidence suggests the existence of 'detectors' in MSTd that are specialised for complex motion patterns and advocates the necessity of combining retinal and extraretinal signals received by MSTd neurones for the accurate perception of heading. In some neurological patients, of which only a small number have been reported to date, lesions involving the human homologue of MST have devastating effects on their ability to navigate in their surroundings. It has been reported that these patients have impaired performance of psychophysical tasks of complex motion discrimination.

First- and second-order motion perception in Gabor micropattern stimuli: psychophysics and computational modelling.

This paper examines the perception of first- and second-order motion in human vision. In an extension of previous work by Boulton and Baker [J.B. Boulton, C.L. Baker, Motion detection is dependent on spatial frequency not size, Vision Res., 31 (1991) 77-87; J.B. Boulton, C.L. Baker, Different parameters control motion perception above and below a critical density, Vision Res., 33 (1993) 1803-1811], the direction of two-frame apparent motion is measured for stimuli composed of Gabor or Gaussian micropatterns. Three conditions are investigated. Condition 1 is that used by Boulton and Baker, in which motion is defined by the displacement of Gabor micropatterns. In condition 2, motion is defined by the displacement of Gaussian micropatterns. In condition 3, the envelopes of Gabor micropatterns are displaced while their carriers remain static. Using sparsely distributed micropatterns, direction judgements in all three conditions are determined by the spacing of the micropatterns. With a dense stimulus, direction judgements vary as a function of displacement in qualitatively different ways for the three conditions. The psychophysical results are predicted by a two-channel computational model. In one channel, motion is calculated directly from stimulus luminance, while in the other it is preceded by a texture-grabbing operation. The relative activities of the two channels dictates which governs direction judgements for any given stimulus.