Acting, seeing, and conscious awareness.

We argue that there is a relation between the judgements that 'I did it' and 'I saw it'. Both are statements are about the individual, not just the world. We show that the dorsal prefrontal cortex is activated both when human subjects judge that they are the agents of their actions and when they judge that they are confident that they have seen a masked visual stimulus. Macaque monkeys have also been taught to report whether they have or have not seen visual stimuli and cells can be found in the dorsal prefrontal cortex that distinguish between 'seen' and 'not seen'. The judgement is abstract in that it applies largely irrespective of the nature and location of the stimulus. We suggest that the reason why the prefrontal cortex is involved is that it evolved in primates, adapted by searching for fruit and leaves and using their hands to retrieve them. There is cell activity in the dorsal prefrontal cortex that relates to eye movements, covert attention and visual search; activity that relates to learning abstract rules; and activity that relates to the planning of the hand movements that are appropriate. We propose that this is the reason why this area is involved in making judgements about both agency and visual detection.

Using action understanding to understand the left inferior parietal cortex in the human brain.

Humans have a sophisticated knowledge of the actions that can be performed with objects. In an fMRI study we tried to establish whether this depends on areas that are homologous with the inferior parietal cortex (area PFG) in macaque monkeys. Cells have been described in area PFG that discharge differentially depending upon whether the observer sees an object being brought to the mouth or put in a container. In our study the observers saw videos in which the use of different objects was demonstrated in pantomime; and after viewing the videos, the subject had to pick the object that was appropriate to the pantomime. We found a cluster of activated voxels in parietal areas PFop and PFt and this cluster was greater in the left hemisphere than in the right. We suggest a mechanism that could account for this asymmetry, relate our results to handedness and suggest that they shed light on the human syndrome of apraxia. Finally, we suggest that during the evolution of the hominids, this same pantomime mechanism could have been used to 'name' or request objects.

Has brain imaging discovered anything new about how the brain works?

There have now been roughly 130,000 papers on fMRI. While these have clearly contributed to our understanding of the functional anatomy of the human brain, it is less clear that they have changed the way in which we think about the brain. The issue, in other words, is whether they have established new principles about how the brain works. In this paper we offer as an example one new principle, partly to lay down the criteria that are required for establishing a new principle, and partly to encourage others to offer other principles. Our example concerns the flexible flow of information through the cortex that must occur according to the demands of the task or current context. We suggest that this flexibility is achieved by feedback connections from the prefrontal and parietal cortex, and that these include connections to sensory and motor areas. However, the nature of the selective effect differs. The parietal cortex can select both within and across processing streams. By across streams we mean that it can have the same influence on different streams, for example the dorsal and ventral visual systems. However, only the prefrontal cortex can also select between processing streams. The difference between the prefrontal and parietal effects is due to their different positions within the processing hierarchy.

Affective response to one's own moral violations.

Morality depends on a set of cultural rules that regulate interpersonal behaviour and provide a basis for social cohesion. The interpretation of moral transgressions and their affective consequences depends on whether the action is intentional or accidental, and whether one is the agent of or witness to the action. We used event-related functional magnetic resonance imaging (fMRI) to investigate whether the amygdala is involved in judging one's own moral violation of social norms. In this study, participants (n = 12) were asked to make evaluations regarding the degree of inappropriateness of social behaviours described in stories in which they themselves, or someone else, transgressed social norms either intentionally or accidentally. Consistent with our hypothesis, the amygdala was activated when participants considered stories narrating their own intentional transgression of social norms. This result suggests the amygdala is important for affective responsiveness to moral transgressions.

Amygdala activation when one is the target of deceit: did he lie to you or to someone else?

The ability to figure out whether a person is being honest or deceitful is an important part of social competence. Reactions to deceit may however differ depending on whether one is being deceived oneself or observes a deceitful exchange between others. In the present study, we investigated whether personal involvement influenced the neural responses associated with the detection of deceit. Subjects watched videos of actors lifting a box and judged whether the actors had been misled about the real weight of the box. Personal involvement was manipulated by having the participants themselves among the actors. The critical finding was that there was activity in amygdala and fusiform gyrus only for the condition in which participants observed themselves being deceived. In contrast, the superior temporal sulcus and anterior cingulate cortex were activated irrespective of whether the participants detected that the experimenter had deceived themselves or another. These four brain areas are all interconnected and are part of the discrete neural system subserving social cognition. Our results provide direct evidence, using judgments of deceit in a social context, that the crucial factor for amygdala activation is the involvement of the subjects because they are the target of the deceit. We interpret the activation of the amygdala in this situation as reflecting the greater affective reaction when one is deceived oneself. Our results suggest that when one is personally involved, deceit is taken as a potential threat.

Components of attentional set-switching.

A series of distinct event-related potentials (ERPs) have been recorded from the scalp of human subjects as they switch from one task to another. It is possible that task switching may depend on different mechanisms depending on whether the switch requires a change in attentional set, in other words the redirecting of attention to different aspects of a sensory stimulus, or whether it requires a change in intentional set, in others words a change in the way that responses are selected. To address this issue, the current study recorded ERPs while subjects switched between attentional sets and the results were compared with those of a previous investigation in which subjects switched between intentional sets. Subjects selected stimuli according to two conflicting attentional sets, each emphasizing one visual stimulus dimension (colour, shape). Pairs of stimuli, only one of which was to be attended, were presented for between eight and seventeen trials then either a switch or a stay cue was shown. The switch cue instructed subjects to switch from the current attentional set to the other set, while the stay cue instructed subjects to maintain the current set. Comparing ERPs time-locked to the switch and stay cues revealed neural correlates of the initiation of a task switch. Comparing the ERPs time locked to the first stimuli after either stay or switch cues identified neural correlates of the implementation of a task switch. A similar modulation over parietal electrodes was seen when subjects were switching between either attentional or intentional sets. While an intentional set switch began with a medial frontal modulation, attentional set switching began with a lateral frontal modulation. Implementing a new attentional set was associated with modulation of relatively early visual potentials, while implementing a new intentional set was associated with modulation of later response-related potentials. The results confirm that task switching consists of a number of constituent processes which may be taxed to different degrees depending on whether a task-switch paradigm requires subjects to change the way in which they select stimuli or responses.

Action observation and acquired motor skills: an FMRI study with expert dancers.

When we observe someone performing an action, do our brains simulate making that action? Acquired motor skills offer a unique way to test this question, since people differ widely in the actions they have learned to perform. We used functional magnetic resonance imaging to study differences in brain activity between watching an action that one has learned to do and an action that one has not, in order to assess whether the brain processes of action observation are modulated by the expertise and motor repertoire of the observer. Experts in classical ballet, experts in capoeira and inexpert control subjects viewed videos of ballet or capoeira actions. Comparing the brain activity when dancers watched their own dance style versus the other style therefore reveals the influence of motor expertise on action observation. We found greater bilateral activations in premotor cortex and intraparietal sulcus, right superior parietal lobe and left posterior superior temporal sulcus when expert dancers viewed movements that they had been trained to perform compared to movements they had not. Our results show that this 'mirror system' integrates observed actions of others with an individual's personal motor repertoire, and suggest that the human brain understands actions by motor simulation.

Prediction error for free monetary reward in the human prefrontal cortex.

Making predictions about future rewards is an important ability for primates, and its neurophysiological mechanisms have been studied extensively. One important approach is to identify neural systems that process errors related to reward prediction (i.e., areas that register the occurrence of unpredicted rewards and the failure of expected rewards). In monkeys that have learned to predict appetitive rewards during reward-directed behaviors, dopamine neurons reliably signal both types of prediction error. The mechanisms in the human brain involved in processing prediction error for monetary rewards are not well understood. Furthermore, nothing is known of how such systems operate when rewards are not contingent on behavior. We used event-related fMRI to localize responses to both classes of prediction error. Subjects were able to predict a monetary reward or a nonreward on the basis of a prior visual cue. On occasional trials, cue-outcome contingencies were reversed (unpredicted rewards and failure of expected rewards). Subjects were not required to make decisions or actions. We compared each type of prediction error trial with its corresponding control trial in which the same prediction did not fail. Each type of prediction error evoked activity in a distinct frontotemporal circuit. Unexpected reward failure evoked activity in the temporal cortex and frontal pole (area 10). Unpredicted rewards evoked activity in the orbitofrontal cortex, the frontal pole, parahippocampal cortex, and cerebellum. Activity time-locked to prediction errors in frontotemporal circuits suggests that they are involved in encoding the associations between visual cues and monetary rewards in the human brain.

Willed action and attention to the selection of action.

Actions are said to be 'willed' if we consciously pay attention to their selection. It has been suggested that they are associated with activations in the dorsal prefrontal cortex (area 46). However, because previous experiments typically used a 'free selection' paradigm to examine this hypothesis, it is unclear whether the results reflected the attention to the selection of action or the freedom of choice allowed by the tasks. In this experiment, we minimized the difference of working memory demand across task conditions by using novel stimuli in each trial. We found that activation in the dorsal prefrontal cortex on a free selection task was not significantly different from that induced by another task that required attention to the selection of action, although the responses were externally specified. This suggests that the dorsal prefrontal cortex is in fact associated with attention to the selection of action, but does not play a unique role in the generation of internally initiated actions. However, the presupplementary motor area (pre-SMA) may subserve this function as activity in this region was found to be tightly associated with the free selection of responses.

Inferring false beliefs from the actions of oneself and others: an fMRI study.

The ability to make judgments about mental states is critical to social interactions. Simulation theory suggests that the observer covertly mimics the activity of the observed person, leading to shared states of mind between the observer and the person observed. We tested this hypothesis by investigating the neural networks activated while subjects watched videos of themselves and of others lifting a box, and judged the beliefs of the actors about the weight of the box. A parietal premotor circuit was recruited during action perception, and the activity started earlier when making judgments about one's own actions as opposed to those of others. This earlier activity in action-related structures can be explained by simulation theory on the basis that when one observes one's own actions, there is a closer match between the simulated and perceived action than there is when one observes the actions of others. When the observers judged the actions to reflect a false belief, there was activation in the superior temporal sulcus, orbitofrontal, paracingulate cortex and cerebellum. We suggest that this reflects a mismatch between the perceived action and the predicted action's outcomes derived from simulation.

Objects automatically potentiate action: an fMRI study of implicit processing.

Behavioural data have shown that the perception of an object automatically potentiates motor components (affordances) of possible actions toward that object, irrespective of the subject's intention. We carried out an event-related fMRI study to investigate the influence of the intrinsic properties of an object on motor responses which were either compatible or incompatible with the action that the object affords. The subjects performed power or precision grip responses based on the categorization of objects into natural or man-made. The objects were either 'small' (usually grasped with a precision grip) or 'large' (usually grasped with a power grip). As expected, the motor responses were fastest to objects that afforded the same grip (congruent) and slowest to objects that afforded the other grip (incongruent). Imaging revealed activations which covaried with compatibility in the parietal, dorsal premotor and inferior frontal cortex. We suggest that the greater the difference in reaction times between congruent and incongruent trials, the greater the competition between the action afforded by the object and the action specified by the task, and thus the greater the activation within this network.

Activations related to "mirror" and "canonical" neurones in the human brain: an fMRI study.

In the macaque monkey ventral premotor cortex (F5), "canonical neurones" are active when the monkey observes an object and when the monkey grasps that object. In the same area, "mirror neurones" fire both when the monkey observes another monkey grasping an object and when the monkey grasps that object. We used event-related fMRI to investigate where in the human brain activation can be found that reflects both canonical and mirror neuronal activity. There was activation in the intraparietal and ventral limbs of the precentral sulcus when subjects observed objects and when they executed movements in response to the objects (canonical neurones). There was activation in the dorsal premotor cortex, the intraparietal cortex, the parietal operculum (SII), and the superior temporal sulcus when subjects observed gestures (mirror neurones). Finally, activations in the ventral premotor cortex and inferior frontal gyrus (area 44) were found when subjects imitated gestures and executed movements in response to objects. We suggest that in the human brain, the ventral limb of the precentral sulcus may form part of the area designated F5 in the macaque monkey. It is possible that area 44 forms an anterior part of F5, though anatomical studies suggest that it may be a transitional area between the premotor and prefrontal cortices.

The effect of cingulate cortex lesions on task switching and working memory.

Anatomic interconnections between the prefrontal and anterior cingulate cortices suggest that these areas may have similar functions. Here we report the effect of anterior cingulate removal on task switching, error monitoring, and working memory. Neuroimaging studies have implicated the cingulate cortex in all these processes. Six macaques were taught task switching (TS) and delayed alternation (DA) paradigms. TS required switching between two conditional response tasks with mutually incompatible response selection rules. DA required alternation between two identically covered food-well positions. In the first set of experiments, anterior cingulate lesions did not consistently impair TS or DA performance. One animal performed worst on both TS and DA and in this animal the cingulate sulcus lesion was most complete. In the second set of experiments, we confirmed that larger anterior cingulate lesions, which included the sulcus, consistently impaired TS but only led to a mild and equivocal impairment of DA. The TS error pattern, however, did not suggest an impairment of TS per se. The consequence of a cingulate lesion is, therefore, distinct to that of a prefrontal lesion. TS error distribution analyses provided some support for a cingulate role in monitoring responses for errors and subsequent correction but the pattern of reaction time change in TS was also indicative of a failure to sustain attention to the task and the responses being made.

The effect of cingulate lesions on social behaviour and emotion.

Functional and structural neuroimaging of the human cingulate cortex has identified this region with emotion and social cognition and suggested that cingulate pathology may be associated with emotional and social behavioural disturbances. The importance of the cingulate cortex for emotion and social behaviour, however, has not been clear from lesion studies. Bilateral lesions in the cingulate cortex were made in three macaques and their social interactions were compared with those of controls. Subsequently, cingulate lesions were made in the three controls and their behaviour was compared before and after surgery. Cingulate lesions were associated with decreases in social interactions, time spent in proximity with other individuals, and vocalisations but an increase in manipulation of an inanimate object. The results are consistent with a cingulate role in social behaviour and emotion.

The anterior cingulate and reward-guided selection of actions.

Macaques were taught a reward-conditional response selection task; they learned to associate each of two different actions to each of two different rewards and to select actions that were appropriate for particular rewards. They were also taught a visual discrimination learning task. Cingulate lesions significantly impaired selection of responses associated with different rewards but did not interfere with visual discrimination learning or performance. The results suggest that 1) the cingulate cortex is concerned with action reward associations and not limited to just detecting when actions lead to errors and 2) that the cingulate cortex's function is limited to action reinforcer associations and it is not concerned with stimulus reward associations.

Components of switching intentional set.

Despite the intuition that we can shift cognitive set on instruction, some behavioral studies have suggested that set shifting might only be accomplished once we engage in performance of the new task. It is possible that set switching consists of more than one component cognitive process and that the component processes might segregated in time. We recorded event-related potentials (ERPs) during two set-switching tasks to test whether different component processes were responsible for (i) set initiation and reconfiguration when presented with the instruction to switch, and (ii) the implementation of the new set once subjects engaged in performing the new task. The response switching (RS) task required shifts of intentional set; subjects selected between responses according to one of two conflicting intentional sets. The results demonstrated the existence of more than one constituent process. Some of the processes were linked to the initiation and reconfiguration of the set prior to actual performance of the new task. Other processes were time locked to performance of new task items. Set initiation started with modulation of medial frontal ERPs and was followed by modulation over parietal electrodes. Implementation of intentional set was associated with modulation of response-related ERPs.

Active maintenance in prefrontal area 46 creates distractor-resistant memory.

How does the brain maintain information in working memory while challenged by incoming distractions? Using functional magnetic resonance imaging (fMRI), we measured human brain activity during the memory delay of a spatial working memory task with distraction. We found that, in the prefrontal cortex (PFC), the magnitude of activity sustained throughout the memory delay was significantly higher on correct trials than it was on error trials. By contrast, the magnitude of sustained activity in posterior areas did not differ between correct and error trials. The correlation of activity between posterior areas was, however, associated with correct memory performance after distraction. On the basis of these findings, we propose that memory representations gain resistance against distraction during a period of active maintenance within working memory. This may be mediated by interactions between prefrontal and posterior areas.

MRI analysis of an inherited speech and language disorder: structural brain abnormalities.

Analyses of brain structure in genetic speech and language disorders provide an opportunity to identify neurobiological phenotypes and further elucidate the neural bases of language and its development. Here we report such investigations in a large family, known as the KE family, half the members of which are affected by a severe disorder of speech and language, which is transmitted as an autosomal-dominant monogenic trait. The structural brain abnormalities associated with this disorder were investigated using two morphometric methods of MRI analysis. A voxel-based morphometric method was used to compare the amounts of grey matter in the brains of three groups of subjects: the affected members of the KE family, the unaffected members and a group of age-matched controls. This method revealed a number of mainly motor- and speech-related brain regions in which the affected family members had significantly different amounts of grey matter compared with the unaffected and control groups, who did not differ from each other. Several of these regions were abnormal bilaterally, including the caudate nucleus, which was of particular interest because this structure was also found to show functional abnormality in a related PET study. We performed a more detailed volumetric analysis of this structure. The results confirmed that the volume of this nucleus was reduced bilaterally in the affected family members compared with both the unaffected members and the group of age-matched controls. This reduction in volume was most evident in the superior portion of the nucleus. The volume of the caudate nucleus was significantly correlated with the performance of affected family members on a test of oral praxis, a test of non-word repetition and the coding subtest of the Wechsler Intelligence Scale. These results thus provide further evidence of a relationship between the abnormal development of this nucleus and the impairments in oromotor control and articulation reported in the KE family.

A role for the ventral visual stream in reporting movements.

In this experiment we contrast the neural activity associated with reporting a stimulus attribute with the activity that occurs when the same stimulus attribute is used to guide behavior. Reporting the characteristics of a stimulus differs from simply tracking that stimulus since reporting requires that a stimulus is explicitly recognized and associated with an arbitrary response. In one condition the subject used his right finger to follow a square that moved randomly on a screen. In a second condition he had to indicate changes in the direction of the square's movements by touching one of two report buttons with his right finger. Two other conditions were added to control for the differences in the form of movement between the two primary conditions. When the reporting condition was contrasted with the tracking condition (controlling for the differences in the form of movement), areas in the ventral visual system (the left ventral prefrontal cortex and the left inferior temporal cortex) were activated. This study shows that contrasting a manual task which involves a report with a manual task which does not activates the ventral visual system. However, the observation of additional activity in other areas suggests that, while activity in the ventral stream is necessary for reporting, it is not sufficient.

Learning arbitrary visuomotor associations: temporal dynamic of brain activity.

Primates can give behavioral responses on the basis of arbitrary, context-dependent rules. When sensory instructions and behavioral responses are associated by arbitrary rules, these rules need to be learned. This study investigates the temporal dynamics of functional segregation at the basis of visuomotor associative learning in humans, isolating specific learning-related changes in neurovascular activity across the whole brain. We have used fMRI to measure human brain activity during performance of two tasks requiring the association of visual patterns with motor responses. Both tasks were learned by trial and error, either before (visuomotor control) or during (visuomotor learning) the scanning session. Epochs of tasks performance ( approximately 30 s) were alternated with a baseline period over the whole scanning session ( approximately 50 min). We have assessed both linear and nonlinear modulations in the differential signal between tasks, independently from overall task differences. The performance indices of the visuomotor learning task smoothly converged onto the values of a steady-state control condition, according to nonlinear timecourses. Specific visuomotor learning-related activity has been found over a distributed cortical network, centred on a temporo-prefrontal circuit. These cortical time-modulated activities were supported early in learning by the hippocampal/parahippocampal complex, and late in learning by the basal ganglia system. These findings suggest the inferior temporal and the ventral prefrontal cortex are critical neural nodes for integrating perceptual information with executive processes.