Artificial intelligence to evaluate postoperative pain based on facial expression recognition.

BACKGROUND: Pain intensity evaluation by self-report is difficult and biased in non-communicating people, which may contribute to inappropriate pain management. The use of artificial intelligence (AI) to evaluate pain intensity based on automated facial expression analysis has not been evaluated in clinical conditions. METHODS: We trained and externally validated a deep-learning system (ResNet-18 convolutional neural network) to identify and classify 2810 facial expressions of 1189 patients, captured before and after surgery, according to their self-reported pain intensity using numeric rating scale (NRS, 0-10). AI performances were evaluated by accuracy (concordance between AI prediction and patient-reported pain intensity), sensitivity and specificity to diagnose pain ≥4/10 and ≥7/10. We then confronted AI performances with those of 33 nurses to evaluate pain intensity from facial expression in the same situation. RESULTS: In the external testing set (120 face images), the deep learning system was able to predict exactly the pain intensity among the 11 possible scores (0-10) in 53% of the cases with a mean error of 2.4 points. Its sensitivities to detect pain ≥4/10 and ≥7/10 were 89.7% and 77.5%, respectively. Nurses estimated the right NRS pain intensity with a mean accuracy of 14.9% and identified pain ≥4/10 and ≥7/10 with sensitivities of 44.9% and 17.0%. CONCLUSIONS: Subject to further improvement of AI performances through further training, these results suggest that AI using facial expression analysis could be used to assist physicians to evaluate pain and detect severe pain, especially in people not able to report appropriately their pain by themselves. SIGNIFICANCE: These original findings represent a major step in the development of a fully automated, rapid, standardized and objective method based on facial expression analysis to measure pain and detect severe pain.

Functional MRI examination of empathy for pain in people with schizophrenia reveals abnormal activation related to cognitive perspective-taking but typical activation linked to affective sharing.

BACKGROUND: Schizophrenia is associated with important disturbances in empathy that are related to everyday functioning. Empathy is classically defined as including affective (sharing others' emotions) and cognitive (taking others' cognitive perspectives) processes. In healthy individuals, studies on empathy for pain revealed specific brain systems associated with these sets of processes, notably the anterior middle cingulate (aMCC) and anterior insula (AI) for affective sharing and the bilateral temporoparietal junction (TPJ) for the cognitive processes, but the integrity of these systems in patients with schizophrenia remains uncertain. METHODS: Patients with schizophrenia and healthy controls performed a pain empathy task while undergoing fMRI scanning. Participants observed pictures of hands in either painful or nonpainful situations and rated the level of pain while imagining either themselves (self) or an unknown person (other) in these situations. RESULTS: We included 27 patients with schizophrenia and 21 healthy controls in our analyses. For the pain versus no pain contrast, patients showed overall typical activation patterns in the aMCC and AI, with only a small part of the aMCC showing reduced activation compared with controls. For the other versus self contrast, patients showed an abnormal modulation of activation in the TPJ bilaterally (extending to the posterior superior temporal sulcus, referred to as the TPJ/pSTS). LIMITATIONS: The design included an unnecessary manipulation of the visual perspective that reduced the number of trials for analysis. The sample size may not account for the heterogeneity of schizophrenia. CONCLUSION: People with schizophrenia showed relatively intact brain activation when observing others' pain, but showed abnormalities when asked to take the cognitive perspectives of others.

Social representations and contextual adjustments as two distinct components of the Theory of Mind brain network: Evidence from the REMICS task.

Theory of mind (ToM) refers to the ability to infer the mental states of others. Behavioral measures of ToM usually present information about both a character and the context in which this character is placed, and these different pieces of information can be used to infer the character's mental states. A set of brain regions designated as the ToM brain network is recognized to support (ToM) inferences. Different brain regions within that network could however support different ToM processes. This functional magnetic resonance imaging (fMRI) study aimed to distinguish the brain regions supporting two aspects inherent to many ToM tasks, i.e., the ability to infer or represent mental states and the ability to use the context to adjust these inferences. Nineteen healthy subjects were scanned during the REMICS task, a novel task designed to orthogonally manipulate mental state inferences (as opposed to physical inferences) and contextual adjustments of inferences (as opposed to inferences that do not require contextual adjustments). We observed that mental state inferences and contextual adjustments, which are important aspects of most behavioral ToM tasks, rely on distinct brain regions or subregions within the classical brain network activated in previous ToM research. Notably, an interesting dissociation emerged within the medial prefrontal cortex (mPFC) and temporo-parietal junctions (TPJ) such that the inferior part of these brain regions responded to mental state inferences while the superior part of these brain regions responded to the requirement for contextual adjustments. This study provides evidence that the overall set of brain regions activated during ToM tasks supports different processes, and highlights that cognitive processes related to contextual adjustments have an important role in ToM and should be further studied.

Changes in visual perspective influence brain activity patterns during cognitive perspective-taking of other people's pain.

Empathy refers to our capacity to share and understand the emotional states of others. It relies on two main processes according to existing models: an effortless affective sharing process based on neural resonance and a more effortful cognitive perspective-taking process enabling the ability to imagine and understand how others feel in specific situations. Until now, studies have focused on factors influencing the affective sharing process but little is known about those influencing the cognitive perspective-taking process and the related brain activations during vicarious pain. In the present fMRI study, we used the well-known physical pain observation task to examine whether the visual perspective can influence, in a bottom-up way, the brain regions involved in taking others' cognitive perspective to attribute their level of pain. We used a pseudo-dynamic version of this classic task which features hands in painful or neutral daily life situations while orthogonally manipulating: (1) the visual perspective with which hands were presented (first-person versus third-person conditions) and (2) the explicit instructions to imagine oneself or an unknown person in those situations (Self versus Other conditions). The cognitive perspective-taking process was investigated by comparing Other and Self conditions. When examined across both visual perspectives, this comparison showed no supra-threshold activation. Instead, the Other versus Self comparison led to a specific recruitment of the bilateral temporo-parietal junction when hands were presented according to a first-person (but not third-person) visual perspective. The present findings identify the visual perspective as a factor that modulates the neural activations related to cognitive perspective-taking during vicarious pain and show that this complex cognitive process can be influenced by perceptual stages of information processing.

Neural basis of semantic priming in schizophrenia during a lexical decision task: a magneto-encephalography study.

Numerous behavioral and electrophysiological studies have provided evidence of abnormal semantic processing in schizophrenia. However, the neural basis of these deficits is poorly understood. We investigated magnetic cortical responses elicited by a word-pair lexical decision task in 20 patients with schizophrenia and 12 healthy control subjects. The task involved presentation of a prime word (200 ms), followed by a blank (250 ms), and then a target stimulus (1200 ms); the subject had to decide whether the target was a real word or not. During this task, bilateral temporal and left prefrontal activations were observed in both groups. However, in contrast to controls, patients with schizophrenia did not show increased activation in the left temporal and anterior cingulate cortices between 200 and 450 ms in response to semantic incongruity. These results suggested that schizophrenia was associated with a functional disturbance in some semantic regions that gave rise to the N400 component. Moreover, a significant modulation in the right temporal cortex was observed in patients, but not in controls. This suggested the existence of alternative processes in patients because both groups showed similar behavioral priming. Finally, we elucidated some functional abnormalities in the semantic network during prime word processing in patients, indicated by prolonged activation compared to healthy controls. Thus, in addition to context integration impairment, abnormal activations during the prime word provided new evidence of context processing deficits in schizophrenia.

Abnormal temporal and parietal magnetic activations during the early stages of theory of mind in schizophrenic patients.

Schizophrenia is associated with abnormal cortical activation during theory of mind (ToM), as demonstrated by several fMRI or PET studies. Electrical and temporal characteristics of these abnormalities, especially in the early stages, remain unexplored. Nineteen medicated schizophrenic patients and 21 healthy controls underwent magnetoencephalography (MEG) recording to measure brain response evoked by nonverbal stimuli requiring mentalizing. Three conditions based on comic-strips were contrasted: attribution of intentions to others (AI), physical causality with human characters (PCCH), and physical causality with objects (PCOB). Minimum norm localization was performed in order to select regions of interest (ROIs) within bilateral temporal and parietal regions that showed significant ToM-related activations in the control group. Time-courses of each ROI were compared across group and condition. Reduced cortical activation within the 200 to 600 ms time-window was observed in the selected regions in patients. Significant group by condition interactions (i.e., reduced modulation in patients) were found in right posterior superior temporal sulcus, right temporoparietal junction, and right inferior parietal lobule during attribution of intentions. As in healthy controls, the presence of characters elicited activation in patients' left posterior temporal regions and temporoparietal junction. No group difference on evoked responses' latencies in AI was found. In conclusion, ToM processes in the early stages are functionally impaired in schizophrenia. MEG provides a promising means to refine our knowledge on schizophrenic social cognitive disorders.

Anatomical and temporal architecture of theory of mind: a MEG insight into the early stages.

Representation of others' actions and mental states leads to the activation of several brain networks: the mentalizing and the "mirror neuron" systems as well as a "low level" social perception component. However, respective activations of the regions belonging to these networks remain unknown with respect to chronometrical data when static drawing stimuli are presented. To determine anatomical and temporal characteristics of theory of mind processes, magnetic signals were measured in 21 subjects during a validated nonverbal attribution of intentions task. Minimum norm estimation provides chronometric and localization data showing that regions known to be involved in the mentalizing, "mirror neuron" and social perception networks have simultaneous activations between 100 and 700 ms post-stimulus, a period which may be thought as corresponding to early stages of social processes. Among some regions, different profiles as well as modulations regarding experimental conditions suggest functional distinctions between these structures, pleading for a cooperative nature of these networks. While the left temporo-parietal area and superior temporal sulcus seem more specialized in social cues coding, we demonstrate that their right homologues, as well as the right inferior parietal cortex, are preferentially recruited during attribution of intentions stimuli compared to scenarios based on physical causality from 200 to 600 ms.

The study of social cognition with neuroimaging methods as a means to explore future directions of deficit evaluation in schizophrenia?

This article discusses the important advances in a recent field of science dealing with the brain processes implicated in understanding social situations and interacting with others. Many behavioral studies on schizophrenia have shown the impairment of these processes and their preferential relation with disorganization and negative syndromes. Brain imaging is a powerful method to identify brain systems participating in these processes in healthy subjects and will be used increasingly to study mental disorders such as schizophrenia. A few preliminary studies have opened this field of research and allowed for the drawing of some limited conclusions. We emphasize the importance of developing an integrated neurocognitive framework to account for the multifaceted nature of social cognition deficits in schizophrenia. Inspired by contemporary models of empathy and social cognition that identify different components such as shared representation, mentalizing, self/other distinction, we show how schizophrenia affects these components at the behavioral and functional levels. We also outline the interest of this model to understand putative abnormalities of contextual integration within the area of mentalization. Finally, we discuss how specialized measures of brain functions during the performance of these precisely defined mental processes might be used as outcome predictors.

The influence of idiomatic salience during the comprehension of ambiguous idioms by patients with schizophrenia.

This study investigates whether figurative comprehension in schizophrenia is influenced by the salience of idiomatic meaning, and whether it is affected by clinical and demographic factors and IQ. Twenty-seven schizophrenic patients and 25 healthy participants performed a semantic relatedness judgement task which required the comprehension of idioms with two plausible meanings (literal and figurative). The study also used literal expressions. The figurative meaning of the idioms was less salient (ILS), more salient (IFS), or equally salient (IES) compared to the literal meaning. The results showed "a salience effect" (i.e., all participants understood the salient meanings better than the less salient meanings). There was also a "figurativeness effect" (i.e., healthy individuals understood the figurative meaning of IES better than the literal meaning but not schizophrenic patients). In patients, their thought disorder influenced the figurative comprehension of IFS. The verbal IQ influenced the figurative comprehension of ILS. The thought disorder, the verbal IQ, and the educational level influenced the figurative comprehension of IES. The patients' clinically evaluated concretism was associated with a reduced figurative comprehension of IFS and IES evaluated at a cognitive level. The results are discussed in relation to cognitive mechanisms which underscore figurative comprehension in schizophrenia.

Field perspective deficit for positive memories characterizes autobiographical memory in euthymic depressed patients.

Research on autobiographical memory (AM) and the ability to retrieve specific autobiographical events in euthymic depressed patients yielded divergent results. The main goal of the present study was to further explore episodic specificity of AM among fully remitted depressed patients. Twenty euthymic depressed patients and 20 matched healthy controls were given a semi-structured interview, which assesses episodic specificity of positive and negative autobiographical memories regarding event and details' specificity, autonoetic consciousness (remember/know procedure) and visual perspective (field/observer procedure). Results showed an impairment of episodic specificity of AM in euthymic depressed patients. This impairment was explained by a field perspective deficit for positive memories only. These results suggest that euthymic patients continue to exhibit discrepancy between their current self and their self for positive past behaviors, which maintains an unfavorable view of their current self. Specific cognitive interventions may improve the self-relevance of their positive memories.