Automatic Recognition of Macaque Facial Expressions for Detection of Affective States.

Internal affective states produce external manifestations such as facial expressions. In humans, the Facial Action Coding System (FACS) is widely used to objectively quantify the elemental facial action units (AUs) that build complex facial expressions. A similar system has been developed for macaque monkeys-the Macaque FACS (MaqFACS); yet, unlike the human counterpart, which is already partially replaced by automatic algorithms, this system still requires labor-intensive coding. Here, we developed and implemented the first prototype for automatic MaqFACS coding. We applied the approach to the analysis of behavioral and neural data recorded from freely interacting macaque monkeys. The method achieved high performance in the recognition of six dominant AUs, generalizing between conspecific individuals (Macaca mulatta) and even between species (Macaca fascicularis). The study lays the foundation for fully automated detection of facial expressions in animals, which is crucial for investigating the neural substrates of social and affective states.

Visual preferences for direct-gaze faces in infant macaques (Macaca mulatta) with limited face exposure.

From birth, human and nonhuman primates attend more to faces with direct gaze compared with averted gaze, and previous studies report that attention to the eyes is linked to the emergence of later social skills. Here, we explored whether early experiences influence attraction to eye contact in infant macaques by examining their attention to face pairs varying in their gaze direction across the first 13 weeks of life. Infants raised by human caretakers had limited conspecific interactions (nursery-reared; N = 16) and were compared to infants raised in rich social environments (mother-reared; N = 20). Both groups looked longer to faces and the eyes of direct compared to averted-gaze faces. Looking to all faces and eyes also increased with age. Nursery-reared infants did not display age-associated increases in attention to direct-gaze faces specifically, suggesting that, while there may be an initial preference for direct-gaze faces from birth, social experiences may support its early development.

Intranasal oxytocin in rhesus monkeys alters brain networks that detect social salience and reward.

In primates, resting state functional neuroimaging (rsfcMRI) has identified several large-scale, intrinsic brain networks, including the salience network (SN), which is involved in detecting stimulus salience. Intranasal oxytocin (IN-OT) has been shown to modulate the salience and rewarding quality of social stimuli in mammals and numerous studies have shown that it can affect the functional connectivity between brain regions. Less is known, however, about how these effects unfold over time following IN-OT administration. This study used rsfcMRI in anesthetized rhesus macaques to track temporal changes in the functional connectivity between brain regions involved in the SN, including the anterior cingulate cortex (ACC), anterior insula (AI), amygdala (amy), and ventral striatum (vstr), lasting 3 hr after IN-OT or Placebo (saline) administration. We found significant temporal changes in the functional connectivity between all regions associated with treatment condition. IN-OT increased the functional connectivity between AI_vstr, ACC_amy (right hemisphere), ACC_vstr (left hemisphere), and amy_vstr (right hemisphere), but reduced the functional connectivity between ACC_AI, and the AI_amygdala. These results suggest that IN-OT may dampen salience detection in rhesus monkeys, consistent with previous findings of reduced social vigilance, while enhancing the connectivity between the SN and regions involved in processing reward.

Robust representations of individual faces in chimpanzees (Pan troglodytes) but not monkeys (Macaca mulatta).

Being able to recognize the faces of our friends and family members no matter where we see them represents a substantial challenge for the visual system because the retinal image of a face can be degraded by both changes in the person (age, expression, pose, hairstyle, etc.) and changes in the viewing conditions (direction and degree of illumination). Yet most of us are able to recognize familiar people effortlessly. A popular theory for how face recognition is achieved has argued that the brain stabilizes facial appearance by building average representations that enhance diagnostic features that reliably vary between people while diluting features that vary between instances of the same person. This explains why people find it easier to recognize average images of people, created by averaging multiple images of the same person together, than single instances (i.e. photographs). Although this theory is gathering momentum in the psychological and computer sciences, there is no evidence of whether this mechanism represents a unique specialization for individual recognition in humans. Here we tested two species, chimpanzees (Pan troglodytes) and rhesus monkeys (Macaca mulatta), to determine whether average images of different familiar individuals were easier to discriminate than photographs of familiar individuals. Using a two-alternative forced-choice, match-to-sample procedure, we report a behaviour response profile that suggests chimpanzees encode the faces of conspecifics differently than rhesus monkeys and in a manner similar to humans.

An evaluation of central penetration from a peripherally administered oxytocin receptor selective antagonist in nonhuman primates.

The physiology of the oxytocin receptor has increasingly become a focus of scientific investigation due to its connection with social behavior and psychiatric disorders with impairments in social funciton. Experimental utilization of small molecule and peptide antagonists for the oxytocin receptor has played a role in deciphering these biological and social behavior connections in rodents. Described herein is the evaluation of a potent and selective oxytocin receptor antagonist, ALS-I-41, and details to consider for its use in nonhuman primate behavioral pharmacology experiments utilizing intranasal or intramuscular administration. The central nervous system penetration and rate of metabolism of ALS-I-41 was investigated via mass spectroscopy analysis of cerebrospinal fluid and plasma in the rhesus macaque after intranasal and intramuscular administration. Positron emission tomography was also utilized with [18F] ALS-I-41 in a macaque to verify observed central nervous system (CNS) penetration and to further evaluate the effects of administration rate on CNS penetration of Sprague-Dawley rats in comparison to previous studies.

Effects of chronic oxytocin on attention to dynamic facial expressions in infant macaques.

Studies in a variety of species have reported enhanced prosocial effects after an acute administration of the neuromodulating hormone, oxytocin (OT). Although the exact mechanisms underlying these effects are not fully understood, there is broad interest in developing OT into a treatment for social deficits. Only a few studies, however, have examined the effects of OT if given repeatedly during early development, the period when early intervention is likely to have the greatest benefits for reversing the progression towards social impairment. Those studies, exclusively in rodents, report mixed results. Some have shown enhancement of prosocial behavior, including increased social exploration, but others have shown anti-social effects, including increased aggression. In the present study, infant rhesus macaques were treated with a high-frequency (3× per week) or low-frequency (1× per week) dose of intranasal oxytocin (IN-OT) or placebo (IN-saline) between two and six months of age, after which their reactions to dynamic facial expressions (neutral, lipsmacking and threats) were measured. Results showed that IN-OT, compared to placebo, increased the time monkeys spent viewing the expression videos, but selectively reduced attention to the eyes in neutral faces in a dose dependent manner. The mechanism for this non-prosocial effect may be that repeated IN-OT administration down-regulates the expression of OT receptors in brain regions important for regulating social attention. Consequently, our results raise questions about the efficacy of implementing chronic IN-OT as a pharmacotherapy for the treatment of social deficits, particularly if given early in development. More work is needed, not only to identify optimal treatment schedules, but also to understand how IN-OT exerts its influences on the brain and behavior.

Experience-dependent changes in the development of face preferences in infant rhesus monkeys.

It is well known that early experience shapes the development of visual perception for faces in humans. However, the effect of experience on the development of social attention in non-human primates is unknown. In two studies, we examined the effect of cumulative social experience on developmental changes in attention to the faces of unfamiliar conspecifics or heterospecifics, and mom versus an unfamiliar female. From birth, infant rhesus monkeys preferred to look at conspecific compared to heterospecific faces, but this pattern reversed over time. In contrast, no consistent differences were found for attention to mom's face compared to an unfamiliar female. These results suggest differential roles of social experience in shaping the development of face preferences in infant monkeys. Results have important implications for establishing normative trajectories for the development of face preferences in an animal model of human social behavior.

The development of visual preferences for direct versus averted gaze faces in infant macaques (Macaca mulatta).

Human and nonhuman primates show a preference for looking at faces with direct gaze. In humans, this preference emerges shortly after birth, but little is known about the development of gaze preferences in monkeys. This study tracked the development of gaze preferences in infant monkeys from birth through 6 months of age using infrared eye-tracking. Although absent in the first week, a strong significant preference for direct compared to averted gaze faces emerged rapidly, peaking around 2 months of age. When looking at the eyes, the monkeys' fixations were equivalent in duration for both gaze types in the first 2 months, but thereafter remained longer for the averted gaze faces. Therefore, the infants spent a greater proportion of time overall, but made shorter fixations, when looking at the direct compared to averted gaze faces. These results suggest that monkeys develop an efficient strategy when viewing the preferred direct gaze faces that involves longer viewing times, but shorter fixations.

The Macaque Social Responsiveness Scale (mSRS): A Rapid Screening Tool for Assessing Variability in the Social Responsiveness of Rhesus Monkeys (Macaca mulatta).

Understanding the biological mechanisms underlying human neuropsychiatric disorders, such as autism spectrum disorder (ASD), has been hindered by the lack of a robust, translational animal model. Rhesus monkeys (Macaca mulatta) display many of the same social behaviors that are affected in ASD, making them an excellent animal species in which to model social impairments. However, the social impairments associated with ASD may reflect extreme ends of a continuous distribution of traits. Thus, to validate the rhesus monkey as an animal model for studying social impairments that has strong translational relevance for ASD, researchers need an easily-implemented measurement tool that can quantify variation in social behavior dimensionally. The Social Responsiveness Scale (SRS) is a 65-item survey that identifies both typical and atypical social behaviors in humans that covary with ASD symptom severity. A chimpanzee SRS has already been validated and the current study adapted this tool for use in the rhesus monkey (mSRS). Fifteen raters completed the mSRS for 105 rhesus monkeys living at the Yerkes National Primate Research Center. The mSRS scores showed a unimodal distribution with a positive skew that identified 6 statistical outliers. Inter-rater reliability was very strong, but only 17 of the 36 questions showed positive intra-item reliability. The results of an exploratory factor analysis identified 3 factors that explained over 60% of the variance, with 12 items significantly loading onto the primary factor. These items reflected behaviors associated with social avoidance, social anxiety or inflexibility and social confidence. These initial findings are encouraging and suggest that variability in the social responsiveness of rhesus monkeys can be quantified using the mSRS: a tool that has strong translational relevance for human disorders. With further modification, the mSRS may provide an promising new direction for research on the biological mechanisms underlying social impairments.

Establishing the reliability of rhesus macaque social network assessment from video observations.

Understanding the properties of a social environment is important for understanding the dynamics of social relationships. Understanding such dynamics is relevant for multiple fields, ranging from animal behaviour to social and cognitive neuroscience. To quantify social environment properties, recent studies have incorporated social network analysis. Social network analysis quantifies both the global and local properties of a social environment, such as social network efficiency and the roles played by specific individuals, respectively. Despite the plethora of studies incorporating social network analysis, methods to determine the amount of data necessary to derive reliable social networks are still being developed. Determining the amount of data necessary for a reliable network is critical for measuring changes in the social environment, for example following an experimental manipulation, and therefore may be critical for using social network analysis to statistically assess social behaviour. In this paper, we extend methods for measuring error in acquired data and for determining the amount of data necessary to generate reliable social networks. We derived social networks from a group of 10 male rhesus macaques, Macaca mulatta, for three behaviours: spatial proximity, grooming and mounting. Behaviours were coded using a video observation technique, where video cameras recorded the compound where the 10 macaques resided. We collected, coded and used 10 h of video data to construct these networks. Using the methods described here, we found in our data that 1 h of spatial proximity observations produced reliable social networks. However, this may not be true for other studies due to differences in data acquisition. Our results have broad implications for measuring and predicting the amount of error in any social network, regardless of species.

Familiar and unfamiliar face recognition in crested macaques (Macaca nigra).

Many species use facial features to identify conspecifics, which is necessary to navigate a complex social environment. The fundamental mechanisms underlying face processing are starting to be well understood in a variety of primate species. However, most studies focus on a limited subset of species tested with unfamiliar faces. As well as limiting our understanding of how widely distributed across species these skills are, this also limits our understanding of how primates process faces of individuals they know, and whether social factors (e.g. dominance and social bonds) influence how readily they recognize others. In this study, socially housed crested macaques voluntarily participated in a series of computerized matching-to-sample tasks investigating their ability to discriminate (i) unfamiliar individuals and (ii) members of their own social group. The macaques performed above chance on all tasks. Familiar faces were not easier to discriminate than unfamiliar faces. However, the subjects were better at discriminating higher ranking familiar individuals, but not unfamiliar ones. This suggests that our subjects applied their knowledge of their dominance hierarchies to the pictorial representation of their group mates. Faces of high-ranking individuals garner more social attention, and therefore might be more deeply encoded than other individuals. Our results extend the study of face recognition to a novel species, and consequently provide valuable data for future comparative studies.

Facial expression recognition in crested macaques (Macaca nigra).

Facial expressions are a main communication channel used by many different species of primate. Despite this, we know relatively little about how primates discriminate between different facial expressions, and most of what we do know comes from a restricted number of well-studied species. In this study, three crested macaques (Macaca nigra) took part in matching-to-sample tasks where they had to discriminate different facial expressions. In a first experiment, the macaques had to match a photograph of a facial expression to another exemplar of the same expression produced by a different individual, against examples of one of three other types of expressions and neutral faces. In a second experiment, they had to match a dynamic video recording of a facial expression to a still photograph of another exemplar of the same facial expression produced by another individual, also against one of four other expressions. The macaques performed above chance in both tasks, identifying expressions as belonging to the same category regardless of individual identity. Using matrix correlations and multidimensional scaling, we analysed the pattern of errors to see whether overall similarity between facial expressions and/or specific morphological features caused the macaques to confuse facial expressions. Overall similarity, measured with the macaque facial action coding system (maqFACS), did not correlate with performances. Instead, functional similarities between facial expressions could be responsible for the observed pattern of error. These results expand previous findings to a novel primate species and highlight the potential of using video stimuli to investigate the perception and categorisation of visual signals in primates.

The default mode network in chimpanzees (Pan troglodytes) is similar to that of humans.

The human default mode network (DMN), comprising medial prefrontal cortex, precuneus, posterior cingulate cortex, lateral parietal cortex, and medial temporal cortex, is highly metabolically active at rest but deactivates during most focused cognitive tasks. The DMN and social cognitive networks overlap significantly in humans. We previously demonstrated that chimpanzees (Pan troglodytes) show highest resting metabolic brain activity in the cortical midline areas of the human DMN. Human DMN is defined by task-induced deactivations, not absolute resting metabolic levels; ergo, resting activity is insufficient to define a DMN in chimpanzees. Here, we assessed the chimpanzee DMN's deactivations relative to rest during cognitive tasks and the effect of social content on these areas' activity. Chimpanzees performed a match-to-sample task with conspecific behavioral stimuli of varying sociality. Using [(18)F]-FDG PET, brain activity during these tasks was compared with activity during a nonsocial task and at rest. Cortical midline areas in chimpanzees deactivated in these tasks relative to rest, suggesting a chimpanzee DMN anatomically and functionally similar to humans. Furthermore, when chimpanzees make social discriminations, these same areas (particularly precuneus) are highly active relative to nonsocial tasks, suggesting that, as in humans, the chimpanzee DMN may play a role in social cognition.

Human faces are slower than chimpanzee faces.

BACKGROUND: While humans (like other primates) communicate with facial expressions, the evolution of speech added a new function to the facial muscles (facial expression muscles). The evolution of speech required the development of a coordinated action between visual (movement of the lips) and auditory signals in a rhythmic fashion to produce "visemes" (visual movements of the lips that correspond to specific sounds). Visemes depend upon facial muscles to regulate shape of the lips, which themselves act as speech articulators. This movement necessitates a more controlled, sustained muscle contraction than that produced during spontaneous facial expressions which occur rapidly and last only a short period of time. Recently, it was found that human tongue musculature contains a higher proportion of slow-twitch myosin fibers than in rhesus macaques, which is related to the slower, more controlled movements of the human tongue in the production of speech. Are there similar unique, evolutionary physiologic biases found in human facial musculature related to the evolution of speech? METHODOLOGY/PRINICIPAL FINDINGS: Using myosin immunohistochemistry, we tested the hypothesis that human facial musculature has a higher percentage of slow-twitch myosin fibers relative to chimpanzees (Pan troglodytes) and rhesus macaques (Macaca mulatta). We sampled the orbicularis oris and zygomaticus major muscles from three cadavers of each species and compared proportions of fiber-types. Results confirmed our hypothesis: humans had the highest proportion of slow-twitch myosin fibers while chimpanzees had the highest proportion of fast-twitch fibers. CONCLUSIONS/SIGNIFICANCE: These findings demonstrate that the human face is slower than that of rhesus macaques and our closest living relative, the chimpanzee. They also support the assertion that human facial musculature and speech co-evolved. Further, these results suggest a unique set of evolutionary selective pressures on human facial musculature to slow down while the function of this muscle group diverged from that of other primates.

Aerosolized oxytocin increases cerebrospinal fluid oxytocin in rhesus macaques.

Intranasal (IN) administration is a widely used method for examining the effect of oxytocin (OT) on social behavior and cognition in healthy subjects and psychiatric populations. IN-OT in humans enhances trust, emotional perception, and empathetic behavior and is under investigation as a potential pharmacotherapy to enhance social functioning in a variety of neuropsychiatric disorders, including autism spectrum disorders (ASD). Nonhuman primates (NHP) are an important model for understanding the effect of OT on social cognition, its neural mechanisms, and the development of IN-OT as a pharmacotherapy for treating social deficits in humans. However, NHP and even some human populations, such as very young infants and children, cannot easily follow the detailed self-administration protocol used in the majority of human IN-OT studies. Therefore, we evaluated the efficacy of several OT-administration routes for elevating central OT concentrations in rhesus macaques. First, we examined the effect of IN and intravenous (IV) routes of OT administration on concentrations of OT and vasopressin (AVP) in plasma and lumbar CSF. Second, we examined these same measures in monkeys after an aerosolized (AE) OT delivery route. All three administration routes significantly increased plasma OT concentrations, but only the AE-OT route significantly increased concentrations of CSF OT. No route affected concentrations of AVP in plasma or CSF. This study confirms that the AE route is the most effective method for increasing central OT concentrations in monkeys, and may also be an effective route, alternative to IN, for administering OT to some human populations.

Of mice, monkeys, and men: physiological and morphological evidence for evolutionary divergence of function in mimetic musculature.

Facial expression is a universal means of visual communication in humans and many other primates. Humans have the most complex facial display repertoire among primates; however, gross morphological studies have not found greater complexity in human mimetic musculature. This study examines the microanatomical aspects of mimetic musculature to test the hypotheses related to human mimetic musculature physiology, function, and evolutionary morphology. Samples from the orbicularis oris muscle (OOM) and the zygomaticus major (ZM) muscle in laboratory mice (N = 3), rhesus macaques (N = 3), and humans (N = 3) were collected. Fiber type proportions (slow-twitch and fast-twitch), fiber cross-sectional area, diameter, and length were calculated, and means were statistically compared among groups. Results showed that macaques had the greatest percentage of fast fibers in both muscles (followed by humans) and that humans had the greatest percentage of slow fibers in both muscles. Macaques and humans typically did not differ from one another in morphometrics except for fiber length where humans had longer fibers. Although sample sizes are low, results from this study may indicate that the rhesus macaque OOM and ZM muscle are specialized primarily to assist with maintenance of the rigid dominance hierarchy via rapid facial displays of submission and aggression, whereas human musculature may have evolved not only under pressure to work in facial expressions but also in development of speech.

Differences in neural activation for object-directed grasping in chimpanzees and humans.

The human faculty for object-mediated action, including tool use and imitation, exceeds that of even our closest primate relatives and is a key foundation of human cognitive and cultural uniqueness. In humans and macaques, observing object-directed grasping actions activates a network of frontal, parietal, and occipitotemporal brain regions, but differences in human and macaque activation suggest that this system has been a focus of selection in the primate lineage. To study the evolution of this system, we performed functional neuroimaging in humans' closest living relatives, chimpanzees. We compare activations during performance of an object-directed manual grasping action, observation of the same action, and observation of a mimed version of the action that consisted of only movements without results. Performance and observation of the same action activated a distributed frontoparietal network similar to that reported in macaques and humans. Like humans and unlike macaques, these regions were also activated by observing movements without results. However, in a direct chimpanzee/human comparison, we also identified unique aspects of human neural responses to observed grasping. Chimpanzee activation showed a prefrontal bias, including significantly more activity in ventrolateral prefrontal cortex, whereas human activation was more evenly distributed across more posterior regions, including significantly more activation in ventral premotor cortex, inferior parietal cortex, and inferotemporal cortex. This indicates a more "bottom-up" representation of observed action in the human brain and suggests that the evolution of tool use, social learning, and cumulative culture may have involved modifications of frontoparietal interactions.

Intranasal oxytocin selectively attenuates rhesus monkeys' attention to negative facial expressions.

Intranasal oxytocin (IN-OT) modulates social perception and cognition in humans and could be an effective pharmacotherapy for treating social impairments associated with neuropsychiatric disorders, like autism. However, it is unknown how IN-OT modulates social cognition, its effect after repeated use, or its impact on the developing brain. Animal models are urgently needed. This study examined the effect of IN-OT on social perception in monkeys using tasks that reveal some of the social impairments seen in autism. Six rhesus macaques (Macaca mulatta, 4 males) received a 48 IU dose of OT or saline placebo using a pediatric nebulizer. An hour later, they performed a computerized task (the dot-probe task) to measure their attentional bias to social, emotional, and nonsocial images. Results showed that IN-OT significantly reduced monkeys' attention to negative facial expressions, but not neutral faces or clip art images and, additionally, showed a trend to enhance monkeys' attention to direct vs. averted gaze faces. This study is the first to demonstrate an effect of IN-OT on social perception in monkeys, IN-OT selectively reduced monkey's attention to negative facial expressions, but not neutral social or nonsocial images. These findings complement several reports in humans showing that IN-OT reduces the aversive quality of social images suggesting that, like humans, monkey social perception is mediated by the oxytocinergic system. Importantly, these results in monkeys suggest that IN-OT does not dampen the emotional salience of social stimuli, but rather acts to affect the evaluation of emotional images during the early stages of information processing.

How the Thatcher illusion reveals evolutionary differences in the face processing of primates.

Face recognition in humans is a complex cognitive skill that requires sensitivity to unique configurations of eyes, mouth, and other facial features. The Thatcher illusion has been used to demonstrate the importance of orientation when processing configural information within faces. Transforming an upright face so that the eyes and mouth are inverted renders the face grotesque; however, when this "Thatcherized" face is inverted, the effect disappears. Due to the use of primate models in social cognition research, it is important to determine the extent to which specialized cognitive functions like face processing occur across species. To date, the Thatcher illusion has been explored in only a few species with mixed results. Here, we used computerized tasks to examine whether nonhuman primates perceive the Thatcher illusion. Chimpanzees and rhesus monkeys were required to discriminate between Thatcherized and unaltered faces presented upright and inverted. Our results confirm that chimpanzees perceived the Thatcher illusion, but rhesus monkeys did not, suggesting species differences in the importance of configural information in face processing. Three further experiments were conducted to understand why our results differed from previously published accounts of the Thatcher illusion in rhesus monkeys.