Cortical-striatal brain network distinguishes deepfake from real speaker identity.

Deepfakes are viral ingredients of digital environments, and they can trick human cognition into misperceiving the fake as real. Here, we test the neurocognitive sensitivity of 25 participants to accept or reject person identities as recreated in audio deepfakes. We generate high-quality voice identity clones from natural speakers by using advanced deepfake technologies. During an identity matching task, participants show intermediate performance with deepfake voices, indicating levels of deception and resistance to deepfake identity spoofing. On the brain level, univariate and multivariate analyses consistently reveal a central cortico-striatal network that decoded the vocal acoustic pattern and deepfake-level (auditory cortex), as well as natural speaker identities (nucleus accumbens), which are valued for their social relevance. This network is embedded in a broader neural identity and object recognition network. Humans can thus be partly tricked by deepfakes, but the neurocognitive mechanisms identified during deepfake processing open windows for strengthening human resilience to fake information.

Psychopathic and autistic traits differentially influence the neural mechanisms of social cognition from communication signals.

Psychopathy is associated with severe deviations in social behavior and cognition. While previous research described such cognitive and neural alterations in the processing of rather specific social information from human expressions, some open questions remain concerning central and differential neurocognitive deficits underlying psychopathic behavior. Here we investigated three rather unexplored factors to explain these deficits, first, by assessing psychopathy subtypes in social cognition, second, by investigating the discrimination of social communication sounds (speech, non-speech) from other non-social sounds, and third, by determining the neural overlap in social cognition impairments with autistic traits, given potential common deficits in the processing of communicative voice signals. The study was exploratory with a focus on how psychopathic and autistic traits differentially influence the function of social cognitive and affective brain networks in response to social voice stimuli. We used a parametric data analysis approach from a sample of 113 participants (47 male, 66 female) with ages ranging between 18 and 40 years (mean 25.59, SD 4.79). Our data revealed four important findings. First, we found a phenotypical overlap between secondary but not primary psychopathy with autistic traits. Second, primary psychopathy showed various neural deficits in neural voice processing nodes (speech, non-speech voices) and in brain systems for social cognition (mirroring, mentalizing, empathy, emotional contagion). Primary psychopathy also showed deficits in the basal ganglia (BG) system that seems specific to the social decoding of communicative voice signals. Third, neural deviations in secondary psychopathy were restricted to social mirroring and mentalizing impairments, but with additional and so far undescribed deficits at the level of auditory sensory processing, potentially concerning deficits in ventral auditory stream mechanisms (auditory object identification). Fourth, high autistic traits also revealed neural deviations in sensory cortices, but rather in the dorsal auditory processing streams (communicative context encoding). Taken together, social cognition of voice signals shows considerable deviations in psychopathy, with differential and newly described deficits in the BG system in primary psychopathy and at the neural level of sensory processing in secondary psychopathy. These deficits seem especially triggered during the social cognition from vocal communication signals.

Categorizing human vocal signals depends on an integrated auditory-frontal cortical network.

Voice signals are relevant for auditory communication and suggested to be processed in dedicated auditory cortex (AC) regions. While recent reports highlighted an additional role of the inferior frontal cortex (IFC), a detailed description of the integrated functioning of the AC-IFC network and its task relevance for voice processing is missing. Using neuroimaging, we tested sound categorization while human participants either focused on the higher-order vocal-sound dimension (voice task) or feature-based intensity dimension (loudness task) while listening to the same sound material. We found differential involvements of the AC and IFC depending on the task performed and whether the voice dimension was of task relevance or not. First, when comparing neural vocal-sound processing of our task-based with previously reported passive listening designs we observed highly similar cortical activations in the AC and IFC. Second, during task-based vocal-sound processing we observed voice-sensitive responses in the AC and IFC whereas intensity processing was restricted to distinct AC regions. Third, the IFC flexibly adapted to the vocal-sounds' task relevance, being only active when the voice dimension was task relevant. Forth and finally, connectivity modeling revealed that vocal signals independent of their task relevance provided significant input to bilateral AC. However, only when attention was on the voice dimension, we found significant modulations of auditory-frontal connections. Our findings suggest an integrated auditory-frontal network to be essential for behaviorally relevant vocal-sounds processing. The IFC seems to be an important hub of the extended voice network when representing higher-order vocal objects and guiding goal-directed behavior.

Understanding the mechanisms of familiar voice-identity recognition in the human brain.

Humans have a remarkable skill for voice-identity recognition: most of us can remember many voices that surround us as 'unique'. In this review, we explore the computational and neural mechanisms which may support our ability to represent and recognise a unique voice-identity. We examine the functional architecture of voice-sensitive regions in the superior temporal gyrus/sulcus, and bring together findings on how these regions may interact with each other, and additional face-sensitive regions, to support voice-identity processing. We also contrast findings from studies on neurotypicals and clinical populations which have examined the processing of familiar and unfamiliar voices. Taken together, the findings suggest that representations of familiar and unfamiliar voices might dissociate in the human brain. Such an observation does not fit well with current models for voice-identity processing, which by-and-large assume a common sequential analysis of the incoming voice signal, regardless of voice familiarity. We provide a revised audio-visual integrative model of voice-identity processing which brings together traditional and prototype models of identity processing. This revised model includes a mechanism of how voice-identity representations are established and provides a novel framework for understanding and examining the potential differences in familiar and unfamiliar voice processing in the human brain.

Obligatory and facultative brain regions for voice-identity recognition.

Recognizing the identity of others by their voice is an important skill for social interactions. To date, it remains controversial which parts of the brain are critical structures for this skill. Based on neuroimaging findings, standard models of person-identity recognition suggest that the right temporal lobe is the hub for voice-identity recognition. Neuropsychological case studies, however, reported selective deficits of voice-identity recognition in patients predominantly with right inferior parietal lobe lesions. Here, our aim was to work towards resolving the discrepancy between neuroimaging studies and neuropsychological case studies to find out which brain structures are critical for voice-identity recognition in humans. We performed a voxel-based lesion-behaviour mapping study in a cohort of patients (n = 58) with unilateral focal brain lesions. The study included a comprehensive behavioural test battery on voice-identity recognition of newly learned (voice-name, voice-face association learning) and familiar voices (famous voice recognition) as well as visual (face-identity recognition) and acoustic control tests (vocal-pitch and vocal-timbre discrimination). The study also comprised clinically established tests (neuropsychological assessment, audiometry) and high-resolution structural brain images. The three key findings were: (i) a strong association between voice-identity recognition performance and right posterior/mid temporal and right inferior parietal lobe lesions; (ii) a selective association between right posterior/mid temporal lobe lesions and voice-identity recognition performance when face-identity recognition performance was factored out; and (iii) an association of right inferior parietal lobe lesions with tasks requiring the association between voices and faces but not voices and names. The results imply that the right posterior/mid temporal lobe is an obligatory structure for voice-identity recognition, while the inferior parietal lobe is only a facultative component of voice-identity recognition in situations where additional face-identity processing is required.

Developmental phonagnosia: Linking neural mechanisms with the behavioural phenotype.

Human voice recognition is critical for many aspects of social communication. Recently, a rare disorder, developmental phonagnosia, which describes the inability to recognise a speaker's voice, has been discovered. The underlying neural mechanisms are unknown. Here, we used two functional magnetic resonance imaging experiments to investigate brain function in two behaviourally well characterised phonagnosia cases, both 32 years old: AS has apperceptive and SP associative phonagnosia. We found distinct malfunctioned brain mechanisms in AS and SP matching their behavioural profiles. In apperceptive phonagnosia, right-hemispheric auditory voice-sensitive regions (i.e., Heschl's gyrus, planum temporale, superior temporal gyrus) showed lower responses than in matched controls (nAS=16) for vocal versus non-vocal sounds and for speaker versus speech recognition. In associative phonagnosia, the connectivity between voice-sensitive (i.e. right posterior middle/inferior temporal gyrus) and supramodal (i.e. amygdala) regions was reduced in comparison to matched controls (nSP=16) during speaker versus speech recognition. Additionally, both cases recruited distinct potential compensatory mechanisms. Our results support a central assumption of current two-system models of voice-identity processing: They provide the first evidence that dysfunction of voice-sensitive regions and impaired connectivity between voice-sensitive and supramodal person recognition regions can selectively contribute to deficits in person recognition by voice.

Voice identity processing in autism spectrum disorder.

People with autism spectrum disorder (ASD) have difficulties in identifying another person by face and voice. This might contribute considerably to the development of social cognition and interaction difficulties. The characteristics of the voice recognition deficit in ASD are unknown. Here, we used a comprehensive behavioral test battery to systematically investigate voice processing in high-functioning ASD (n = 16) and typically developed pair-wise matched controls (n = 16). The ASD group had particular difficulties with discriminating, learning, and recognizing unfamiliar voices, while recognizing famous voices was relatively intact. Tests on acoustic processing abilities showed that the ASD group had a specific deficit in vocal pitch perception that was dissociable from otherwise intact acoustic processing (i.e., musical pitch, musical, and vocal timbre perception). Our results allow a characterization of the voice recognition deficit in ASD: The findings indicate that in high-functioning ASD, the difficulty to recognize voices is particularly pronounced for learning novel voices and the recognition of unfamiliar peoples' voices. This pattern might be indicative of difficulties with integrating the acoustic characteristics of the voice into a coherent percept-a function that has been previously associated with voice-selective regions in the posterior superior temporal sulcus/gyrus of the human brain. Autism Res 2017, 10: 155-168. © 2016 International Society for Autism Research, Wiley Periodicals, Inc.

Two cases of selective developmental voice-recognition impairments.

Recognizing other individuals is an essential skill in humans and in other species. Over the last decade, it has become increasingly clear that person-identity recognition abilities are highly variable. Roughly 2% of the population has developmental prosopagnosia, a congenital deficit in recognizing others by their faces. It is currently unclear whether developmental phonagnosia, a deficit in recognizing others by their voices, is equally prevalent, or even whether it actually exists. Here, we aimed to identify cases of developmental phonagnosia. We collected more than 1,000 data sets from self-selected German individuals by using a web-based screening test that was designed to assess their voice-recognition abilities. We then examined potentially phonagnosic individuals by using a comprehensive laboratory test battery. We found two novel cases of phonagnosia: AS, a 32-year-old female, and SP, a 32-year-old male; both are otherwise healthy academics, have normal hearing, and show no pathological abnormalities in brain structure. The two cases have comparable patterns of impairments: both performed at least 2 SDs below the level of matched controls on tests that required learning new voices, judging the familiarity of famous voices, and discriminating pitch differences between voices. In both cases, only voice-identity processing per se was affected: face recognition, speech intelligibility, emotion recognition, and musical ability were all comparable to controls. The findings confirm the existence of developmental phonagnosia as a modality-specific impairment and allow a first rough prevalence estimate.