A simple psychophysical procedure separates representational and noise components in impairments of speech prosody perception after right-hemisphere stroke.

After a right hemisphere stroke, more than half of the patients are impaired in their capacity to produce or comprehend speech prosody. Yet, and despite its social-cognitive consequences for patients, aprosodia following stroke has received scant attention. In this report, we introduce a novel, simple psychophysical procedure which, by combining systematic digital manipulations of speech stimuli and reverse-correlation analysis, allows estimating the internal sensory representations that subtend how individual patients perceive speech prosody, and the level of internal noise that govern behavioral variability in how patients apply these representations. Tested on a sample of N = 22 right-hemisphere stroke survivors and N = 21 age-matched controls, the representation + noise model provides a promising alternative to the clinical gold standard for evaluating aprosodia (MEC): both parameters strongly associate with receptive, and not expressive, aprosodia measured by MEC within the patient group; they have better sensitivity than MEC for separating high-functioning patients from controls; and have good specificity with respect to non-prosody-related impairments of auditory attention and processing. Taken together, individual differences in either internal representation, internal noise, or both, paint a potent portrait of the variety of sensory/cognitive mechanisms that can explain impairments of prosody processing after stroke.

Cortical responses to looming sources are explained away by the auditory periphery.

A wealth of behavioral evidence indicate that sounds with increasing intensity (i.e. appear to be looming towards the listener) are processed with increased attentional and physiological resources compared to receding sounds. However, the neurophysiological mechanism responsible for such cognitive amplification remains elusive. Here, we show that the large differences seen between cortical responses to looming and receding sounds are in fact almost entirely explained away by nonlinear encoding at the level of the auditory periphery. We collected electroencephalography (EEG) data during an oddball paradigm to elicit mismatch negativity (MMN) and others Event Related Potentials (EPRs), in response to deviant stimuli with both dynamic (looming and receding) and constant level (flat) differences to the standard in the same participants. We then combined a computational model of the auditory periphery with generative EEG methods (temporal response functions, TRFs) to model the single-participant ERPs responses to flat deviants, and used them to predict the effect of the same mechanism on looming and receding stimuli. The flat model explained 45% variance of the looming response, and 33% of the receding response. This provide striking evidence that difference wave responses to looming and receding sounds result from the same cortical mechanism that generate responses to constant-level deviants: all such differences are the sole consequence of their particular physical morphology getting amplified and integrated by peripheral auditory mechanisms. Thus, not all effects seen cortically proceed from top-down modulations by high-level decision variables, but can rather be performed early and efficiently by feed-forward peripheral mechanisms that evolved precisely to sparing subsequent networks with the necessity to implement such mechanisms.

Impact of the practice of touch-massage® by a nurse on the anxiety of patients with hematological disorders hospitalized in a sterile environment, a randomized, controlled study.

CONTEXT: In addition to curative care, supportive care is beneficial in managing the anxiety symptoms common in patients in sterile hematology unit. We hypothesize that personal massage can help the patient, particularly in this isolated setting where physical contact is extremely limited. The main objective of this study was to show that anxiety could be reduced after a touch-massage® performed by a nurse trained in this therapy. METHODS: A single-center, randomized, unblinded controlled study in the sterile hematology unit of a French university hospital, validated by an ethics committee. The patients, aged between 18 and 65 years old, and suffering from a serious and progressive hematological pathology, were hospitalized in sterile hematology unit for a minimum of three weeks, patients were randomized into either a group receiving 15-minute touch-massage® sessions or a control group receiving an equivalent amount of quiet time once a week for three weeks. In the treated group, anxiety was assessed before and after each touch-massage® session, using the State-Trait Anxiety Inventory questionnaire with subscale state (STAI-State). In the control group, anxiety was assessed before and after a 15-minute quiet period. For each patient, the difference in the STAI-State score before and after each session (or period) was calculated, the primary endpoint was based on the average of these three differences. Each patient completed the Rosenberg Self-Esteem Questionnaire before the first session and after the last session. RESULTS: Sixty-two patients were randomized. Touch-massage® significantly decreased patient anxiety: a mean decrease in STAI-State scale score of 10.6 [7.65-13.54] was obtained for the massage group (p ≤ 0.001) compared with the control group. The improvement in self-esteem score was not significant. CONCLUSION: This study provides convincing evidence for integrating touch-massage® in the treatment of patients in sterile hematology unit. TRIAL REGISTRATION: NCT02343965.

Combining GAN with reverse correlation to construct personalized facial expressions.

Recent deep-learning techniques have made it possible to manipulate facial expressions in digital photographs or videos, however, these techniques still lack fine and personalized ways to control their creation. Moreover, current technologies are highly dependent on large labeled databases, which limits the range and complexity of expressions that can be modeled. Thus, these technologies cannot deal with non-basic emotions. In this paper, we propose a novel interdisciplinary approach combining the Generative Adversarial Network (GAN) with a technique inspired by cognitive sciences, psychophysical reverse correlation. Reverse correlation is a data-driven method able to extract an observer's 'mental representation' of what a given facial expression should look like. Our approach can generate 1) personalized facial expression prototypes, 2) of basic emotions, and non-basic emotions that are not available in existing databases, and 3) without the need for expertise. Personalized prototypes obtained with reverse correlation can then be applied to manipulate facial expressions. In addition, our system challenges the universality of facial expression prototypes by proposing the concepts of dominant and complementary action units to describe facial expression prototypes. The evaluations we conducted on a limited number of emotions validate the effectiveness of our proposed method. The code is available at https://github.com/yansen0508/Mental-Deep-Reverse-Engineering.

Algorithmic voice transformations reveal the phonological basis of language-familiarity effects in cross-cultural emotion judgments.

People have a well-described advantage in identifying individuals and emotions in their own culture, a phenomenon also known as the other-race and language-familiarity effect. However, it is unclear whether native-language advantages arise from genuinely enhanced capacities to extract relevant cues in familiar speech or, more simply, from cultural differences in emotional expressions. Here, to rule out production differences, we use algorithmic voice transformations to create French and Japanese stimulus pairs that differed by exactly the same acoustical characteristics. In two cross-cultural experiments, participants performed better in their native language when categorizing vocal emotional cues and detecting non-emotional pitch changes. This advantage persisted over three types of stimulus degradation (jabberwocky, shuffled and reversed sentences), which disturbed semantics, syntax, and supra-segmental patterns, respectively. These results provide evidence that production differences are not the sole drivers of the language-familiarity effect in cross-cultural emotion perception. Listeners' unfamiliarity with the phonology of another language, rather than with its syntax or semantics, impairs the detection of pitch prosodic cues and, in turn, the recognition of expressive prosody.

Pupil dilation reflects the dynamic integration of audiovisual emotional speech.

Emotional speech perception is a multisensory process. When speaking with an individual we concurrently integrate the information from their voice and face to decode e.g., their feelings, moods, and emotions. However, the physiological reactions-such as the reflexive dilation of the pupil-associated to these processes remain mostly unknown. That is the aim of the current article, to investigate whether pupillary reactions can index the processes underlying the audiovisual integration of emotional signals. To investigate this question, we used an algorithm able to increase or decrease the smiles seen in a person's face or heard in their voice, while preserving the temporal synchrony between visual and auditory channels. Using this algorithm, we created congruent and incongruent audiovisual smiles, and investigated participants' gaze and pupillary reactions to manipulated stimuli. We found that pupil reactions can reflect emotional information mismatch in audiovisual speech. In our data, when participants were explicitly asked to extract emotional information from stimuli, the first fixation within emotionally mismatching areas (i.e., the mouth) triggered pupil dilation. These results reveal that pupil dilation can reflect the dynamic integration of audiovisual emotional speech and provide insights on how these reactions are triggered during stimulus perception.

Integrating Selection and Risk Assessment of Chemical Mixtures: A Novel Approach Applied to a Breast Milk Survey.

BACKGROUND: One of the main challenges of modern risk assessment is to account for combined exposure to the multitude of various substances present in food and the environment. OBJECTIVE: The present work proposes a methodological approach to perform chemical risk assessment of contaminant mixtures across regulatory silos regarding an extensive range of substances and to do so when comprehensive relevant data concerning the specific effects and modes of action of the mixture components are not available. METHODS: We developed a complete step-by-step approach using statistical methods to prioritize substances involved in combined exposure, and we used a component-based approach to cumulate the risk using dose additivity. The most relevant toxicological end point and the associated reference point were selected from the literature to construct a toxicological threshold for each substance. DISCUSSION: By applying the proposed method to contaminants in breast milk, we observed that among the 19 substances comprising the selected mixture, ∑DDT, ∑PCBi, and arsenic were main joint contributors to the risk of neurodevelopmental and thyroid effects for infants. In addition, ∑PCCD/F contributed to the thyroid effect and ∑aldrin-dieldrin to the neurodevelopmental effect. Our case study on contaminants in breast milk demonstrated the importance of crossing regulatory silos when studying mixtures and the importance of identifying risk drivers to regulate the risk related to environmental contamination. Applying this method to another set of data, such as human biomonitoring or in ecotoxicology, will reinforce its relevance for risk assessment. https://doi.org/10.1289/EHP8262.

It's not what you say, it's how you say it: A retrospective study of the impact of prosody on own-name P300 in comatose patients.

OBJECTIVE: The acoustic characteristics of stimuli influence the characteristics of the corresponding evoked potentials in healthy subjects. Own-name stimuli are used in clinical practice to assess the level of consciousness in intensive care units. The influence of the acoustic variability of these stimuli has never been evaluated. Here, we explored the influence of this variability on the characteristics of the subject's own name (SON) P300. METHODS: We retrospectively analyzed 251 disorders of consciousness patients from Lyon and Paris Hospitals who underwent an "own-name protocol". A reverse correlation analysis was performed to test for an association between acoustic properties of own-names stimuli used and the characteristics of the P300 wave observed. RESULTS: Own-names pronounced with increasing pitch prosody showed P300 responses 66 ms earlier than own-names that had a decreasing prosody [IC95% = 6.36; 125.9 ms]. CONCLUSIONS: Speech prosody of the stimuli in the "own name protocol" is associated with latencies differences of the P300 response among patients for whom these responses were observed. Further investigations are needed to confirm these results. SIGNIFICANCE: Speech prosody of the stimuli in the "own name protocol" is a non-negligible parameter, associated with P300 latency differences. Speech prosody should be standardized in SON P300 studies.

The shallow of your smile: the ethics of expressive vocal deep-fakes.

Rapid technological advances in artificial intelligence are creating opportunities for real-time algorithmic modulations of a person's facial and vocal expressions, or 'deep-fakes'. These developments raise unprecedented societal and ethical questions which, despite much recent public awareness, are still poorly understood from the point of view of moral psychology. We report here on an experimental ethics study conducted on a sample of N = 303 participants (predominantly young, western and educated), who evaluated the acceptability of vignettes describing potential applications of expressive voice transformation technology. We found that vocal deep-fakes were generally well accepted in the population, notably in a therapeutic context and for emotions judged otherwise difficult to control, and surprisingly, even if the user lies to their interlocutors about using them. Unlike other emerging technologies like autonomous vehicles, there was no evidence of social dilemma in which one would, for example, accept for others what they resent for themselves. The only real obstacle to the massive deployment of vocal deep-fakes appears to be situations where they are applied to a speaker without their knowing, but even the acceptability of such situations was modulated by individual differences in moral values and attitude towards science fiction. This article is part of the theme issue 'Voice modulation: from origin and mechanism to social impact (Part II)'.

Facial mimicry in the congenitally blind.

Imitation is one of the core building blocks of human social cognition, supporting capacities as diverse as empathy, social learning, and knowledge acquisition1. Newborns' ability to match others' motor acts, while quite limited initially, drastically improves during the first months of development2. Of notable importance to human sociality is our tendency to rapidly mimic facial expressions of emotion. Facial mimicry develops around six months of age3, but because of its late emergence, the factors supporting its development are relatively unknown. One possibility is that the development of facial mimicry depends on seeing emotional imitative behavior in others4. Alternatively, the drive to imitate facial expressions of emotion may be independent of visual learning and be supported by modality-general processes. Here we report evidence for the latter, by showing that congenitally blind participants facially imitate smiles heard in speech, despite having never seen a facial expression.

Quantification of SARS-CoV-2 neutralizing antibody by wild-type plaque reduction neutralization, microneutralization and pseudotyped virus neutralization assays.

Virus neutralization assays measure neutralizing antibodies in serum and plasma, and the plaque reduction neutralization test (PRNT) is considered the gold standard for measuring levels of these antibodies for many viral diseases. We have developed procedures for the standard PRNT, microneutralization assay (MNA) and pseudotyped virus neutralization assay (PNA) for severe acute respiratory syndrome coronavirus 2. The MNA offers advantages over the PRNT by reducing assay time, allowing increased throughput and reducing operator workload while remaining dependent upon the use of wild-type virus. This ensures that all severe acute respiratory syndrome coronavirus 2 antigens are present, but Biosafety Level 3 facilities are required. In addition to the advantages of MNA, PNA can be performed with lower biocontainment (Biosafety Level 2 facilities) and allows for further increases in throughput. For each new vaccine, it is critical to ensure good correlation of the neutralizing activity measured using PNA against the PRNT or MNA. These assays have been used in the development and licensure of the ChAdOx1 nCoV-19 (AstraZeneca; Oxford University) and Ad26.COV2.S (Janssen) coronavirus disease 2019 vaccines and are critical for demonstrating bioequivalence of future vaccines.

Distinct signatures of subjective confidence and objective accuracy in speech prosody.

Whether speech prosody truly and naturally reflects a speaker's subjective confidence, rather than other dimensions such as objective accuracy, is unclear. Here, using a new approach combing psychophysics with acoustic analysis and automatic classification of verbal reports, we tease apart the contributions of sensory evidence, accuracy, and subjective confidence to speech prosody. We find that subjective confidence and objective accuracy are distinctly reflected in the loudness, duration and intonation of verbal reports. Strikingly, we show that a speaker's accuracy is encoded in speech prosody beyond their own metacognitive awareness, and that it can be automatically decoded from this information alone with performances up to 60%. These findings demonstrate that confidence and accuracy have separable prosodic signatures that are manifested with different timings, and on different acoustic dimensions. Thus, both subjective mental states of confidence, and objective states related to competence, can be directly inferred from this natural behavior.

Listeners' perceptions of the certainty and honesty of a speaker are associated with a common prosodic signature.

The success of human cooperation crucially depends on mechanisms enabling individuals to detect unreliability in their conspecifics. Yet, how such epistemic vigilance is achieved from naturalistic sensory inputs remains unclear. Here we show that listeners' perceptions of the certainty and honesty of other speakers from their speech are based on a common prosodic signature. Using a data-driven method, we separately decode the prosodic features driving listeners' perceptions of a speaker's certainty and honesty across pitch, duration and loudness. We find that these two kinds of judgments rely on a common prosodic signature that is perceived independently from individuals' conceptual knowledge and native language. Finally, we show that listeners extract this prosodic signature automatically, and that this impacts the way they memorize spoken words. These findings shed light on a unique auditory adaptation that enables human listeners to quickly detect and react to unreliability during linguistic interactions.

Emergent Shared Intentions Support Coordination During Collective Musical Improvisations.

Human interactions are often improvised rather than scripted, which suggests that efficient coordination can emerge even when collective plans are largely underspecified. One possibility is that such forms of coordination primarily rely on mutual influences between interactive partners, and on perception-action couplings such as entrainment or mimicry. Yet some forms of improvised joint actions appear difficult to explain solely by appealing to these emergent mechanisms. Here, we focus on collective free improvisation, a form of highly unplanned creative practice where both agents' subjective reports and the complexity of their interactions suggest that shared intentions may sometimes emerge to support coordination during the course of the improvisation, even in the absence of verbal communication. In four experiments, we show that shared intentions spontaneously emerge during collective musical improvisations, and that they foster coordination on multiple levels, over and beyond the mere influence of shared information. We also show that musicians deploy communicative strategies to manifest and propagate their intentions within the group, and that this predicts better coordination. Overall, our results suggest that improvised and scripted joint actions are more continuous with one another than it first seems, and that they differ merely in the extent to which they rely on emergent or planned coordination mechanisms.

Vocal signals only impact speakers' own emotions when they are self-attributed.

Emotions are often accompanied by vocalizations whose acoustic features provide information about the physiological state of the speaker. Here, we ask if perceiving these affective signals in one's own voice has an impact on one's own emotional state, and if it is necessary to identify these signals as self-originated for the emotional effect to occur. Participants had to deliberate out loud about how they would feel in various familiar emotional scenarios, while we covertly manipulated their voices in order to make them sound happy or sad. Perceiving the artificial affective signals in their own voice altered participants' judgements about how they would feel in these situations. Crucially, this effect disappeared when participants detected the vocal manipulation, either explicitly or implicitly. The original valence of the scenarios also modulated the vocal feedback effect. These results highlight the role of the exteroception of self-attributed affective signals in the emergence of emotional feelings.

Dietary exposure to mycotoxins in the French infant total diet study.

The present study evaluated the exposure of children aged from one to 36 months to seven groups of mycotoxins, in the context of the infant French Total Diet Study (iTDS). Exposure was then compared to the health-based guidance values (HBGVs) for each mycotoxin. The value of the 90th percentile of exposure to nivalenol, patulin, fumonisins and zearalenone was less than 40% of the HBGV considered relevant for children. On the other hand, a risk could not be excluded for ochratoxin A and aflatoxins as exposure was close to the HBGV for ochratoxin A and the margin of exposure was much lower than the critical margin of 10,000 for aflatoxins. The HBGVs for toxins T2 and HT2, and for deoxynivalenol (DON) and its acetylated compounds were exceeded. Five percent to 10% of the children aged 5-12 months exceeded the HBGV considering the lower bound hypothesis for toxins T2 and HT2 and 7.5%-27% of the children aged 5 months and above exceeded the HBGV for DON. Consequently, the exposure of young children raises safety concerns for T2/HT2 and DON. Efforts should therefore be pursued to decrease their exposure to these molecules.

Health risk assessment to dioxins, furans and PCBs in young children: The first French evaluation.

A total diet study (TDS) was conducted between 2010 and 2016 to characterize the health risk related to chemical residues in food of French not breastfed children under three years of age (infant TDS). Among the targeted substances, polychlorinated dibenzo-p-dioxins (PCDDs), polychlorinated dibenzofurans (PCDFs) and polychlorinated biphenyls (PCBs) have been characterized as they accumulate through the food chain, especially in lipid-rich food items, and because they have been associated with a number of adverse effects in humans. Food samples (n = 180) were collected to be representative of the dioxins and PCB exposure through the whole diet of non-breastfed children from 1 to 36 months old and prepared as consumed (including cooking) prior to analysis. Dietary exposure was then assessed for 705 representative children under 3 years of age based on their food consumptions recorded through a 3-consecutive-days record. Levels of PCDD/Fs and PCBs in infant food were lower than those observed in common food, leading to significant differences in exposure according to age groups. Mean exposures to PCDD/Fs ranged from 0.22 to 0.44 pg TEQWHO05.kg bw-1.d-1 (0.40-0.65 at the 90th percentile), depending on the age group and the hypothesis considered to manage left-censored data. Mean exposure to non-dioxin-like PCBs ranged from 0.87 ng kg bw-1.d-1 (1.55 at the 90th percentile) in the 1-4 months old children to 3.53 ng kg bw-1.d-1 (5.44 at the 90th percentile) in the 13-36 months old children. For dioxins and NDL-PCBs, the tolerable daily intake (TDI) was exceeded for some age groups, in particular for older ones. Therefore, appropriate management measures must continue for reducing exposure; it concerns mainly common milk in youngest children, ultra-fresh dairy products and fish. For PCBs, recommendations on fish consumption should be reminded. Moreover, toxicity studies focusing on mixtures of dioxin-like compounds should be encouraged in order to take into account effect of mixtures.

Neural entrainment to music is sensitive to melodic spectral complexity.

During auditory perception, neural oscillations are known to entrain to acoustic dynamics but their role in the processing of auditory information remains unclear. As a complex temporal structure that can be parameterized acoustically, music is particularly suited to address this issue. In a combined behavioral and EEG experiment in human participants, we investigated the relative contribution of temporal (acoustic dynamics) and nontemporal (melodic spectral complexity) dimensions of stimulation on neural entrainment, a stimulus-brain coupling phenomenon operationally defined here as the temporal coherence between acoustical and neural dynamics. We first highlight that low-frequency neural oscillations robustly entrain to complex acoustic temporal modulations, which underscores the fine-grained nature of this coupling mechanism. We also reveal that enhancing melodic spectral complexity, in terms of pitch, harmony, and pitch variation, increases neural entrainment. Importantly, this manipulation enhances activity in the theta (5 Hz) range, a frequency-selective effect independent of the note rate of the melodies, which may reflect internal temporal constraints of the neural processes involved. Moreover, while both emotional arousal ratings and neural entrainment were positively modulated by spectral complexity, no direct relationship between arousal and neural entrainment was observed. Overall, these results indicate that neural entrainment to music is sensitive to the spectral content of auditory information and indexes an auditory level of processing that should be distinguished from higher-order emotional processing stages.NEW & NOTEWORTHY Low-frequency (<10 Hz) cortical neural oscillations are known to entrain to acoustic dynamics, the so-called neural entrainment phenomenon, but their functional implication in the processing of auditory information remains unclear. In a behavioral and EEG experiment capitalizing on parameterized musical textures, we disentangle the contribution of stimulus dynamics, melodic spectral complexity, and emotional judgments on neural entrainment and highlight their respective spatial and spectral neural signature.