The effect of visual speech information on linguistic release from masking.

Listeners often experience challenges understanding a person (target) in the presence of competing talkers (maskers). This difficulty reduces with the availability of visual speech information (VSI; lip movements, degree of mouth opening) and during linguistic release from masking (LRM; masking decreases with dissimilar language maskers). We investigate whether and how LRM occurs with VSI. We presented English targets with either Dutch or English maskers in audio-only and audiovisual conditions to 62 American English participants. The signal-to-noise ratio (SNR) was easy at 0 audio-only and -8 dB audiovisual in Experiment 1 and hard at -8 and -16 dB in Experiment 2 to assess the effects of modality on LRM across the same and different SNRs. We found LRM in the audiovisual condition for all SNRs and in audio-only for -8 dB, demonstrating reliable LRM for audiovisual conditions. Results also revealed that LRM is modulated by modality with larger LRM in audio-only indicating that introducing VSI weakens LRM. Furthermore, participants showed higher performance for Dutch maskers compared to English maskers with and without VSI. This establishes that listeners use both VSI and dissimilar language maskers to overcome masking. Our study shows that LRM persists in the audiovisual modality and its strength depends on the modality.

Would you respect a norm if it sounds foreign? Foreign-accented speech affects decision-making processes.

Does listening to a foreign-accented speaker bias native speakers' behavior? We investigated whether the accent, i.e., a foreign accent versus a native accent, in which a social norm is presented affects native speakers' decision to respect the norm (Experiments 1 and 2) and the judgement for not respecting it (Experiment 2). In Experiment 1, we presented 128 native Spanish speakers with new social norms, adapted from the measures imposed by the Spanish Government to fight the Covid-19 pandemic (e.g., 'To avoid the spread of the Covid-19 virus, keep your distance'), whereas in Experiment 2, we presented 240 native Spanish speakers with everyday social norms learned from childhood (e.g., 'Not littering on the street or in public places'), that have an intrinsic cultural and linguistic link. In Experiment 1, the norms were uttered either in a native accent, or in a foreign accent unfamiliar to our participants to avoid stereotypes. In Experiment 2, we added an accent negatively perceived in Spain to assess the role of language attitudes on decision making. Overall, accent did not directly impact participants' final decisions, but it influenced the decision-making process. The factors that seem to underlie this effect are emotionality and language attitudes. These findings add up to the recent Foreign Accent effect observed on moral judgements and further highlight the role of the speaker's identity in decision making.

The Effects of Being Informed About Chemotherapy-Related Cognitive Symptoms With And Without Self-Affirmation on Perceived Cognitive Symptoms of Breast Cancer Patients: A Randomized Prospective, Longitudinal Study.

BACKGROUND: Informing patients about chemotherapy-related cognitive symptoms (CRCS) may increase perceived cognitive symptoms. This longitudinal randomized study evaluated this Adverse Information Effect (AIE) in breast cancer patients and examined whether self-affirmation (SA) can reduce AIEs (ClinicalTrials.gov identifier: NCT04813965). PATIENTS AND METHODS: Before (neo) adjuvant chemotherapy, 160 newly diagnosed breast cancer patients were randomly allocated to receive: standard information on side-effects (control), standard information with additional information about CRCS (information), or standard and additional information with a subsequent self-affirmative text (information+SA). Online-questionnaires assessed the perceived frequency (MOS-cog) and severity (MDASI-cog) of cognitive symptoms before chemotherapy (baseline, T0), and 2.5-months (T1) and 6.5-months (T2) post-chemotherapy. Higher scores indicate less frequent, and more severe symptoms, respectively. Baseline-to-follow-up analyses using a mixed-effects modeling approach compared groups over time. RESULTS: At T0-T2, 148, 140 and 133 patients responded, respectively (attrition rates: 8%, 5%, 5%). Frequency (ES = -0.36, P =.003) and severity (ES = 0.54, P <.001) of symptoms worsened from baseline to T1, without differences between groups. At T2, symptom frequency remained stable for informed (ES=-0.3, P =.021) and self-affirmed (ES=-0.3, P =.019) patients, but returned to baseline levels for controls. At T2, symptom severity remained increased for informed patients (ES = 0.3, P =.006), but normalized for self-affirmed patients (ES = 0.2, P =.178) and controls. CONCLUSION: No AIEs occurred until T2. The initial overall increase in perceived cognitive symptoms recovered at T2 for controls, but not for patients who received additional information about CRCS. Self-affirmation attenuated these longer-term AIEs for the perceived severity but not the frequency of symptoms.

The Effects of Iconic Gestures and Babble Language on Word Intelligibility in Sentence Context.

PURPOSE: This study investigated to what extent iconic co-speech gestures help word intelligibility in sentence context in two different linguistic maskers (native vs. foreign). It was hypothesized that sentence recognition improves with the presence of iconic co-speech gestures and with foreign compared to native babble. METHOD: Thirty-two native Dutch participants performed a Dutch word recognition task in context in which they were presented with videos in which an actress uttered short Dutch sentences (e.g., Ze begint te openen, "She starts to open"). Participants were presented with a total of six audiovisual conditions: no background noise (i.e., clear condition) without gesture, no background noise with gesture, French babble without gesture, French babble with gesture, Dutch babble without gesture, and Dutch babble with gesture; and they were asked to type down what was said by the Dutch actress. The accurate identification of the action verbs at the end of the target sentences was measured. RESULTS: The results demonstrated that performance on the task was better in the gesture compared to the nongesture conditions (i.e., gesture enhancement effect). In addition, performance was better in French babble than in Dutch babble. CONCLUSIONS: Listeners benefit from iconic co-speech gestures during communication and from foreign background speech compared to native. These insights into multimodal communication may be valuable to everyone who engages in multimodal communication and especially to a public who often works in public places where competing speech is present in the background.

"Lass frooby noo!" the interference of song lyrics and meaning on speech intelligibility.

This study examined whether song lyrics and their semantic meaning interfere with speech intelligibility. In three experiments, a total of 108 native Dutch participants listened to Dutch target sentences in the presence of three versions of the pop songs Last Friday Night (T.G.I.F.) (Experiment 1) or Hot N Cold (Experiment 2a and 2b) by singer Katy Perry at different signal-to-noise ratios. The versions consisted of the original English songs, the karaoke versions of the songs without lyrics, and anomalous versions of the songs in the fictional language Simlish, which was created for the video game The Sims. The songs were played in chronological (Experiments 1 and 2a) or in random order (Experiment 2b). Participants' task was to type the target sentence they had heard. In all experiments, speech intelligibility was better in nonlyrical (karaoke) than lyrical music (English and Simlish). In addition, listeners performed better in lyrics without semantic meaning (Simlish) than with semantic meaning (English). Finally, speech intelligibility was better when the song in the background was played in chronological rather than in random order. These findings aid in understanding the mechanisms involved during speech-in-music intelligibility. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Is There a Foreign Accent Effect on Moral Judgment?

Recent studies have shown that people make more utilitarian decisions when dealing with a moral dilemma in a foreign language than in their native language. Emotion, cognitive load, and psychological distance have been put forward as explanations for this foreign language effect. The question that arises is whether a similar effect would be observed when processing a dilemma in one's own language but spoken by a foreign-accented speaker. Indeed, foreign-accented speech has been shown to modulate emotion processing, to disrupt processing fluency and to increase psychological distance due to social categorisation. We tested this hypothesis by presenting 435 participants with two moral dilemmas, the trolley dilemma and the footbridge dilemma online, either in a native accent or a foreign accent. In Experiment 1, 184 native Spanish speakers listened to the dilemmas in Spanish recorded by a native speaker, a British English or a Cameroonian native speaker. In Experiment 2, 251 Dutch native speakers listened to the dilemmas in Dutch in their native accent, in a British English, a Turkish, or in a French accent. Results showed an increase in utilitarian decisions for the Cameroonian- and French-accented speech compared to the Spanish or Dutch native accent, respectively. When collapsing all the speakers from the two experiments, a similar increase in the foreign accent condition compared with the native accent condition was observed. This study is the first demonstration of a foreign accent effect on moral judgements, and despite the variability in the effect across accents, the findings suggest that a foreign accent, like a foreign language, is a linguistic context that modulates (neuro)cognitive mechanisms, and consequently, impacts our behaviour. More research is needed to follow up on this exploratory study and to understand the influence of factors such as emotion reduction, cognitive load, psychological distance, and speaker's idiosyncratic features on moral judgments.

Individual Differences in Children's (Language) Learning Skills Moderate Effects of Robot-Assisted Second Language Learning.

The current study investigated how individual differences among children affect the added value of social robots for teaching second language (L2) vocabulary to young children. Specifically, we investigated the moderating role of three individual child characteristics deemed relevant for language learning: first language (L1) vocabulary knowledge, phonological memory, and selective attention. We expected children low in these abilities to particularly benefit from being assisted by a robot in a vocabulary training. An L2 English vocabulary training intervention consisting of seven sessions was administered to 193 monolingual Dutch five-year-old children over a three- to four-week period. Children were randomly assigned to one of three experimental conditions: 1) a tablet only, 2) a tablet and a robot that used deictic (pointing) gestures (the no-iconic-gestures condition), or 3) a tablet and a robot that used both deictic and iconic gestures (i.e., gestures depicting the target word; the iconic-gestures condition). There also was a control condition in which children did not receive a vocabulary training, but played dancing games with the robot. L2 word knowledge was measured directly after the training and two to four weeks later. In these post-tests, children in the experimental conditions outperformed children in the control condition on word knowledge, but there were no differences between the three experimental conditions. Several moderation effects were found. The robot's presence particularly benefited children with larger L1 vocabularies or poorer phonological memory, while children with smaller L1 vocabularies or better phonological memory performed better in the tablet-only condition. Children with larger L1 vocabularies and better phonological memory performed better in the no-iconic-gestures condition than in the iconic-gestures condition, while children with better selective attention performed better in the iconic-gestures condition than the no-iconic-gestures condition. Together, the results showed that the effects of the robot and its gestures differ across children, which should be taken into account when designing and evaluating robot-assisted L2 teaching interventions.

Word Recognition and Word Prediction in Preschoolers With (a Suspicion of) a Developmental Language Disorder: Evidence From Eye Tracking.

Purpose This study compares online word recognition and prediction in preschoolers with (a suspicion of) a developmental language disorder (DLD) and typically developing (TD) controls. Furthermore, it investigates correlations between these measures and the link between online and off-line language scores in the DLD group. Method Using the visual world paradigm, Dutch children ages 3;6 (years;months) with (a suspicion of) DLD (n = 51) and TD peers (n = 31) listened to utterances such as, "Kijk, een hoed!" (Look, a hat!) in a word recognition task, and sentences such as, "Hé, hij leest gewoon een boek" (literal translation: Hey, he reads just a book) in a word prediction task, while watching a target and distractor picture. Results Both groups demonstrated a significant word recognition effect that looked similar directly after target onset. However, the DLD group looked longer at the target than the TD group and shifted slower from the distractor to target pictures. Within the DLD group, word recognition was linked to off-line expressive language scores. For word prediction, the DLD group showed a smaller effect and slower shifts from verb onset compared to the TD group. Interestingly, within the DLD group, prediction behavior varied considerably, and was linked to receptive and expressive language scores. Finally, slower shifts in word recognition were related to smaller prediction effects. Conclusions While the groups' word recognition abilities looked similar, and only differed in processing speed and dwell time, the DLD group showed atypical verb-based prediction behavior. This may be due to limitations in their processing capacity and/or their linguistic knowledge, in particular of verb argument structure.

A Tale of Two Modalities: Sign and Speech Influence Each Other in Bimodal Bilinguals.

Bimodal bilinguals are hearing individuals fluent in a sign and a spoken language. Can the two languages influence each other in such individuals despite differences in the visual (sign) and vocal (speech) modalities of expression? We investigated cross-linguistic influences on bimodal bilinguals' expression of spatial relations. Unlike spoken languages, sign uses iconic linguistic forms that resemble physical features of objects in a spatial relation and thus expresses specific semantic information. Hearing bimodal bilinguals (n = 21) fluent in Dutch and Sign Language of the Netherlands and their hearing nonsigning and deaf signing peers (n = 20 each) described left/right relations between two objects. Bimodal bilinguals expressed more specific information about physical features of objects in speech than nonsigners, showing influence from sign language. They also used fewer iconic signs with specific semantic information than deaf signers, demonstrating influence from speech. Bimodal bilinguals' speech and signs are shaped by two languages from different modalities.

Iconicity in spatial language guides visual attention: A comparison between signers' and speakers' eye gaze during message preparation.

To talk about space, spoken languages rely on arbitrary and categorical forms (e.g., left, right). In sign languages, however, the visual-spatial modality allows for iconic encodings (motivated form-meaning mappings) of space in which form and location of the hands bear resemblance to the objects and spatial relations depicted. We assessed whether the iconic encodings in sign languages guide visual attention to spatial relations differently than spatial encodings in spoken languages during message preparation at the sentence level. Using a visual world production eye-tracking paradigm, we compared 20 deaf native signers of Sign-Language-of-the-Netherlands and 20 Dutch speakers' visual attention to describe left versus right configurations of objects (e.g., "pen is to the left/right of cup"). Participants viewed 4-picture displays in which each picture contained the same 2 objects but in different spatial relations (lateral [left/right], sagittal [front/behind], topological [in/on]) to each other. They described the target picture (left/right) highlighted by an arrow. During message preparation, signers, but not speakers, experienced increasing eye-gaze competition from other spatial configurations. This effect was absent during picture viewing prior to message preparation of relational encoding. Moreover, signers' visual attention to lateral and/or sagittal relations was predicted by the type of iconicity (i.e., object and space resemblance vs. space resemblance only) in their spatial descriptions. Findings are discussed in relation to how "thinking for speaking" differs from "thinking for signing" and how iconicity can mediate the link between language and human experience and guides signers' but not speakers' attention to visual aspects of the world. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

The role of foreign accent and short-term exposure in speech-in-speech recognition.

Daily speech communication often takes place in suboptimal listening conditions, in which interlocutors typically need to segregate the target signal from the background sounds. The present study investigated the influence on speech recognition of a relatively familiar foreign accent in background speech (Exp. 1) and whether short-term immediate exposure to the target talker's voice (Exp. 2) or the background babble (Exp. 3) would either help or hinder the segregation of target from background. A total of 72 native Dutch participants were asked to listen to Dutch target sentences in the presence of Dutch or German-accented Dutch babble without (Exp. 1) or with (Exps. 2 and 3) an exposure phase. Their task was to write down what they heard. The results of Experiment 1 revealed that listeners gained a release from masking when the background speech was accented, indicating that dissimilar and less familiar signals are easier to segregate effectively. Experiment 2 demonstrated that short-term immediate exposure to the target talker had no effect on speech-in-speech recognition, whereas exposure to the background babble could hinder separating the target voice from the background speech (Exp. 3). However, this reduced release from masking only appeared in the more difficult and more familiar babble condition (Dutch in Dutch), in which the speech recognition system may have remained attuned to the babble as a potential source of communicatively relevant information. Overall, this research provides evidence that both short-term adaptation and the degrees of target-background similarity and familiarity are of importance for speech-in-speech recognition.

Verb-based prediction during language processing: the case of Dutch and Turkish.

This study investigated whether cross-linguistic differences affect semantic prediction. We assessed this by looking at two languages, Dutch and Turkish, that differ in word order and thus vary in how words come together to create sentence meaning. In an eye-tracking task, Dutch and Turkish four-year-olds (N = 40), five-year-olds (N = 58), and adults (N = 40) were presented with a visual display containing two familiar objects (e.g., a cake and a tree). Participants heard semantically constraining (e.g., "The boy eats the big cake") or neutral sentences (e.g., "The boy sees the big cake") in their native language. The Dutch data revealed a prediction effect for children and adults; however, it was larger for the adults. The Turkish data revealed no prediction effect for the children but only for the adults. These findings reveal that experience with word order structures and/or automatization of language processing routines may lead to timecourse differences in semantic prediction.

No perceptual reorganization for Limburgian tones? A cross-linguistic investigation with 6- to 12-month-old infants.

Despite the fact that many of the world's languages use lexical tone, the majority of language acquisition studies has focused on non-tone languages. Research on tone languages has typically investigated well-known tone languages such as Mandarin and Cantonese Chinese. The current study looked at a Limburgian dialect of Dutch that uses lexical pitch differences, albeit in a rather restricted way. Using a visual habituation paradigm, 6- to 12-month-old Limburgian and Dutch infants were tested for their ability to discriminate Limburgian tones. The results showed that both Limburgian and Dutch infants discriminate the Limburgian tones throughout their first year of life. The role of linguistic experience, acoustic salience, and the degree of similarity to the native prosodic system are discussed.

How Native Prosody Affects Pitch Processing during Word Learning in Limburgian and Dutch Toddlers and Adults.

In this study, Limburgian and Dutch 2.5- to 4-year-olds and adults took part in a word learning experiment. Following the procedure employed by Quam and Swingley (2010) and Singh et al. (2014), participants learned two novel word-object mappings. After training, word recognition was tested in correct pronunciation (CP) trials and mispronunciation (MP) trials featuring a pitch change. Since Limburgian is considered a restricted tone language, we expected that the pitch change would hinder word recognition in Limburgian, but not in non-tonal Dutch listeners. Contrary to our expectations, both Limburgian and Dutch children appeared to be sensitive to pitch changes in newly learned words, indicated by a significant decrease in target fixation in MP trials compared to CP trials. Limburgian and Dutch adults showed very strong naming effects in both trial types. The results are discussed against the background of the influence of the native prosodic system.

Masking release effects of a standard and a regional linguistic variety.

Previous research has shown that the more similar the target and the masker signal, the harder it is to segregate the two streams effectively [i.e., target-masker linguistic similarity hypothesis, e.g., Brouwer, Van Engen, Calandruccio, and Bradlow (2012). J. Acoust. Soc. Am. 131(2), 1449-1464]. The present study examined whether this hypothesis holds when a standard variety of a language (Dutch) is paired with one of its regional varieties (Limburgian). Dutch and Limburgian listeners were tested on a speech-in-speech recognition task to investigate whether familiarity with the target and/or maskers influenced their performance. The findings provide support for the hypothesis and suggest an influence of Limburgians' bidialectal status.

Processing grammatical gender in Dutch: Evidence from eye movements.

Previous research has demonstrated that grammatical gender in Dutch is typically acquired late. Most of this work used production data only, and consequently children's knowledge of Dutch gender may have been underestimated. In this study, therefore, we examined whether 49 4- to 7-year-old Dutch-speaking children (and 19 adult controls) were able to use gender marking in the article preceding the object label during online sentence processing to (a) anticipate the upcoming object label or to (b) facilitate the processing of that label as it is presented. In addition, we investigated whether children's online processing and production of gender marking on articles were related. In an eye-tracking task, participants were presented with sentences and visual displays with two objects, representing nouns of either the same gender (uninformative) or different genders (informative). Children were divided into a non-targetlike group and a targetlike group on the basis of their scores for neuter nouns in the production task. Our analyses examined whether participants could use gender marking anticipatorily (i.e., before the onset of the noun) and facilitatively (i.e., from noun onset). Results showed that Dutch-speaking adults and children who were successful in production used gender marking anticipatorily. However, children who did not systematically produce gender-marked articles used gender marking only facilitatively. These findings reveal that successful online comprehension may in part be possible before targetlike production is completely in place, but at the same time targetlike production may be a trigger for online comprehension to be completely successful.

The Temporal Dynamics of Spoken Word Recognition in Adverse Listening Conditions.

This study examined the temporal dynamics of spoken word recognition in noise and background speech. In two visual-world experiments, English participants listened to target words while looking at four pictures on the screen: a target (e.g. candle), an onset competitor (e.g. candy), a rhyme competitor (e.g. sandal), and an unrelated distractor (e.g. lemon). Target words were presented in quiet, mixed with broadband noise, or mixed with background speech. Results showed that lexical competition changes throughout the observation window as a function of what is presented in the background. These findings suggest that, rather than being strictly sequential, stream segregation and lexical competition interact during spoken word recognition.

Delayed Anticipatory Spoken Language Processing in Adults with Dyslexia­Evidence from Eye-tracking.

It is now well established that anticipation of upcoming input is a key characteristic of spoken language comprehension. It has also frequently been observed that literacy influences spoken language processing. Here, we investigated whether anticipatory spoken language processing is related to individuals' word reading abilities. Dutch adults with dyslexia and a control group participated in two eye-tracking experiments. Experiment 1 was conducted to assess whether adults with dyslexia show the typical language-mediated eye gaze patterns. Eye movements of both adults with and without dyslexia closely replicated earlier research: spoken language is used to direct attention to relevant objects in the environment in a closely time-locked manner. In Experiment 2, participants received instructions (e.g., 'Kijk naar de(COM) afgebeelde piano(COM)', look at the displayed piano) while viewing four objects. Articles (Dutch 'het' or 'de') were gender marked such that the article agreed in gender only with the target, and thus, participants could use gender information from the article to predict the target object. The adults with dyslexia anticipated the target objects but much later than the controls. Moreover, participants' word reading scores correlated positively with their anticipatory eye movements. We conclude by discussing the mechanisms by which reading abilities may influence predictive language processing.

Interdependent processing and encoding of speech and concurrent background noise.

Speech processing can often take place in adverse listening conditions that involve the mixing of speech and background noise. In this study, we investigated processing dependencies between background noise and indexical speech features, using a speeded classification paradigm (Garner, 1974; Exp. 1), and whether background noise is encoded and represented in memory for spoken words in a continuous recognition memory paradigm (Exp. 2). Whether or not the noise spectrally overlapped with the speech signal was also manipulated. The results of Experiment 1 indicated that background noise and indexical features of speech (gender, talker identity) cannot be completely segregated during processing, even when the two auditory streams are spectrally nonoverlapping. Perceptual interference was asymmetric, whereby irrelevant indexical feature variation in the speech signal slowed noise classification to a greater extent than irrelevant noise variation slowed speech classification. This asymmetry may stem from the fact that speech features have greater functional relevance to listeners, and are thus more difficult to selectively ignore than background noise. Experiment 2 revealed that a recognition cost for words embedded in different types of background noise on the first and second occurrences only emerged when the noise and the speech signal were spectrally overlapping. Together, these data suggest integral processing of speech and background noise, modulated by the level of processing and the spectral separation of the speech and noise.

Contextual variability during speech-in-speech recognition.

This study examined the influence of background language variation on speech recognition. English listeners performed an English sentence recognition task in either "pure" background conditions in which all trials had either English or Dutch background babble or in mixed background conditions in which the background language varied across trials (i.e., a mix of English and Dutch or one of these background languages mixed with quiet trials). This design allowed the authors to compare performance on identical trials across pure and mixed conditions. The data reveal that speech-in-speech recognition is sensitive to contextual variation in terms of the target-background language (mis)match depending on the relative ease/difficulty of the test trials in relation to the surrounding trials.