Early or synchronized gestures facilitate speech recall-a study based on motion capture data.

Introduction: Temporal co-ordination between speech and gestures has been thoroughly studied in natural production. In most cases gesture strokes precede or coincide with the stressed syllable in words that they are semantically associated with. Methods: To understand whether processing of speech and gestures is attuned to such temporal coordination, we investigated the effect of delaying, preposing or eliminating individual gestures on the memory for words in an experimental study in which 83 participants watched video sequences of naturalistic 3D-animated speakers generated based on motion capture data. A target word in the sequence appeared (a) with a gesture presented in its original position synchronized with speech, (b) temporally shifted 500 ms before or (c) after the original position, or (d) with the gesture eliminated. Participants were asked to retell the videos in a free recall task. The strength of recall was operationalized as the inclusion of the target word in the free recall. Results: Both eliminated and delayed gesture strokes resulted in reduced recall rates compared to synchronized strokes, whereas there was no difference between advanced (preposed) and synchronized strokes. An item-level analysis also showed that the greater the interval between the onsets of delayed strokes and stressed syllables in target words, the greater the negative effect was on recall. Discussion: These results indicate that speech-gesture synchrony affects memory for speech, and that temporal patterns that are common in production lead to the best recall. Importantly, the study also showcases a procedure for using motion capture-based 3D-animated speakers to create an experimental paradigm for the study of speech-gesture comprehension.

Development of an Auditory Passage Comprehension Task for Swedish Primary School Children of Cultural and Linguistic Diversity.

Purpose This study reports on the development of an auditory passage comprehension task for Swedish primary school children of cultural and linguistic diversity. It also reports on their performance on the task in quiet and in noise. Method Eighty-eight children aged 7-9 years and showing normal hearing participated. The children were divided into three groups based on presumed language exposure: 13 children were categorized as Swedish-speaking monolinguals, 19 children were categorized as simultaneous bilinguals, and 56 children were categorized as sequential bilinguals. No significant difference in working memory capacity was seen between the three language groups. Two passages and associated multiple-choice questions were developed. During development of the passage comprehension task, steps were taken to reduce the impact of culture-specific prior experience and knowledge on performance. This was achieved by using the story grammar principles, universal topics and plots, and simple language that avoided complex or unusual grammatical structures and words. Results The findings indicate no significant difference between the two passages and similar response distributions. Passage comprehension performance was significantly better in quiet than in noise, regardless of language exposure group. The monolinguals outperformed both simultaneous and sequential bilinguals in both listening conditions. Conclusions Because the task was designed to minimize the effect of cultural knowledge on auditory passage comprehension, this suggests that compared with monolinguals, both simultaneous and sequential bilinguals have a disadvantage in auditory passage comprehension. As expected, the findings demonstrate that noise has a negative effect on auditory passage comprehension. The magnitude of this effect does not relate to language exposure. The developed auditory passage comprehension task seems suitable for assessing auditory passage comprehension in primary school children of linguistic and cultural diversity.

Listening effort and fatigue in native and non-native primary school children.

Background noise makes listening effortful and may lead to fatigue. This may compromise classroom learning, especially for children with a non-native background. In the current study, we used pupillometry to investigate listening effort and fatigue during listening comprehension under typical (0 dB signal-to-noise ratio [SNR]) and favorable (+10 dB SNR) listening conditions in 63 Swedish primary school children (7-9 years of age) performing a narrative speech-picture verification task. Our sample comprised both native (n = 25) and non-native (n = 38) speakers of Swedish. Results revealed greater pupil dilation, indicating more listening effort, in the typical listening condition compared with the favorable listening condition, and it was primarily the non-native speakers who contributed to this effect (and who also had lower performance accuracy than the native speakers). Furthermore, the native speakers had greater pupil dilation during successful trials, whereas the non-native speakers showed greatest pupil dilation during unsuccessful trials, especially in the typical listening condition. This set of results indicates that whereas native speakers can apply listening effort to good effect, non-native speakers may have reached their effort ceiling, resulting in poorer listening comprehension. Finally, we found that baseline pupil size decreased over trials, which potentially indicates more listening-related fatigue, and this effect was greater in the typical listening condition compared with the favorable listening condition. Collectively, these results provide novel insight into the underlying dynamics of listening effort, fatigue, and listening comprehension in typical classroom conditions compared with favorable classroom conditions, and they demonstrate for the first time how sensitive this interplay is to language experience.

Motion capture-based animated characters for the study of speech-gesture integration.

Digitally animated characters are promising tools in research studying how we integrate information from speech and visual sources such as gestures because they allow specific gesture features to be manipulated in isolation. We present an approach combining motion capture and 3D-animated characters that allows us to manipulate natural individual gesture strokes for experimental purposes, for example to temporally shift and present gestures in ecologically valid sequences. We exemplify how such stimuli can be used in an experiment investigating implicit detection of speech-gesture (a) synchrony, and discuss the general applicability of the workflow for research in this domain.

The slower the better? Does the speaker's speech rate influence children's performance on a language comprehension test?

The aim of this study was to examine the effects of speech rate on children's performance on a widely used language comprehension test, the Test for Reception of Grammar, version 2 (TROG'2), and to explore how test performance interacts with task difficulty and with the child's working memory capacity. Participants were 102 typically-developing Swedish-speaking children randomly assigned to one of the three conditions; the TROG'2 sentences spoken by a speech-language pathologist with slow, normal or fast speech rate. Results showed that the fast speech rate had a negative effect on the TROG'2 scores and that slow rate was more beneficial in general. However, for more difficult tasks the beneficial effect of slow speech was only pronounced for children with better scores on a working memory task. The interpretation is that slow speech is particularly helpful when children do not yet fully master a task but are just about to grasp it. These results emphasize the necessity of careful considerations of the role dynamic aspects of examiner's speech might play in test administration and favour digitalized procedures in standardized language comprehension assessment.