Precursors to infant sensorimotor synchronization to speech and non-speech rhythms: A longitudinal study.

Impaired sensorimotor synchronization (SMS) to acoustic rhythm may be a marker of atypical language development. Here, Motion Capture was used to assess gross motor rhythmic movement at six time points between 5- and 11 months of age. Infants were recorded drumming to acoustic stimuli of varying linguistic and temporal complexity: drumbeats, repeated syllables and nursery rhymes. Here we show, for the first time, developmental change in infants' movement timing in response to auditory stimuli over the first year of life. Longitudinal analyses revealed that whilst infants could not yet reliably synchronize their movement to auditory rhythms, infant spontaneous motor tempo became faster with age, and by 11 months, a subset of infants decelerate from their spontaneous motor tempo, which better accords with the incoming tempo. Further, infants became more regular drummers with age, with marked decreases in the variability of spontaneous motor tempo and variability in response to drumbeats. This latter effect was subdued in response to linguistic stimuli. The current work lays the foundation for using individual differences in precursors of SMS in infancy to predict later language outcomes. RESEARCH HIGHLIGHT: We present the first longitudinal investigation of infant rhythmic movement over the first year of life Whilst infants generally move more quickly and with higher regularity over their first year, by 11 months infants begin to counter this pattern when hearing slower infant-directed song Infant movement is more variable to speech than non-speech stimuli In the context of the larger Cambridge UK BabyRhythm Project, we lay the foundation for rhythmic movement in infancy to predict later language outcomes.

Cortical tracking of visual rhythmic speech by 5- and 8-month-old infants: Individual differences in phase angle relate to language outcomes up to 2 years.

It is known that the rhythms of speech are visible on the face, accurately mirroring changes in the vocal tract. These low-frequency visual temporal movements are tightly correlated with speech output, and both visual speech (e.g., mouth motion) and the acoustic speech amplitude envelope entrain neural oscillations. Low-frequency visual temporal information ('visual prosody') is known from behavioural studies to be perceived by infants, but oscillatory studies are currently lacking. Here we measure cortical tracking of low-frequency visual temporal information by 5- and 8-month-old infants using a rhythmic speech paradigm (repetition of the syllable 'ta' at 2 Hz). Eye-tracking data were collected simultaneously with EEG, enabling computation of cortical tracking and phase angle during visual-only speech presentation. Significantly higher power at the stimulus frequency indicated that cortical tracking occurred across both ages. Further, individual differences in preferred phase to visual speech related to subsequent measures of language acquisition. The difference in phase between visual-only speech and the same speech presented as auditory-visual at 6- and 9-months was also examined. These neural data suggest that individual differences in early language acquisition may be related to the phase of entrainment to visual rhythmic input in infancy. RESEARCH HIGHLIGHTS: Infant preferred phase to visual rhythmic speech predicts language outcomes. Significant cortical tracking of visual speech is present at 5 and 8 months. Phase angle to visual speech at 8 months predicted greater receptive and productive vocabulary at 24 months.

Decoding speech information from EEG data with 4-, 7- and 11-month-old infants: Using convolutional neural network, mutual information-based and backward linear models.

BACKGROUND: Computational models that successfully decode neural activity into speech are increasing in the adult literature, with convolutional neural networks (CNNs), backward linear models, and mutual information (MI) models all being applied to neural data in relation to speech input. This is not the case in the infant literature. NEW METHOD: Three different computational models, two novel for infants, were applied to decode low-frequency speech envelope information. Previously-employed backward linear models were compared to novel CNN and MI-based models. Fifty infants provided EEG recordings when aged 4, 7, and 11 months, while listening passively to natural speech (sung or chanted nursery rhymes) presented by video with a female singer. RESULTS: Each model computed speech information for these nursery rhymes in two different low-frequency bands, delta and theta, thought to provide different types of linguistic information. All three models demonstrated significant levels of performance for delta-band neural activity from 4 months of age, with two of three models also showing significant performance for theta-band activity. All models also demonstrated higher accuracy for the delta-band neural responses. None of the models showed developmental (age-related) effects. COMPARISONS WITH EXISTING METHODS: The data demonstrate that the choice of algorithm used to decode speech envelope information from neural activity in the infant brain determines the developmental conclusions that can be drawn. CONCLUSIONS: The modelling shows that better understanding of the strengths and weaknesses of each modelling approach is fundamental to improving our understanding of how the human brain builds a language system.

Neural responses to natural and enhanced speech edges in children with and without dyslexia.

Sensory-neural studies indicate that children with developmental dyslexia show impairments in processing acoustic speech envelope information. Prior studies suggest that this arises in part from reduced sensory sensitivity to amplitude rise times (ARTs or speech "edges") in the envelope, accompanied by less accurate neural encoding of low-frequency envelope information. Accordingly, enhancing these characteristics of the speech envelope may enhance neural speech processing in children with dyslexia. Here we applied an envelope modulation enhancement (EME) algorithm to a 10-min story read in child-directed speech (CDS), enhancing ARTs and also enhancing low-frequency envelope information. We compared neural speech processing (as measured using MEG) for the EME story with the same story read in natural CDS for 9-year-old children with and without dyslexia. The EME story affected neural processing in the power domain for children with dyslexia, particularly in the delta band (0.5-4 Hz) in the superior temporal gyrus. This may suggest that prolonged experience with EME speech could ameliorate some of the impairments shown in natural speech processing by children with dyslexia.

Neural phase angle from two months when tracking speech and non-speech rhythm linked to language performance from 12 to 24 months.

Atypical phase alignment of low-frequency neural oscillations to speech rhythm has been implicated in phonological deficits in developmental dyslexia. Atypical phase alignment to rhythm could thus also characterize infants at risk for later language difficulties. Here, we investigate phase-language mechanisms in a neurotypical infant sample. 122 two-, six- and nine-month-old infants were played speech and non-speech rhythms while EEG was recorded in a longitudinal design. The phase of infants' neural oscillations aligned consistently to the stimuli, with group-level convergence towards a common phase. Individual low-frequency phase alignment related to subsequent measures of language acquisition up to 24 months of age. Accordingly, individual differences in language acquisition are related to the phase alignment of cortical tracking of auditory and audiovisual rhythms in infancy, an automatic neural mechanism. Automatic rhythmic phase-language mechanisms could eventually serve as biomarkers, identifying at-risk infants and enabling intervention at the earliest stages of development.

Decoding of speech information using EEG in children with dyslexia: Less accurate low-frequency representations of speech, not "Noisy" representations.

The amplitude envelope of speech carries crucial low-frequency acoustic information that assists linguistic decoding. The sensory-neural Temporal Sampling (TS) theory of developmental dyslexia proposes atypical encoding of speech envelope information < 10 Hz, leading to atypical phonological representations. Here a backward linear TRF model and story listening were employed to estimate the speech information encoded in the electroencephalogram in the canonical delta, theta and alpha bands by 9-year-old children with and without dyslexia. TRF decoding accuracy provided an estimate of how faithfully the children's brains encoded low-frequency envelope information. Between-group analyses showed that the children with dyslexia exhibited impaired reconstruction of speech information in the delta band. However, when the quality of speech encoding for each child was estimated using child-by-child decoding models, then the dyslexic children did not differ from controls. This suggests that children with dyslexia encode neither "noisy" nor "normal" representations of the speech signal, but different representations.

Local Temporal Regularities in Child-Directed Speech in Spanish.

PURPOSE: The purpose of this study is to characterize the local (utterance-level) temporal regularities of child-directed speech (CDS) that might facilitate phonological development in Spanish, classically termed a syllable-timed language. METHOD: Eighteen female adults addressed their 4-year-old children versus other adults spontaneously and also read aloud (CDS vs. adult-directed speech [ADS]). We compared CDS and ADS speech productions using a spectrotemporal model (Leong & Goswami, 2015), obtaining three temporal metrics: (a) distribution of modulation energy, (b) temporal regularity of stressed syllables, and (c) syllable rate. RESULTS: CDS was characterized by (a) significantly greater modulation energy in the lower frequencies (0.5-4 Hz), (b) more regular rhythmic occurrence of stressed syllables, and (c) a slower syllable rate than ADS, across both spontaneous and read conditions. DISCUSSION: CDS is characterized by a robust local temporal organization (i.e., within utterances) with amplitude modulation bands aligning with delta and theta electrophysiological frequency bands, respectively, showing greater phase synchronization than in ADS, facilitating parsing of stress units and syllables. These temporal regularities, together with the slower rate of production of CDS, might support the automatic extraction of phonological units in speech and hence support the phonological development of children. SUPPLEMENTAL MATERIAL: https://doi.org/10.23641/asha.21210893.

Cortical Tracking of Sung Speech in Adults vs Infants: A Developmental Analysis.

Here we duplicate a neural tracking paradigm, previously published with infants (aged 4 to 11 months), with adult participants, in order to explore potential developmental similarities and differences in entrainment. Adults listened and watched passively as nursery rhymes were sung or chanted in infant-directed speech. Whole-head EEG (128 channels) was recorded, and cortical tracking of the sung speech in the delta (0.5-4 Hz), theta (4-8 Hz) and alpha (8-12 Hz) frequency bands was computed using linear decoders (multivariate Temporal Response Function models, mTRFs). Phase-amplitude coupling (PAC) was also computed to assess whether delta and theta phases temporally organize higher-frequency amplitudes for adults in the same pattern as found in the infant brain. Similar to previous infant participants, the adults showed significant cortical tracking of the sung speech in both delta and theta bands. However, the frequencies associated with peaks in stimulus-induced spectral power (PSD) in the two populations were different. PAC was also different in the adults compared to the infants. PAC was stronger for theta- versus delta- driven coupling in adults but was equal for delta- versus theta-driven coupling in infants. Adults also showed a stimulus-induced increase in low alpha power that was absent in infants. This may suggest adult recruitment of other cognitive processes, possibly related to comprehension or attention. The comparative data suggest that while infant and adult brains utilize essentially the same cortical mechanisms to track linguistic input, the operation of and interplay between these mechanisms may change with age and language experience.

Neural sampling of the speech signal at different timescales by children with dyslexia.

Phonological difficulties characterize individuals with dyslexia across languages. Currently debated is whether these difficulties arise from atypical neural sampling of (or entrainment to) auditory information in speech at slow rates (<10 Hz, related to speech rhythm), faster rates, or neither. MEG studies with adults suggest that atypical sampling in dyslexia affects faster modulations in the neurophysiological gamma band, related to phoneme-level representation. However, dyslexic adults have had years of reduced experience in converting graphemes to phonemes, which could itself cause atypical gamma-band activity. The present study was designed to identify specific linguistic timescales at which English children with dyslexia may show atypical entrainment. Adopting a developmental focus, we hypothesized that children with dyslexia would show atypical entrainment to the prosodic and syllable-level information that is exaggerated in infant-directed speech and carried primarily by amplitude modulations <10 Hz. MEG was recorded in a naturalistic story-listening paradigm. The modulation bands related to different types of linguistic information were derived directly from the speech materials, and lagged coherence at multiple temporal rates spanning 0.9-40 Hz was computed. Group differences in lagged speech-brain coherence between children with dyslexia and control children were most marked in neurophysiological bands corresponding to stress and syllable-level information (<5 Hz in our materials), and phoneme-level information (12-40 Hz). Functional connectivity analyses showed network differences between groups in both hemispheres, with dyslexic children showing significantly reduced global network efficiency. Global network efficiency correlated with dyslexic children's oral language development and with control children's reading development. These developmental data suggest that dyslexia is characterized by atypical neural sampling of auditory information at slower rates. They also throw new light on the nature of the gamma band temporal sampling differences reported in MEG dyslexia studies with adults.

Neural detection of changes in amplitude rise time in infancy.

Amplitude rise times play a crucial role in the perception of rhythm in speech, and reduced perceptual sensitivity to differences in rise time is related to developmental language difficulties. Amplitude rise times also play a mechanistic role in neural entrainment to the speech amplitude envelope. Using an ERP paradigm, here we examined for the first time whether infants at the ages of seven and eleven months exhibit an auditory mismatch response to changes in the rise times of simple repeating auditory stimuli. We found that infants exhibited a mismatch response (MMR) to all of the oddball rise times used for the study. The MMR was more positive at seven than eleven months of age. At eleven months, there was a shift to a mismatch negativity (MMN) that was more pronounced over left fronto-central electrodes. The MMR over right fronto-central electrodes was sensitive to the size of the difference in rise time. The results indicate that neural processing of changes in rise time is present at seven months, supporting the possibility that early speech processing is facilitated by neural sensitivity to these important acoustic cues.

Delta- and theta-band cortical tracking and phase-amplitude coupling to sung speech by infants.

The amplitude envelope of speech carries crucial low-frequency acoustic information that assists linguistic decoding at multiple time scales. Neurophysiological signals are known to track the amplitude envelope of adult-directed speech (ADS), particularly in the theta-band. Acoustic analysis of infant-directed speech (IDS) has revealed significantly greater modulation energy than ADS in an amplitude-modulation (AM) band centred on ∼2 Hz. Accordingly, cortical tracking of IDS by delta-band neural signals may be key to language acquisition. Speech also contains acoustic information within its higher-frequency bands (beta, gamma). Adult EEG and MEG studies reveal an oscillatory hierarchy, whereby low-frequency (delta, theta) neural phase dynamics temporally organize the amplitude of high-frequency signals (phase amplitude coupling, PAC). Whilst consensus is growing around the role of PAC in the matured adult brain, its role in the development of speech processing is unexplored. Here, we examined the presence and maturation of low-frequency (<12 Hz) cortical speech tracking in infants by recording EEG longitudinally from 60 participants when aged 4-, 7- and 11- months as they listened to nursery rhymes. After establishing stimulus-related neural signals in delta and theta, cortical tracking at each age was assessed in the delta, theta and alpha [control] bands using a multivariate temporal response function (mTRF) method. Delta-beta, delta-gamma, theta-beta and theta-gamma phase-amplitude coupling (PAC) was also assessed. Significant delta and theta but not alpha tracking was found. Significant PAC was present at all ages, with both delta and theta -driven coupling observed.

Development of binaural temporal fine structure sensitivity in children.

The highest frequency for which the temporal fine structure (TFS) of a sinewave can be compared across ears varies between listeners with an upper limit of about 1400 Hz for young normal-hearing adults (YNHA). In this study, binaural TFS sensitivity was investigated for 63 typically developing children, aged 5 years, 6 months to 9 years, 4 months using the temporal fine structure-adaptive frequency (TFS-AF) test of Füllgrabe, Harland, Sek, and Moore [Int. J. Audiol. 56, 926-935 (2017)]. The test assesses the highest frequency at which an interaural phase difference (IPD) of ϕ° can be distinguished from an IPD of 0°. The values of ϕ were 30° and 180°. The starting frequency was 200 Hz. The thresholds for the children were significantly lower (worse) than the thresholds reported by Füllgrabe, Harland, Sek, and Moore [Int. J. Audiol. 56, 926-935 (2017)] for YNHA. For both values of ϕ, the median age at which children performed above chance level was significantly higher (p < 0.001) than for those who performed at chance. For the subgroup of 40 children who performed above chance for ϕ = 180°, the linear regression analyses showed that the thresholds for ϕ = 180° increased (improved) significantly with increasing age (p < 0.001) with adult-like thresholds predicted to be reached at 10 years, 2 months of age. The implications for spatial release from masking are discussed.

Machine learning accurately classifies neural responses to rhythmic speech vs. non-speech from 8-week-old infant EEG.

Currently there are no reliable means of identifying infants at-risk for later language disorders. Infant neural responses to rhythmic stimuli may offer a solution, as neural tracking of rhythm is atypical in children with developmental language disorders. However, infant brain recordings are noisy. As a first step to developing accurate neural biomarkers, we investigate whether infant brain responses to rhythmic stimuli can be classified reliably using EEG from 95 eight-week-old infants listening to natural stimuli (repeated syllables or drumbeats). Both Convolutional Neural Network (CNN) and Support Vector Machine (SVM) approaches were employed. Applied to one infant at a time, the CNN discriminated syllables from drumbeats with a mean AUC of 0.87, against two levels of noise. The SVM classified with AUC 0.95 and 0.86 respectively, showing reduced performance as noise increased. Our proof-of-concept modelling opens the way to the development of clinical biomarkers for language disorders related to rhythmic entrainment.

The temporal modulation structure of illiterate versus literate adult speech.

The temporal modulation structure of speech plays a key role in neural encoding of the speech signal. Amplitude modulations (AMs, quasi-rhythmic changes in signal energy or intensity) in speech are encoded by neuronal oscillations (rhythmic variations in neural excitability in large cell networks) that oscillate at matching temporal rates. To date, however, all neural studies have investigated adult-directed speech (ADS) as produced and perceived by highly literate adults. Whether temporal features of ADS vary with the skills of the speaker, for example literacy skills, is currently unknown. Here we analyse the temporal structure of ADS spoken by illiterate, low literate (≤ 4 years of literacy) and highly literate (≥ 12 years of literacy) adults. We find that illiterates produce speech differently. Spontaneous conversational speech produced by illiterate adults showed significantly less synchronised coupling between AM bands (less phase synchronisation) than conversational speech produced by low literate and highly literate adults, and contained significantly fewer syllables per second. There was also a significant relationship between years of literacy and the amount of theta-band energy in conversational speech. When asked to produce rhythmic proverbs learned in childhood, all groups could produce speech with similar AM phase synchronisation, suggesting that the differences in spontaneous conversational speech were not caused by physiological constraints. The data suggest that the temporal modulation structure of spoken language changes with the acquisition of cultural skills like literacy that are usually a product of schooling. There is a cultural effect on the temporal modulation structure of spoken language.

The role of phase synchronisation between low frequency amplitude modulations in child phonology and morphology speech tasks.

Recent models of the neural encoding of speech suggest a core role for amplitude modulation (AM) structure, particularly regarding AM phase alignment. Accordingly, speech tasks that measure linguistic development in children may exhibit systematic properties regarding AM structure. Here, the acoustic structure of spoken items in child phonological and morphological tasks, phoneme deletion and plural elicitation, was investigated. The phase synchronisation index (PSI), reflecting the degree of phase alignment between pairs of AMs, was computed for 3 AM bands (delta, theta, beta/low gamma; 0.9-2.5 Hz, 2.5-12 Hz, 12-40 Hz, respectively), for five spectral bands covering 100-7250 Hz. For phoneme deletion, data from 94 child participants with and without dyslexia was used to relate AM structure to behavioural performance. Results revealed that a significant change in magnitude of the phase synchronisation index (ΔPSI) of slower AMs (delta-theta) systematically accompanied both phoneme deletion and plural elicitation. Further, children with dyslexia made more linguistic errors as the delta-theta ΔPSI increased. Accordingly, ΔPSI between slower temporal modulations in the speech signal systematically distinguished test items from accurate responses and predicted task performance. This may suggest that sensitivity to slower AM information in speech is a core aspect of phonological and morphological development.

Speech Processing to Improve the Perception of Speech in Background Noise for Children With Auditory Processing Disorder and Typically Developing Peers.

Auditory processing disorder (APD) may be diagnosed when a child has listening difficulties but has normal audiometric thresholds. For adults with normal hearing and with mild-to-moderate hearing impairment, an algorithm called spectral shaping with dynamic range compression (SSDRC) has been shown to increase the intelligibility of speech when background noise is added after the processing. Here, we assessed the effect of such processing using 8 children with APD and 10 age-matched control children. The loudness of the processed and unprocessed sentences was matched using a loudness model. The task was to repeat back sentences produced by a female speaker when presented with either speech-shaped noise (SSN) or a male competing speaker (CS) at two signal-to-background ratios (SBRs). Speech identification was significantly better with SSDRC processing than without, for both groups. The benefit of SSDRC processing was greater for the SSN than for the CS background. For the SSN, scores were similar for the two groups at both SBRs. For the CS, the APD group performed significantly more poorly than the control group. The overall improvement produced by SSDRC processing could be useful for enhancing communication in a classroom where the teacher's voice is broadcast using a wireless system.

A Re-examination of the Effect of Masker Phase Curvature on Non-simultaneous Masking.

Forward masking of a sinusoidal signal is determined not only by the masker's power spectrum but also by its phase spectrum. Specifically, when the phase spectrum is such that the output of an auditory filter centred on the signal has a highly modulated ("peaked") envelope, there is less masking than when that envelope is flat. This finding has been attributed to non-linearities, such as compression, reducing the average neural response to maskers that produce more peaked auditory filter outputs (Carlyon and Datta, J Acoust Soc Am 101:3636-3647, 1997). Here we evaluate an alternative explanation proposed by Wotcjzak and Oxenham (Wojtczak and Oxenham, J Assoc Res Otolaryngol 10:595-607, 2009). They reported a masker phase effect for 6-kHz signals when the masker components were at least an octave below the signal frequency. Wotcjzak and Oxenham argued that this effect was inconsistent with cochlear compression, and, because it did not occur at lower signal frequencies, was also inconsistent with more central compression. It was instead attributed to activation of the efferent system reducing the response to the subsequent probe. Here, experiment 1 replicated their main findings. Experiment 2 showed that the phase effect on off-frequency forward masking is similar at signal frequencies of 2 and 6 kHz, provided that one equates the number of components likely to interact within an auditory filter centred on the signal, thereby roughly equating the effect of masker phase on the peakiness of that filter output. Experiment 3 showed that for some subjects, masker phase also had a strong influence on off-frequency backward masking of the signal, and that the size of this effect correlated across subjects with that observed in forward masking. We conclude that the masker phase effect is mediated mainly by cochlear non-linearities, with a possible additional effect of more central compression. The data are not consistent with a role for the efferent system.

Evaluation of near-end speech enhancement under equal-loudness constraint for listeners with normal-hearing and mild-to-moderate hearing loss.

Four algorithms designed to enhance the intelligibility of speech when noise is added after processing were evaluated under the constraint that the speech should have the same loudness before and after processing, as determined using a loudness model. The algorithms applied spectral modifications and two of them included dynamic-range compression. On average, the methods with dynamic-range compression required the least level adjustment to equate loudness for the unprocessed and processed speech. Subjects with normal-hearing (experiment 1) and mild-to-moderate hearing loss (experiment 2) were tested using unmodified and enhanced speech presented in speech-shaped noise (SSN) and a competing speaker (CS). The results showed (a) the algorithms with dynamic-range compression yielded the largest intelligibility gains in both experiments and for both types of background; (b) the algorithms without dynamic-range compression either yielded benefit only with the SSN or yielded no consistent benefit; (c) speech reception thresholds for unprocessed speech were higher for hearing-impaired than for normal-hearing subjects, by about 2 dB for the SSN and 6 dB for the CS. It is concluded that the enhancement methods incorporating dynamic-range compression can improve intelligibility under the equal-loudness constraint for both normal-hearing and hearing-impaired subjects and for both steady and fluctuating backgrounds.

Effectiveness of a loudness model for time-varying sounds in equating the loudness of sentences subjected to different forms of signal processing.

A model for the loudness of time-varying sounds [Glasberg and Moore (2012). J. Audio. Eng. Soc. 50, 331-342] was assessed for its ability to predict the loudness of sentences that were processed to either decrease or increase their dynamic fluctuations. In a paired-comparison task, subjects compared the loudness of unprocessed and processed sentences that had been equalized in (1) root-mean square (RMS) level; (2) the peak long-term loudness predicted by the model; (3) the mean long-term loudness predicted by the model. Method 2 was most effective in equating the loudness of the original and processed sentences.