Internal speech is faster than external speech: Evidence for feedback-based temporal control.

A recent model of temporal control in speech holds that speakers use sensory feedback to control speech rate and articulatory timing. An experiment was conducted to assess whether there is evidence in support of this hypothesis by comparing durations of phrases in external speech (with sensory feedback) and internal speech (without sensory feedback). Phrase lengths were varied by including one to three disyllabic nouns in a target phrase that was always surrounded by overt speech. An inferred duration method was used to estimate the durations of target phrases produced internally. The results showed that internal speech is faster than external speech, supporting the hypothesis. In addition, the results indicate that there is a slow-down associated with transitioning between internal and external modes of production.

A Preliminary Study of Speech Rhythm Differences as Markers of Stuttering Persistence in Preschool-Age Children.

PURPOSE: The purpose of this study was twofold: (a) to determine whether there are speech rhythm differences between preschool-age children who stutter that were eventually diagnosed as persisting (CWS-Per) or recovered (CWS-Rec) and children who do not stutter (CWNS), using empirical spectral analysis and empirical mode decomposition of the speech amplitude envelope, and (b) to determine whether speech rhythm characteristics close to onset are predictive of later persistence. METHOD: Fifty children (3-4 years of age) participated in the study. Approximately 2-2.5 years after the experimental testing took place, children were assigned to the following groups: CWS-Per (nine boys, one girl), CWS-Rec (18 boys, two girls), and CWNS (18 boys, two girls). All children produced a narrative based on a text-free storybook. From the audio recordings of these narratives, fluent utterances were selected for each child from which seven envelope-based measures were extracted. Group-based differences on each measure as well as predictive analyses were conducted to identify measures that discriminate CWS-Per versus CWS-Rec. RESULTS: CWS-Per were found to have a relatively higher degree of power in suprasyllabic oscillations and greater variability in the timing of syllabic rhythms especially for longer utterances. A logistic regression model using two speech rhythm measures was able to discriminate the eventual outcome of recovery versus persistence, with 80% sensitivity and 75% specificity. CONCLUSION: Findings suggest that envelope-based speech rhythm measures are a promising approach to assess speech rhythm differences in developmental stuttering, and its potential for identification of children at risk of developing persistent stuttering should be investigated further.

An informal logic of feedback-based temporal control.

A conceptual framework and mathematical model of the control of articulatory timing are presented, in which feedback systems play a fundamental role. The model applies both to relatively small timescales, such as within syllables, and to relatively large timescales, such as multi-phrase utterances. A crucial distinction is drawn between internal/predictive feedback and external/sensory feedback. It is argued that speakers modulate attention to feedback to speed up and slow down speech. A number of theoretical implications of the framework are discussed, including consequences for the understanding of syllable structure and prosodic phrase organization.

Localizing category-related information in speech with multi-scale analyses.

Measurements of the physical outputs of speech-vocal tract geometry and acoustic energy-are high-dimensional, but linguistic theories posit a low-dimensional set of categories such as phonemes and phrase types. How can it be determined when and where in high-dimensional articulatory and acoustic signals there is information related to theoretical categories? For a variety of reasons, it is problematic to directly quantify mutual information between hypothesized categories and signals. To address this issue, a multi-scale analysis method is proposed for localizing category-related information in an ensemble of speech signals using machine learning algorithms. By analyzing how classification accuracy on unseen data varies as the temporal extent of training input is systematically restricted, inferences can be drawn regarding the temporal distribution of category-related information. The method can also be used to investigate redundancy between subsets of signal dimensions. Two types of theoretical categories are examined in this paper: phonemic/gestural categories and syntactic relative clause categories. Moreover, two different machine learning algorithms were examined: linear discriminant analysis and neural networks with long short-term memory units. Both algorithms detected category-related information earlier and later in signals than would be expected given standard theoretical assumptions about when linguistic categories should influence speech. The neural network algorithm was able to identify category-related information to a greater extent than the discriminant analyses.

Motoric Mechanisms for the Emergence of Non-local Phonological Patterns.

Non-local phonological patterns can be difficult to analyze in the context of speech production models. Some patterns - e.g., vowel harmonies, nasal harmonies - can be readily analyzed to arise from temporal extension of articulatory gestures (i.e., spreading); such patterns can be viewed as articulatorily local. However, there are other patterns - e.g., nasal consonant harmony, laryngeal feature harmony - which cannot be analyzed as spreading; instead these patterns appear to enforce agreement between features of similar segments without affecting intervening segments. Indeed, there are numerous typological differences between spreading harmonies and agreement harmonies, and this suggests that there is a mechanistic difference in the ways that spreading and agreement harmonies arise. This paper argues that in order to properly understand spreading and agreement patterns, the gestural framework of Articulatory Phonology must be enriched with respect to how targets of the vocal tract are controlled in planning and production. Specifically, it is proposed that production models should distinguish between excitatory and inhibitory articulatory gestures, and that gestures which are below a selection threshold can influence the state of the vocal tract, despite not being active. These ideas are motivated by several empirical phenomena, which include anticipatory posturing before production of a word form, and dissimilatory interactions in distractor-target response paradigms. Based on these ideas, a model is developed which provides two distinct mechanisms for the emergence of non-local phonological patterns.

Space and time in models of speech rhythm.

How do rhythmic patterns in speech arise? There are many representational frameworks for describing rhythmic patterns, but none of these directly connect representations to articulatory processes, which have physical manifestations in the acoustic signal. Here, a new model of speech rhythm is presented, one in which rhythmic patterns arise from spatial mechanisms that govern the organization of articulatory gestures. The roles of time and space in symbolic representations of the metrical structure are analyzed, and conventional understandings of stress and accent are called into question. One aspect of rhythmic patterns, in particular-the directionality of accentual patterns-is examined closely. A novel dynamical model is developed, which proposes a reinterpretation of directionality and other temporal phenomena in speech.

Anticipatory Posturing of the Vocal Tract Reveals Dissociation of Speech Movement Plans from Linguistic Units.

Models of speech production typically assume that control over the timing of speech movements is governed by the selection of higher-level linguistic units, such as segments or syllables. This study used real-time magnetic resonance imaging of the vocal tract to investigate the anticipatory movements speakers make prior to producing a vocal response. Two factors were varied: preparation (whether or not speakers had foreknowledge of the target response) and pre-response constraint (whether or not speakers were required to maintain a specific vocal tract posture prior to the response). In prepared responses, many speakers were observed to produce pre-response anticipatory movements with a variety of articulators, showing that that speech movements can be readily dissociated from higher-level linguistic units. Substantial variation was observed across speakers with regard to the articulators used for anticipatory posturing and the contexts in which anticipatory movements occurred. The findings of this study have important consequences for models of speech production and for our understanding of the normal range of variation in anticipatory speech behaviors.

Speech rhythm analysis with decomposition of the amplitude envelope: characterizing rhythmic patterns within and across languages.

This study presents a method for analyzing speech rhythm using empirical mode decomposition of the speech amplitude envelope, which allows for extraction and quantification of syllabic- and supra-syllabic time-scale components of the envelope. The method of empirical mode decomposition of a vocalic energy amplitude envelope is illustrated in detail, and several types of rhythm metrics derived from this method are presented. Spontaneous speech extracted from the Buckeye Corpus is used to assess the effect of utterance length on metrics, and it is shown how metrics representing variability in the supra-syllabic time-scale components of the envelope can be used to identify stretches of speech with targeted rhythmic characteristics. Furthermore, the envelope-based metrics are used to characterize cross-linguistic differences in speech rhythm in the UC San Diego Speech Lab corpus of English, German, Greek, Italian, Korean, and Spanish speech elicited in read sentences, read passages, and spontaneous speech. The envelope-based metrics exhibit significant effects of language and elicitation method that argue for a nuanced view of cross-linguistic rhythm patterns.

A dynamical model of hierarchical selection and coordination in speech planning.

studies of the control of complex sequential movements have dissociated two aspects of movement planning: control over the sequential selection of movement plans, and control over the precise timing of movement execution. This distinction is particularly relevant in the production of speech: utterances contain sequentially ordered words and syllables, but articulatory movements are often executed in a non-sequential, overlapping manner with precisely coordinated relative timing. This study presents a hybrid dynamical model in which competitive activation controls selection of movement plans and coupled oscillatory systems govern coordination. The model departs from previous approaches by ascribing an important role to competitive selection of articulatory plans within a syllable. Numerical simulations show that the model reproduces a variety of speech production phenomena, such as effects of preparation and utterance composition on reaction time, and asymmetries in patterns of articulatory timing associated with onsets and codas. The model furthermore provides a unified understanding of a diverse group of phonetic and phonological phenomena which have not previously been related.

Articulatory gestures are individually selected in production.

Most models of speech planning and production incorporate a selection mechanism, whereby units are activated in parallel and chosen for execution sequentially. The lowest level units which can be selected are assumed to be segments, i.e. consonants and vowels. The features or articulatory gestures affiliated with segments are presumed to be automatically selected as a consequence of segmental selection. An alternative possibility is that articulatory gestures themselves are subject to a selection process; this predicts that there can be circumstances in which gestures affiliated with the same segment fail to co-occur. We conducted a stop-signal task in which subjects produced /pa/- or /ka/-initial monosyllables and disyllables in response to a go-signal; on 50% of trials subjects halted production as quickly as possible when given a stop-signal within ±300 ms of the go-signal. Articulatory kinematics were recorded using a speech magnetometer. We found that vowel-affiliated gestures of glottal adduction, tongue body lowering, and bilabial opening did not necessarily co-occur in the context of halting speech. This finding indicates that gestures are selected individually, rather than as an automatic consequence of segmental selection.

Multitimescale dynamical interactions between speech rhythm and gesture.

Temporal patterns in human movement, and in speech in particular, occur on multiple timescales. Regularities in such patterns have been observed between speech gestures, which are relatively quick movements of articulators (e.g., tongue fronting and lip protrusion), and also between rhythmic units (e.g., syllables and metrical feet), which occur more slowly. Previous work has shown that patterns in both domains can be usefully modeled with oscillatory dynamical systems. To investigate how rhythmic and gestural domains interact, an experiment was conducted in which speakers performed a phrase repetition task, and gestural kinematics were recorded using electromagnetic articulometry. Variance in relative timing of gestural movements was correlated with variance in rhythmic timing, indicating that gestural and rhythmic systems interact in the process of planning and producing speech. A model of rhythmic and gestural planning oscillators with multifrequency coupling is presented, which can simulate the observed covariability between rhythmic and gestural timing.

Low-frequency Fourier analysis of speech rhythm.

A method for studying speech rhythm is presented, using Fourier analysis of the amplitude envelope of bandpass-filtered speech. Rather than quantifying rhythm with time-domain measurements of interval durations, a frequency-domain representation is used--the rhythm spectrum. This paper describes the method in detail, and discusses approaches to characterizing rhythm with low-frequency spectral information.