Transposition Effects in an Aksharic Writing System: The Case of Hindi.

We tested for transposition effects (TEs) in Hindi (a Modern Indo-Aryan language) using unprimed lexical decision. TEs are defined as less accurate and slower responses to transposed-nonwords (e.g., , formed from base-word ) than corresponding replaced-nonwords (e.g., ). In Hindi's orthography, letters map transparently to phonemes (except schwa), but the letters are arranged into "akshars," (n]V>) which encode open syllables. This formal characteristic makes Hindi's orthography typologically "aksharic." We used TEs to determine whether the orthography's typological units, letters and akshars, are also functional units for readers. We conducted three visual word recognition experiments with adult readers whose native language was Hindi. In Experiment 1, we found TEs for consonant () and matra (, a vowel diacritic) letters, using different stimulus sets for each type of transposition. In the next two experiments, we used the same base words to form all of the transposed and replaced items. In Experiment 2, we replicated the findings of Experiment 1 in a different stimulus set; additionally, we found TEs for transpositions between a letter and a akshar. In Experiment 3, we replicated results of the first two experiments by finding TEs for both consonants and matras in another stimulus set; additionally, we found similar TEs for akshars. These results show that and letters are functional units for Hindi readers; the transposition results for akshars are tentative. TEs for letters show that the aksharic grouping of letters does not prevent readers from decoding the constituent letters of akshars. Hindi is read alphabetically.

Phonetic drift in Spanish-English bilinguals: Experiment and a self-organizing model.

Studies of speech accommodation provide evidence for change in use of language structures beyond the critical/sensitive period. For example, Sancier and Fowler (1997) found changes in the voice-onset-times (VOTs) of both languages of a Portuguese-English bilingual as a function of her language context. Though accommodation has been studied widely within a monolingual context, it has received less attention in and between the languages of bilinguals. We tested whether these findings of phonetic accommodation, speech accommodation at the phonetic level, would generalize to a sample of Spanish-English bilinguals. We recorded participants reading Spanish and English sentences after 3-4 months in the US and after 2-4 weeks in a Spanish speaking country and measured the VOTs of their voiceless plosives. Our statistical analyses show that participants' English VOTs drifted towards those of the ambient language, but their Spanish VOTs did not. We found considerable variation in the extent of individual participants' drift in English. Further analysis of our results suggested that native-likeness of L2 VOTs and extent of active language use predict the extent of drift. We provide a model based on principles of self-organizing dynamical systems to account for our Spanish-English phonetic drift findings and the Portuguese-English findings.

"Perception of the speech code" revisited: Speech is alphabetic after all.

We revisit an article, "Perception of the Speech Code" (PSC), published in this journal 50 years ago (Liberman, Cooper, Shankweiler, & Studdert-Kennedy, 1967) and address one of its legacies concerning the status of phonetic segments, which persists in theories of speech today. In the perspective of PSC, segments both exist (in language as known) and do not exist (in articulation or the acoustic speech signal). Findings interpreted as showing that speech is not a sound alphabet, but, rather, phonemes are encoded in the signal, coupled with findings that listeners perceive articulation, led to the motor theory of speech perception, a highly controversial legacy of PSC. However, a second legacy, the paradoxical perspective on segments has been mostly unquestioned. We remove the paradox by offering an alternative supported by converging evidence that segments exist in language both as known and as used. We support the existence of segments in both language knowledge and in production by showing that phonetic segments are articulatory and dynamic and that coarticulation does not eliminate them. We show that segments leave an acoustic signature that listeners can track. This suggests that speech is well-adapted to public communication in facilitating, not creating a barrier to, exchange of language forms.

Seeking a reading machine for the blind and discovering the speech code.

A machine that can read printed material to the blind became a priority at the end of World War II with the appointment of a U.S. Government committee to instigate research on sensory aids to improve the lot of blinded veterans. The committee chose Haskins Laboratories to lead a multisite research program. Initially, Haskins researchers overestimated the capacities of users to learn an acoustic code based on the letters of a text, resulting in unsuitable designs. Progress was slow because the researchers clung to a mistaken view that speech is a sound alphabet and because of persisting gaps in man-machine technology. The tortuous route to a practical reading machine transformed the scientific understanding of speech perception and reading at Haskins Labs and elsewhere, leading to novel lines of basic research and new technologies. Research at Haskins Laboratories made valuable contributions in clarifying the physical basis of speech. Researchers recognized that coarticulatory overlap eliminated the possibility of alphabet-like discrete acoustic segments in speech. This work advanced the study of speech perception and contributed to our understanding of the relation of speech perception to production. Basic findings on speech enabled the development of speech synthesis, part science and part technology, essential for development of a reading machine, which has found many applications. Findings on the nature of speech further stimulated a new understanding of word recognition in reading across languages and scripts and contributed to our understanding of reading development and reading disabilities.

Information for coarticulation: Static signal properties or formant dynamics?

Perception of a speech segment changes depending on properties of surrounding segments in a phenomenon called compensation for coarticulation (Mann, 1980). The nature of information that drives these perceptual changes is a matter of debate. One account attributes perceptual shifts to low-level auditory system contrast effects based on static portions of the signal (e.g., third formant [F3] center or average frequency; Lotto & Kluender, 1998). An alternative account is that listeners' perceptual shifts result from listeners attuning to the acoustic effects of gestural overlap and that this information for coarticulation is necessarily dynamic (Fowler, 2006). In a pair of experiments, we used sinewave speech precursors to investigate the nature of information for compensation for coarticulation. In Experiment 1, as expected by both accounts, we found that sinewave speech precursors produce shifts in following segments. In Experiment 2, we investigated whether effects in Experiment 1 were driven by static F3 offsets of sinewave speech precursors, or by dynamic relationships among their formants. We temporally reversed F1 and F2 in sinewave precursors, preserving static F3 offset and average F1, F2 and F3 frequencies, but disrupting dynamic formant relationships. Despite having identical F3s, selectively reversed precursors produced effects that were significantly smaller and restricted to only a small portion of the continuum. We conclude that dynamic formant relations rather than static properties of the precursor provide information for compensation for coarticulation.

Talking as doing: Language forms and public language.

I discuss language forms as the primary means that language communities provide to enable public language use. As such, they are adapted to public use most notably in being linguistically significant vocal tract actions, not the categories in the mind as proposed in phonological theories. Their primary function is to serve as vehicles for production of syntactically structured sequences of words. However, more than that, phonological actions themselves do work in public language use. In particular, they foster interpersonal coordination in social activities. An intriguing property of language forms that likely reflects their emergence in social communicative activities is that phonological forms that should be meaningless (in order to serve their role in the openness of language at the level of the lexicon) are not wholly meaningless. In fact, the form-meaning "rift" is bridged bidirectionally: The smallest language forms are meaningful, and the meanings of lexical language forms generally inhere, in part, in their embodiment by understanders.

Listening with a foreign-accent: The interlanguage speech intelligibility benefit in Mandarin speakers of English.

This study examined the intelligibility of native and Mandarin-accented English speech for native English and native Mandarin listeners. In the latter group, it also examined the role of the language environment and English proficiency. Three groups of listeners were tested: native English listeners (NE), Mandarin-speaking Chinese listeners in the US (M-US) and Mandarin listeners in Beijing, China (M-BJ). As a group, M-US and M-BJ listeners were matched on English proficiency and age of acquisition. A nonword transcription task was used. Identification accuracy for word-final stops in the nonwords established two independent interlanguage intelligibility effects. An interlanguage speech intelligibility benefit for listeners (ISIB-L) was manifest by both groups of Mandarin listeners outperforming native English listeners in identification of Mandarin-accented speech. In the benefit for talkers (ISIB-T), only M-BJ listeners were more accurate identifying Mandarin-accented speech than native English speech. Thus, both Mandarin groups demonstrated an ISIB-L while only the M-BJ group overall demonstrated an ISIB-T. The English proficiency of listeners was found to modulate the magnitude of the ISIB-T in both groups. Regression analyses also suggested that the listener groups differ in their use of acoustic information to identify voicing in stop consonants.

An ecological alternative to a "sad response": public language use transcends the boundaries of the skin.

Embedding theories of language production and comprehension in theories of action-perception is realistic and highlights that production and comprehension processes are interleaved. However, layers of internal models that repeatedly predict future linguistic actions and perceptions are implausible. I sketch an ecological alternative whereby perceiver/actors are modeled as dynamical systems coupled to one another and to the environment.

Similar response patterns do not imply identical origins: an energetic masking account of nonspeech effects in compensation for coarticulation.

Nonspeech materials are widely used to identify basic mechanisms underlying speech perception. For instance, they have been used to examine the origin of compensation for coarticulation, the observation that listeners' categorization of phonetic segments depends on neighboring segments (Mann, 1980). Specifically, nonspeech precursors matched to critical formant frequencies of speech precursors have been shown to produce similar categorization shifts as speech contexts. This observation has been interpreted to mean that spectrally contrastive frequency relations between neighboring segments underlie the categorization shifts observed after speech, as well as nonspeech precursors (Lotto & Kluender, 1998). From the gestural perspective, however, categorization shifts in speech contexts occur because of listeners' sensitivity to acoustic information for coarticulatory gestural overlap in production; in nonspeech contexts, this occurs because of energetic masking of acoustic information for gestures. In 2 experiments, we distinguish the energetic masking and spectral contrast accounts. In Experiment 1, we investigated the effects of varying precursor tone frequency on speech categorization. Consistent only with the masking account, tonal effects were greater for frequencies close enough to those in the target syllables for masking to occur. In Experiment 2, we filtered the target stimuli to simulate effects of masking and obtained behavioral outcomes that closely resemble those with nonspeech tones. We conclude that masking provides the more plausible account of nonspeech context effects. More generally, we suggest that similar results from the use of speech and nonspeech materials do not automatically imply identical origins and that the use of nonspeech in speech studies entails careful examination of the nature of information in the nonspeech materials.

Functional activation for imitation of seen and heard speech.

This study examined fMRI activation when perceivers either passively observed or observed and imitated matched or mismatched audiovisual ("McGurk") speech stimuli. Greater activation was observed in the inferior frontal gyrus (IFG) overall for imitation than for perception of audiovisual speech and for imitation of the McGurk-type mismatched stimuli than matched audiovisual stimuli. This unique activation in the IFG during imitation of incongruent audiovisual speech may reflect activation associated with direct matching of incongruent auditory and visual stimuli or conflict between category responses. This study provides novel data about the underlying neurobiology of imitation and integration of AV speech.

Articulatory events are imitated under rapid shadowing.

We tested the hypothesis that rapid shadowers imitate the articulatory gestures that structure acoustic speech signals-not just acoustic patterns in the signals themselves-overcoming highly practiced motor routines and phonological conditioning in the process. In a first experiment, acoustic evidence indicated that participants reproduced allophonic differences between American English /l/ types (light and dark) in the absence of the positional variation cues more typically present with lateral allophony. However, imitative effects were small. In a second experiment, varieties of /l/ with exaggerated light/dark differences were presented by ear. Acoustic measures indicated that all participants reproduced differences between /l/ types; larger average imitative effects obtained. Finally, we examined evidence for imitation in articulation. Participants ranged in behavior from one who did not imitate to another who reproduced distinctions among light laterals, dark laterals and /w/, but displayed a slight but inconsistent tendency toward enhancing imitation of lingual gestures through a slight lip protrusion. Overall, results indicated that most rapid shadowers need not substitute familiar allophones as they imitate reorganized gestural constellations even in the absence of explicit instruction to imitate, but that the extent of the imitation is small. Implications for theories of speech perception are discussed.

Locus equations are an acoustic expression of articulator synergy.

The study investigated the articulatory basis of locus equations, regression lines relating F2 at the start of a Consonant-Vowel (CV) transition to F2 at the middle of the vowel, with C fixed and V varying. Several studies have shown that consonants of different places of articulation have locus equation slopes that descend from labial to velar to alveolar, and intercept magnitudes that increase in the opposite order. Using formulas from the theory of bivariate regression that express regression slopes and intercepts in terms of standard deviations and averages of the variables, it is shown that the slope directly encodes a well-established measure of coarticulation resistance. It is also shown that intercepts are directly related to the degree to which the tongue body assists the formation of the constriction for the consonant. Moreover, it is shown that the linearity of locus equations and the linear relation between locus equation slopes and intercepts originates in linearity in articulation between the horizontal position of the tongue dorsum in the consonant and to that in the vowel. It is concluded that slopes and intercepts of acoustic locus equations are measures of articulator synergy.

Compensation for coarticulation: disentangling auditory and gestural theories of perception of coarticulatory effects in speech.

According to one approach to speech perception, listeners perceive speech by applying general pattern matching mechanisms to the acoustic signal (e.g., Diehl, Lotto, & Holt, 2004). An alternative is that listeners perceive the phonetic gestures that structured the acoustic signal (e.g., Fowler, 1986). The two accounts have offered different explanations for the phenomenon of compensation for coarticulation (CfC). An example of CfC is that if a speaker produces a gesture with a front place of articulation, it may be pulled slightly backwards if it follows a back place of articulation, and listeners' category boundaries shift (compensate) accordingly. The gestural account appeals to direct attunement to coarticulation to explain CfC, whereas the auditory account explains it by spectral contrast. In previous studies, spectral contrast and gestural consequences of coarticulation have been correlated, such that both accounts made identical predictions. We identify a liquid context in Tamil that disentangles contrast and coarticulation, such that the two accounts make different predictions. In a standard CfC task in Experiment 1, gestural coarticulation rather than spectral contrast determined the direction of CfC. Experiments 2, 3, and 4 demonstrated that tone analogues of the speech precursors failed to produce the same effects observed in Experiment 1, suggesting that simple spectral contrast cannot account for the findings of Experiment 1.

Listeners' perception of compensatory shortening.

English exhibits compensatory shortening, whereby a stressed syllable followed by an unstressed syllable is measured to be shorter than the same stressed syllable alone. This anticipatory shortening is much greater than backward shortening, whereby an unstressed syllable is measured to shorten a following stressed syllable. We speculated that measured shortening reflects not true shortening, but coarticulatory hiding. Hence, we asked whether listeners are sensitive to parts of stressed syllables hidden by following or preceding unstressed syllables. In two experiments (Experiments 1A and 1B), we found the point of subjective equality-that is, the durational difference between a stressed syllable in isolation and one followed by an unstressed syllable-at which listeners cannot tell which is longer. In a third experiment (Experiment 2), we found the point of subjective equality for stressed monosyllables and disyllables with a weak-strong stress pattern. In all of the experiments, the points of subjective equality occurred when stressed syllables in disyllables were measured to be shorter than those in monosyllables, as if the listeners heard the coarticulatory onset or the continuation of a stressed syllable within unstressed syllables.

The reality of phonological forms: a reply to Port.

I suggest four grounds on which an argument can be made that phonological language forms are not merely emergent properties of the public language use of members of a language community. They are: 1) the existence of spontaneous errors of speech production in which whole consonants or vowels misorder or are replaced; 2) the necessary existence of language "particles" used by individual language users in order for words to be able to be coined; 3) the remarkable effectiveness of alphabetic writing systems and the tight coupling among skilled readers of orthographic and phonological language forms; 4) the finding that, by late infancy, children have discovered phonological constancies despite phonetic variation.

Embodied, Embedded Language Use.

Language use has a public face that is as important to study as the private faces under intensive psycholinguistic study. In the domain of phonology, public use of speech must meet an interpersonal "parity" constraint if it is to serve to communicate. That is, spoken language forms must reliably be identified by listeners. To that end, language forms are embodied, at the lowest level of description, as phonetic gestures of the vocal tract that lawfully structure informational media such as air and light. Over time, under the parity constraint, sound inventories emerge over communicative exchanges that have the property of sufficient identifiability.Communicative activities involve more than vocal tract actions. Talkers gesture and use facial expressions and eye gaze to communicate. Listeners embody their language understandings, exhibiting dispositions to behave in ways related to language understanding. Moreover, linguistic interchanges are embedded in the larger context of language use. Talkers recruit the environment in their communicative activities, for example, in using deictic points. Moreover, in using language as a "coordination device," interlocutors mutually entrain.

New Affordances for Language: Distributed, Dynamical, and Dialogical Resources.

In introducing the articles of this special issue on language, which grew out of the conference "Grounding Language in Perception and (Inter) Action," we take the opportunity to reflect on fundamental aspects of speaking and listening to others that are often overlooked. The act of conversing is marked by context sensitivity, interdependency, impredicativity, irreversibility, and responsibility, among other things. Language entails real work: it involves real movements in physical, social, and moral orders that are distributed across a wide array of spatial-temporal scales (e.g., evolutionary, historical); yet there is a dimension of play "at work" as well. These workings of language are embedded and embodied in distributed ways that reveal the fundamentally social, public nature of the activity. It is a form of coaction that is dialogical and dynamic in ways that may point to deeper understandings of what it means for perception to be direct and for action to be specific. Language locates us.

Perceptuomotor compatibility effects in speech.

Kerzel and Bekkering (2000) found perceptuomotor compatibility effects between spoken syllables and visible speech gestures and interpreted them as evidence in favor of the distinctive claim of the motor theory of speech perception that the motor system is recruited for perceiving speech. We present three experiments aimed at testing this interpretation. In Experiment 1, we replicated the original findings by Kerzel and Bekkering but with audible syllables. In Experiments 2 and 3, we tested the results of Experiment 1 under more stringent conditions, with different materials and different experimental designs. In all of our experiments, we found the same result: Perceiving syllables affects uttering syllables. The result is consistent both with the results of a number of other behavioral and neural studies related to speech and with more general findings of perceptuomotor interactions. Taken together, these studies provide evidence in support of the motor theory claim that the motor system is recruited for perceiving speech.

Sentence comprehension affects the dynamics of bimanual coordination: implications for embodied cognition.

Recent work in embodied cognition has demonstrated that language comprehension involves the motor system (e.g., Glenberg & Kaschak, 2002). Such findings are often attributed to mechanisms involving simulations of linguistically described events (Barsalou, 1999; Fischer & Zwaan, 2008). We propose that research paradigms in which simulation is the central focus need to be augmented with paradigms that probe the organization of the motor system during language comprehension. The use of well-studied motor tasks may be appropriate to this endeavour. To this end, we present a study in which participants perform a bimanual rhythmic task (Kugler & Turvey, 1987) while judging the plausibility of sentences. We show that the dynamics of the bimanual task differ when participants judge sentences describing performable actions as opposed to sentences describing events that are not performable. We discuss the general implications of our results for accounts of embodied cognition.

A critical examination of the spectral contrast account of compensation for coarticulation.

Vocal tract gestures for adjacent phones overlap temporally, rendering the acoustic speech signal highly context dependent. For example, following a segment with an anterior place of articulation, a posterior segment's place of articulation is pulled frontward, and listeners' category boundaries shift appropriately. Some theories assume that listeners perceptually attune or compensate for coarticulatory context. An alternative is that shifts result from spectral contrast. Indeed, shifts occur when speech precursors are replaced by pure tones, frequency matched to the formant offset at the assumed locus of contrast (Lotto & Kluender, 1998). However, tone analogues differ from natural formants in several ways, raising the possibility that conditions for contrast may not exist in natural speech. When we matched tones to natural formant intensities and trajectories, boundary shifts diminished. When we presented only the critical spectral region of natural speech tokens, no compensation was observed. These results suggest that conditions for spectral contrast do not exist in typical speech.