Language/culture/mind/brain. Progress at the margins between disciplines.

At the forefront of research on language are new data demonstrating infants' strategies in the early acquisition of language. The data show that infants perceptually "map" critical aspects of ambient language in the first year of life before they can speak. Statistical and abstract properties of speech are picked up through exposure to ambient language. Moreover, linguistic experience alters infants' perception of speech, warping perception in a way that enhances native-language speech processing. Infants' strategies are unexpected and unpredicted by historical views. At the same time, research in three additional disciplines is contributing to our understanding of language and its acquisition by children. Cultural anthropologists are demonstrating the universality of adult speech behavior when addressing infants and children across cultures, and this is creating a new view of the role adult speakers play in bringing about language in the child. Neuroscientists, using the techniques of modern brain imaging, are revealing the temporal and structural aspects of language processing by the brain and suggesting new views of the critical period for language. Computer scientists, modeling the computational aspects of childrens' language acquisition, are meeting success using biologically inspired neural networks. Although a consilient view cannot yet be offered, the cross-disciplinary interaction now seen among scientists pursuing one of humans' greatest achievements, language, is quite promising.

A new view of language acquisition.

At the forefront of debates on language are new data demonstrating infants' early acquisition of information about their native language. The data show that infants perceptually "map" critical aspects of ambient language in the first year of life before they can speak. Statistical properties of speech are picked up through exposure to ambient language. Moreover, linguistic experience alters infants' perception of speech, warping perception in the service of language. Infants' strategies are unexpected and unpredicted by historical views. A new theoretical position has emerged, and six postulates of this position are described.

Perceptual magnet and phoneme boundary effects in speech perception: do they arise from a common mechanism?

The question of whether sensitivity peaks at vowel boundaries (i.e., phoneme boundary effects) and sensitivity minima near excellent category exemplars (i.e., perceptual magnet effects) stem from the same stage of perceptual processing was examined in two experiments. In Experiment 1, participants gave phoneme identification and goodness ratings for 13 synthesized English /i/ and /e/ vowels. In Experiment 2, participants discriminated pairs of these vowels. Either the listeners discriminated the entire range of stimuli within each block of trials, or the range within each block was restricted to a single stimulus pair. In addition, listeners discriminated either one-step or two-step intervals along the stimulus series. The results demonstrated that sensitivity peaks at vowel boundaries were more influenced by stimulus range than were perceptual magnet effects; peaks in sensitivity near the /i/-/e/ boundary were reduced with restricted stimulus ranges and one-step intervals, but minima in discrimination near the best exemplars of /i/ were present in all conditions.

Birdsong and human speech: common themes and mechanisms.

Human speech and birdsong have numerous parallels. Both humans and songbirds learn their complex vocalizations early in life, exhibiting a strong dependence on hearing the adults they will imitate, as well as themselves as they practice, and a waning of this dependence as they mature. Innate predispositions for perceiving and learning the correct sounds exist in both groups, although more evidence of innate descriptions of species-specific signals exists in songbirds, where numerous species of vocal learners have been compared. Humans also share with songbirds an early phase of learning that is primarily perceptual, which then serves to guide later vocal production. Both humans and songbirds have evolved a complex hierarchy of specialized forebrain areas in which motor and auditory centers interact closely, and which control the lower vocal motor areas also found in nonlearners. In both these vocal learners, however, how auditory feedback of self is processed in these brain areas is surprisingly unclear. Finally, humans and songbirds have similar critical periods for vocal learning, with a much greater ability to learn early in life. In both groups, the capacity for late vocal learning may be decreased by the act of learning itself, as well as by biological factors such as the hormones of puberty. Although some features of birdsong and speech are clearly not analogous, such as the capacity of language for meaning, abstraction, and flexible associations, there are striking similarities in how sensory experience is internalized and used to shape vocal outputs, and how learning is enhanced during a critical period of development. Similar neural mechanisms may therefore be involved.

Cross-language analysis of phonetic units in language addressed to infants.

In the early months of life, infants acquire information about the phonetic properties of their native language simply by listening to adults speak. The acoustic properties of phonetic units in language input to young infants in the United States, Russia, and Sweden were examined. In all three countries, mothers addressing their infants produced acoustically more extreme vowels than they did when addressing adults, resulting in a "stretching" of vowel space. The findings show that language input to infants provides exceptionally well-specified information about the linguistic units that form the building blocks for words.

The encoding of rate and talker information during phonetic perception.

The acoustic structure of the speech signal is extremely variable due to a variety of contextual factors, including talker characteristics and speaking rate. To account for the listener's ability to adjust to this variability, speech researchers have posited the existence of talker and rate normalization processes. The current study examined how the perceptual system encoded information about talker and speaking rate during phonetic perception. Experiments 1-3 examined this question, using a speeded classification paradigm developed by Garner (1974). The results of these experiments indicated that decisions about phonemic identity were affected by both talker and rate information: irrelevant variation in either dimension interfered with phonemic classification. While rate classification was also affected by phoneme variation, talker classification was not. Experiment 4 examined the impact of talker and rate variation on the voicing boundary under different blocking conditions. The results indicated that talker characteristics influenced the voicing boundary when talker variation occurred within a block of trials only under certain conditions. Rate variation, however, influenced the voicing boundary regardless of whether or not there was rate variation within a block of trials. The findings from these experiments indicate that phoneme and the rate information are encoded in an integral manner during speech perception, while talker characteristics are encoded separately.

Infant vocalizations in response to speech: vocal imitation and developmental change.

Infants' development of speech begins with a language-universal pattern of production that eventually becomes language specific. One mechanism contributing to this change is vocal imitation. The present study was undertaken to examine developmental change in infants' vocalizations in response to adults' vowels at 12, 16, and 20 weeks of age and test for vocal imitation. Two methodological aspects of the experiment are noteworthy: (a) three different vowel stimuli (/a/, /i/, and /u/) were videotaped and presented to infants by machine so that the adult model could not artifactually influence infant utterances, and (b) infants' vocalizations were analyzed both physically, using computerized spectrographic techniques, and perceptually by trained phoneticians who transcribed the utterances. The spectrographic analyses revealed a developmental change in the production of vowels. Infants' vowel categories become more separated in vowel space from 12 to 20 weeks of age. Moreover, vocal imitation was documented, infants listening to a particular vowel produced vocalizations resembling that vowel. A hypothesis is advanced extending Kuhl's native language magnet (NLM) model to encompass infants' speech production. It is hypothesized that infants listening to ambient language store perceptually derived representations of the speech sounds they hear which in turn serve as targets for the production of speech utterances. NLM unifies previous findings on the effects of ambient language experience on infants' speech perception and the findings reported here that short-term laboratory experience with speech is sufficient to influence infants' speech production.

Influences of phonetic identification and category goodness on American listeners' perception of /r/ and /l/.

Recent experiments have demonstrated that category goodness influences the perception of vowels [Iverson and Kuhl, J. Acoust. Soc. Am. 97, 553-562 (1995)]; listeners show a perceptual magnet effect characterized by shrunken perceptual distances near excellent exemplars of vowel categories and stretched distances near poor exemplars. The present study extends this investigation by examining the relative influence of phonetic identification and category goodness on the perception of American English /r/ and /l/. Eighteen /ra/ and /la/ tokens were synthesized by varying F2 and F3 frequencies. Adult listeners identified and rated the goodness of individual stimuli, and rated the similarity of stimulus pairs. Multidimensional scaling analyses revealed that the perceptual space was shrunk near the best exemplars of each category and stretched near the category boundary. In addition, individual differences in /r/ identification corresponded to the degree of shrinking near the best exemplars of the /r/ category. The results demonstrate that category goodness and phonetic identification both contribute to the perception of /r/ and /l/.

Mapping the perceptual magnet effect for speech using signal detection theory and multidimensional scaling.

Recent experiments have demonstrated that the category goodness of speech sounds strongly influences perception in both adult and infants [Kuhl, Percept. Psychophys. 50, 93-107 (1991); Kuhl et al., Science 255, 606-608 (1992)]. Stimuli judged as exceptionally good instances of phonetic categories (prototypes) make neighboring tokens in the vowel space seem more similar, exhibiting a perceptual magnet effect. Three experiments further examined the perceptual magnet effect in adults. Experiment 1 collected goodness and identification judgments for 13 variants of the vowel /i/. Experiment 2 used signal detection theory to assess the discrimination of these tokens using a bias-free measure (d'). Experiment 3 employed multidimensional scaling (MDS) to geometrically model the distortion of the perceptual space due to the magnet effect. The results demonstrated a strong relationship between category goodness and discrimination. Vowel tokens receiving high goodness ratings in experiment 1 were more difficult to discriminate in experiment 2 and were more tightly clustered in the MDS solutions of experiment 3. These findings support the existence of a perceptual magnet effect, and may help explain some aspects of first language learning in infants and second language learning in adults.

Learning and representation in speech and language.

Infants learn language with remarkable speed. By the end of their second year they speak in sentences with an 'accent' typical of a native speaker. How does an individual acquire a specific language? While acknowledging the biological preparation for language, this review focuses on the effects of early language experience on infants' perceptual and perceptual-motor systems. The data show that by the time infants begin to master the higher levels of language--sound-meaning correspondences, contrastive phonology, and grammatical rules--their perceptual and perceptual-motor systems are already tuned to a specific language. The consequences of this are described in a developmental theory at the phonetic level that holds promise for higher levels of language.

Talker continuity and the use of rate information during phonetic perception.

Research has shown that speaking rate provides an important context for the perception of certain acoustic properties of speech. For example, syllable duration, which varies as a function of speaking rate, has been shown to influence the perception of voice onset time (VOT) for syllable-initial stop consonants. The purpose of the present experiments was to examine the influence of syllable duration when the initial portion of the syllable was produced by one talker and the remainder of the syllable was produced by a different talker. A short-duration and a long-duration /bi/-/pi/ continuum were synthesized with pitch and formant values appropriate to a female talker. When presented to listeners for identification, these stimuli demonstrated the typical effect of syllable duration on the voicing boundary: a shorter VOT boundary for the short stimuli than for the long stimuli. An /i/ vowel, synthesized with pitch and formant values appropriate to a male talker, was added to the end of each of the short tokens, producing a new hybrid continuum. Although the overall syllable duration of the hybrid stimuli equaled the original long stimuli, they produced a VOT boundary similar to that for the short stimuli. In a second experiment, two new /i/ vowels were synthesized. One had a pitch appropriate to a female talker with formant values appropriate to a male talker; the other had a pitch appropriate to a male talker and formants appropriate to a female talker. These vowels were used to create two new hybrid continua. In a third experiment, new hybrid continua were created by using more extreme male formant values. The results of both experiments demonstrated that the hybrid tokens with a change in pitch acted like the short stimuli, whereas the tokens with a change in formants acted like the long stimuli. A fourth experiment demonstrated that listeners could hear a change in talker with both sets of hybrid tokens. These results indicate that continuity of pitch but not formant structure appears to be the critical factor in the calculation of speaking rate within a syllable.

Perceptual strategies in prelingual speech segmentation.

Previous work has suggested that infants may segment continuous speech by a BRACKETING STRATEGY that segregates portions of the speech stream based on prosodic cues to their endpoints. The two present studies were designed to assess whether infants also can deploy a CLUSTERING STRATEGY that exploits asymmetries in transitional probabilities between successive elements, aggregating elements with high transitional probabilities and identifying points of low transitional probabilities as boundaries between units. These studies examined effects of the structure and redundancy of speech context on infants' discrimination of two target syllables using an operant head-turning procedure. After discrimination training on the target syllables in isolation, discrimination maintenance was tested when the target syllables were embedded in one of three contexts. Invariant Order contexts were structured to promote clustering, whereas the Redundant and Variable Order contexts were not. Thirty-six seven-month-olds were tested in Experiment I, in which stimuli were produced with varying intonation contours; 36 eight-month-olds were tested in Experiment 2, in which stimuli were produced with comparable flat pitch contours. In both experiments, performance of the three groups was equivalent in an initial 20-trial test. However, in a second 20-trial test, significant improvements in performance were shown by infants in the Invariant Order condition. No such gains were shown by infants in the other two conditions. These studies suggest that clustering may complement bracketing in infants' discovery of units of language.

Linguistic experience alters phonetic perception in infants by 6 months of age.

Linguistic experience affects phonetic perception. However, the critical period during which experience affects perception and the mechanism responsible for these effects are unknown. This study of 6-month-old infants from two countries, the United States and Sweden, shows that exposure to a specific language in the first half year of life alters infants' phonetic perception.

Integrating speech information across talkers, gender, and sensory modality: female faces and male voices in the McGurk effect.

Studies of the McGurk effect have shown that when discrepant phonetic information is delivered to the auditory and visual modalities, the information is combined into a new percept not originally presented to either modality. In typical experiments, the auditory and visual speech signals are generated by the same talker. The present experiment examined whether a discrepancy in the gender of the talker between the auditory and visual signals would influence the magnitude of the McGurk effect. A male talker's voice was dubbed onto a videotape containing a female talker's face, and vice versa. The gender-incongruent videotapes were compared with gender-congruent videotapes, in which a male talker's voice was dubbed onto a male face and a female talker's voice was dubbed onto a female face. Even though there was a clear incompatibility in talker characteristics between the auditory and visual signals on the incongruent videotapes, the resulting magnitude of the McGurk effect was not significantly different for the incongruent as opposed to the congruent videotapes. The results indicate that the mechanism for integrating speech information from the auditory and the visual modalities is not disrupted by a gender incompatibility even when it is perceptually apparent. The findings are compatible with the theoretical notion that information about voice characteristics of the talker is extracted and used to normalize the speech signal at an early stage of phonetic processing, prior to the integration of the auditory and the visual information.

Cross-modal speech perception in adults and infants using nonspeech auditory stimuli.

Adults and infants were tested for the capacity to detect correspondences between nonspeech sounds and real vowels. The /i/ and /a/ vowels were presented in 3 different ways: auditory speech, silent visual faces articulating the vowels, or mentally imagined vowels. The nonspeech sounds were either pure tones or 3-tone complexes that isolated a single feature of the vowel without allowing the vowel to be identified. Adults perceived an orderly relation between the nonspeech sounds and vowels. They matched high-pitched nonspeech sounds to /i/ vowels and low-pitched nonspeech sounds to /a/ vowels. In contrast, infants could not match nonspeech sounds to the visually presented vowels. Infants' detection of correspondence between auditory and visual speech appears to require the whole speech signal; with development, an isolated feature of the vowel is sufficient for detection of the cross-modal correspondence.

Human adults and human infants show a "perceptual magnet effect" for the prototypes of speech categories, monkeys do not.

Many perceptual categories exhibit internal structure in which category prototypes play an important role. In the four experiments reported here, the internal structure of phonetic categories was explored in studies involving adults, infants, and monkeys. In Experiment 1, adults rated the category goodness of 64 variants of the vowel i parallel on a scale from 1 to 7. The results showed that there was a certain location in vowel space where listeners rated the i parallel vowels as best instances, or prototypes. The perceived goodness of i parallel vowels declined systematically as stimuli were further removed from the prototypic i parallel vowel. Experiment 2 went beyond this initial demonstration and examined the effect of speech prototypes on perception. Either the prototypic or a nonprototypic i parallel vowels was used as the referent stimulus and adults' generalization to other members of the category was examined. Results showed that the typicality of the speech stimulus strongly affected perception. When the prototype of the category served as the referent vowel, there was significantly greater generalization to other i parallel vowels, relative to the situation in which the nonprototype served as the referent. The notion of a perceptual magnet was introduced. The prototype of the category functioned like a perceptual magnet for other category members; it assimilated neighboring stimuli, effectively pulling them toward the prototype. In Experiment 3, the ontogenetic origins of the perceptual magnet effect were explored by testing 6-month-old infants. The results showed that infants' perception of vowels was also strongly affected by speech prototypes. Infants showed significantly greater generalization when the prototype of the vowel category served as the referent; moreover, their responses were highly correlated with those of adults. In Experiment 4, Rhesus monkeys were tested to examine whether or not the prototype's magnet effect was unique to humans. The animals did not provide any evidence of speech prototypes; they did not exhibit the magnet effect. It is suggested that the internal organization of phonetic categories around prototypic members is an ontogenetically early, species-specific, aspect of the speech code.

Integral processing of visual place and auditory voicing information during phonetic perception.

Results of auditory speech experiments show that reaction times (RTs) for place classification in a test condition in which stimuli vary along the dimensions of both place and voicing are longer than RTs in a control condition in which stimuli vary only in place. Similar results are obtained when subjects are asked to classify the stimuli along the voicing dimension. By taking advantage of the "McGurk" effect (McGurk & MacDonald, 1976), the present study investigated whether a similar pattern of interference extends to situations in which variation along the place dimension occurs in the visual modality. The results showed that RTs for classifying phonetic features in the test condition were significantly longer than in the control condition for the place and voicing dimensions. These results indicate a mutual and symmetric interference exists in the classification of the two dimensions, even when the variation along the dimensions occurs in separate modalities.

The role of visual information in the processing of place and manner features in speech perception.

Visual information provided by a talker's mouth movements can influence the perception of certain speech features. Thus, the "McGurk effect" shows that when the syllable (bi) is presented audibly, in synchrony with the syllable (gi), as it is presented visually, a person perceives the talker as saying (di). Moreover, studies have shown that interactions occur between place and voicing features in phonetic perception, when information is presented audibly. In our first experiment, we asked whether feature interactions occur when place information is specificed by a combination of auditory and visual information. Members of an auditory continuum ranging from (ibi) to (ipi) were paired with a video display of a talker saying (igi). The auditory tokens were heard as ranging from (ibi) to (ipi), but the auditory-visual tokens were perceived as ranging from (idi) to (iti). The results demonstrated that the voicing boundary for the auditory-visual tokens was located at a significantly longer VOT value than the voicing boundary for the auditory continuum presented without the visual information. These results demonstrate that place-voice interactions are not limited to situations in which place information is specified audibly.(ABSTRACT TRUNCATED AT 250 WORDS)

The transmission of prosodic information via an electrotactile speechreading aid.

Variations in voice fundamental frequency were extracted from naturally produced speech samples and transmitted to an electrocutaneous display consisting of 10 electrodes arranged in a linear array along the forearm. Changes in fundamental frequency were encoded as changes in stimulus location. Speechreading performance, with and without the electrocutaneous aid, was examined for both sentence and connected discourse materials with one profoundly hearing-impaired listener and one normally hearing listener whose hearing was masked. After 20 hours of speechreading training in connected discourse tracking procedure, both subjects showed higher tracking rates and faster learning rates with the aid than without the aid. Results from closed-set sentence-identification tests showed that patterns of intonation, stress, and phrase structure, which are not easily speechread, are readily available with the aid.