Emotional tones of voice affect the acoustics and perception of Mandarin tones.

Lexical tones and emotions are conveyed by a similar set of acoustic parameters; therefore, listeners of tonal languages face the challenge of processing lexical tones and emotions in the acoustic signal concurrently. This study examined how emotions affect the acoustics and perception of Mandarin tones. In Experiment 1, Mandarin tones were produced by professional actors with angry, fear, happy, sad, and neutral tones of voice. Acoustic analyses on mean F0, F0 range, mean amplitude, and duration were conducted on syllables excised from a carrier phrase. The results showed that emotions affect Mandarin tone acoustics to different degrees depending on specific Mandarin tones and specific emotions. In Experiment 2, selected syllables from Experiment 1 were presented in isolation or in context. Listeners were asked to identify the Mandarin tones and emotions of the syllables. The results showed that emotions affect Mandarin tone identification to a greater extent than Mandarin tones affect emotion recognition. Both Mandarin tones and emotions were identified more accurately in syllables presented with the carrier phrase, but the carrier phrase affected Mandarin tone identification and emotion recognition to different degrees. These findings suggest that lexical tones and emotions interact in complex but systematic ways.

An Efficient Grid-Based Geocasting Scheme for Wireless Sensor Networks.

In a wireless sensor network (WSN), geocasting is a location-based routing protocol used for data collection or information delivery. In geocasting, a target region usually contains many sensor nodes with limited battery capacity, and sensor nodes in multiple target regions need to transmit data to the sink. Therefore, how to use location information to construct an energy efficient geocasting path is a very important issue. FERMA is a geocasting scheme for WSNs based on Fermat points. In this paper, an efficient grid-based geocasting scheme for WSNs, which is called GB-FERMA, is proposed. The scheme uses the Fermat point theorem to search for the specific nodes as Fermat points in a grid-based WSN, and it selects the optimal relay nodes (gateways) in the grid structure to realize energy-aware forwarding. In the simulations, when the initial power 0.25 J, the average energy consumption of GB-FERMA is about 53% of FERMA-QL, 37% of FERMA, and 23% of GEAR; however, when with the initial power 0.5 J, the average energy consumption of GB-FERMA is about 77% of FERMA-QL, 65% of FERMA, and 43% of GEAR. The proposed GB-FERMA can effectively reduce the energy consumption and thus prolong the lifetime of the WSN.

An Energy Efficient Load Balancing Tree-Based Data Aggregation Scheme for Grid-Based Wireless Sensor Networks.

A wireless sensor network (WSN) consists of a very large number of sensors which are deployed in the specific area of interest. A sensor is an electronic device equipped with a small processor and has a small-capacity memory. The WSN has the functions of low cost, easy deployment, and random reconfiguration. In this paper, an energy-efficient load balancing tree-based data aggregation scheme (LB-TBDAS) for grid-based WSNs is proposed. In this scheme, the sensing area is partitioned into many cells of a grid and then the sensor node with the maximum residual energy is elected to be the cell head in each cell. Then, the tree-like path is established by using the minimum spanning tree algorithm. In the tree construction, it must meet the three constraints, which are the minimum energy consumption spanning tree, the network depth, and the maximum number of child nodes. In the data transmission process, the cell head is responsible for collecting the sensing data in each cell, and the collected data are transmitted along the tree-like path to the base station (BS). Simulation results show that the total energy consumption of LB-TBDAS is significantly less than that of GB-PEDAP and PEDAP. Compared to GB-PEDAP and PEDAP, the proposed LB-TBDAS extends the network lifetime by more than 100%. The proposed LB-TBDAS can avoid excessive energy consumption of sensor nodes during multi-hop data transmission and can also avoid the hotspot problem of WSNs.

Multi-Talker Speech Promotes Greater Knowledge-Based Spoken Mandarin Word Recognition in First and Second Language Listeners.

Spoken word recognition involves a perceptual tradeoff between the reliance on the incoming acoustic signal and knowledge about likely sound categories and their co-occurrences as words. This study examined how adult second language (L2) learners navigate between acoustic-based and knowledge-based spoken word recognition when listening to highly variable, multi-talker truncated speech, and whether this perceptual tradeoff changes as L2 listeners gradually become more proficient in their L2 after multiple months of structured classroom learning. First language (L1) Mandarin Chinese listeners and L1 English-L2 Mandarin adult listeners took part in a gating experiment. The L2 listeners were tested twice - once at the start of their intermediate/advanced L2 language class and again 2 months later. L1 listeners were only tested once. Participants were asked to identify syllable-tone words that varied in syllable token frequency (high/low according to a spoken word corpus) and syllable-conditioned tonal probability (most probable/least probable in speech given the syllable). The stimuli were recorded by 16 different talkers and presented at eight gates ranging from onset-only (gate 1) through onset +40 ms increments (gates 2 through 7) to the full word (gate 8). Mixed-effects regression modeling was used to compare performance to our previous study which used single-talker stimuli (Wiener et al., 2019). The results indicated that multi-talker speech caused both L1 and L2 listeners to rely greater on knowledge-based processing of tone. L1 listeners were able to draw on distributional knowledge of syllable-tone probabilities in early gates and switch to predominantly acoustic-based processing when more of the signal was available. In contrast, L2 listeners, with their limited experience with talker range normalization, were less able to effectively transition from probability-based to acoustic-based processing. Moreover, for the L2 listeners, the reliance on such distributional information for spoken word recognition appeared to be conditioned by the nature of the acoustic signal. Single-talker speech did not result in the same pattern of probability-based tone processing, suggesting that knowledge-based processing of L2 speech may only occur under certain acoustic conditions, such as multi-talker speech.

Processing Lexical and Speaker Information in Repetition and Semantic/Associative Priming.

The purpose of this study is to investigate the interaction between processing lexical and speaker-specific information in spoken word recognition. The specific question is whether repetition and semantic/associative priming is reduced when the prime and target are produced by different speakers. In Experiment 1, the prime and target were repeated (e.g., queen-queen) or unrelated (e.g., bell-queen). In Experiment 2, the prime and target were semantically/associatively related (e.g., king-queen) or unrelated (e.g., bell-queen). In both experiments, the prime and target were either produced by the same male speaker or two different male speakers. Two interstimulus intervals between the prime and target were used to examine the time course of processing speaker information. The tasks for the participants included judging the lexical status of the target (lexical decision), followed by judging whether the prime and target were produced by the same speaker or different speakers (speaker discrimination). The results showed that both lexical decision and speaker discrimination were facilitated to a smaller extent when the prime and target were produced by different speakers, indicating reduced repetition priming by speaker variability. In contrast, semantic/associative priming was not affected by speaker variability. The ISI between the prime and target did not affect either type of priming. In conclusion, speaker variability affects accessing a word's form but not its meaning, suggesting that speaker-specific information is processed at a relatively shallow level.

Subcortical frequency-coding errors are linked to speaker-variability intolerance in normal-hearing adults.

Processing speaker-specific information is an important task in daily communication. This study examined how fundamental frequency (F0) cues were encoded at the subcortical level, as reflected by scalp-recorded frequency-following responses, and their relationship with the listener's ability in processing speech stimuli produced by multiple speakers. By using Mandarin tones with distinctive F0 contours, the results indicated that subcortical frequency-coding errors were significantly correlated with the listener's speaker-variability intolerance for both percent correct and reaction time measures. These findings lay a foundation to help improve the understanding of how speaker information is processed in individuals with normal and impaired auditory systems.

Production of word-initial fricatives of Mandarin Chinese in prelingually deafened children with cochlear implants.

PURPOSE: This study examined the production of fricatives by prelingually deafened Mandarin-speaking children with cochlear implants (CIs). METHOD: Fourteen cochlear implant (CI) children (2.9-8.3 years old) and 60 age-matched normal-hearing (NH) children were recorded producing a list of 13 Mandarin words with four fricatives, /f, s, ɕ, ʂ/, occurring at the syllable-initial position evoked with a picture-naming task. Two phonetically-trained native Mandarin speakers transcribed the fricative productions. Acoustic analysis was conducted to examine acoustic measures including duration, normalised amplitude, spectral peak location and four spectral moments. RESULT: The CI children showed much lower accuracy rates and more diverse error patterns on all four fricatives than their NH peers. Among these four fricatives, both CI and NH children showed the highest rate of mispronunciation of /s/. The acoustic results showed that the speech of the CI children differed from the NH children in spectral peak location, normalised amplitude, spectral mean and spectral skewness. In addition, the fricatives produced by the CI children showed less distinctive patterns of acoustic measures relative to the NH children. CONCLUSION: In general, these results indicate that the CI children have not established distinct categories for the Mandarin fricatives in terms of the place of articulation.

Processing speaker variability in repetition and semantic/associative priming.

The effect of speaker variability on accessing the form and meaning of spoken words was evaluated in two short-term priming experiments. In the repetition priming experiment, participants listened to repeated or unrelated prime-target pairs, in which the prime and target were produced by the same speaker or different speakers. The results showed robust repetition priming, but only partial evidence of reduction of priming by speaker variability. In the semantic/associative priming experiment, participants listened to semantically/associatively related or unrelated prime-target pairs, in which the prime and target were produced by the same speaker or different speakers. The results showed robust semantic/associative priming, but the reduction of priming by speaker variability took place only for targets produced by the female speaker. There is no evidence that the speaker variability effect varied as a function of inter-stimulus interval. These findings suggest that speaker variability could affect access to word form and meaning, but the impact is relatively weak.

Perception of pitch height in lexical and musical tones by English-speaking musicians and nonmusicians.

The purpose of this study was to explore the music-speech relationship by examining pitch height perception in lexical and musical tones. English-speaking musicians and nonmusicians identified multispeaker Taiwanese level tones without typical cues for speaker normalization. The musicians also identified note names of piano, viola, and pure tones without a reference pitch. In the Taiwanese task, both the musicians and nonmusicians were able to identify tone height above chance, but only for tones at the extremes of the speakers' overall vocal range. The musicians only had a slight advantage over the nonmusicians. In the music task, none of the musicians met the criterion for absolute pitch. Timbre did not affect how accurately the musical tones were identified. No correlations were found between performance in the Taiwanese task and that in the music task. It was concluded that musicians' advantage in lexical tone perception arose from the ability to track F0 contours. The ability to identify pitch height in lexical tones appears to involve calibrating acoustic input according to gender-specific, internally stored pitch templates.

"The Sound of Fear": assessing vocal fundamental frequency as a physiological indicator of social anxiety disorder.

The relationship between vocal pitch and social anxiety disorder (SAD) has been examined with encouraging initial results, highlighting increased fundamental frequency (F0) as a physiological indicator of SAD. The present series of studies examined the relationship between F0 emitted during social threat and SAD symptoms. Two independent samples of SAD patients, and a sample of demographically-equivalent non-socially anxious controls (NSACs), completed varying social threat tasks which involved speech. Mean F0 emitted throughout the tasks was examined. Male SAD patients emitted greater F0 in comparison to NSACs across studies. For females, this relationship was significant only when examined in patients with SAD of the generalized subtype, and in response to in vivo social exposures. Furthermore, gender-specific thresholds for overall F0 emitted during social threat were identified which demonstrated excellent differentiation between patients with generalized SAD and NSACs. These results provide additional support for increased F0 as a physiological indicator of SAD.

Effects of speaker variability and noise on Mandarin fricative identification by native and non-native listeners.

Speaker variability and noise are two common sources of acoustic variability. The goal of this study was to examine whether these two sources of acoustic variability affected native and non-native perception of Mandarin fricatives to different degrees. Multispeaker Mandarin fricative stimuli were presented to 40 native and 52 non-native listeners in two presentation formats (blocked by speaker and mixed across speakers). The stimuli were also mixed with speech-shaped noise to create five levels of signal-to- noise ratios. The results showed that noise affected non-native identification disproportionately. By contrast, the effect of speaker variability was comparable between the native and non-native listeners. Confusion patterns were interpreted with reference to the results of acoustic analysis, suggesting native and non-native listeners used distinct acoustic cues for fricative identification. It was concluded that not all sources of acoustic variability are treated equally by native and non-native listeners. Whereas noise compromised non-native fricative perception disproportionately, speaker variability did not pose a special challenge to the non-native listeners.

Perception of musical and lexical tones by Taiwanese-speaking musicians.

This study explored the relationship between music and speech by examining absolute pitch and lexical tone perception. Taiwanese-speaking musicians were asked to identify musical tones without a reference pitch and multispeaker Taiwanese level tones without acoustic cues typically present for speaker normalization. The results showed that a high percentage of the participants (65% with an exact match required and 81% with one-semitone errors allowed) possessed absolute pitch, as measured by the musical tone identification task. A negative correlation was found between occurrence of absolute pitch and age of onset of musical training, suggesting that the acquisition of absolute pitch resembles the acquisition of speech. The participants were able to identify multispeaker Taiwanese level tones with above-chance accuracy, even though the acoustic cues typically present for speaker normalization were not available in the stimuli. No correlations were found between the performance in musical tone identification and the performance in Taiwanese tone identification. Potential reasons for the lack of association between the two tasks are discussed.

The role of speaker gender identification in relative fundamental frequency height estimation from multispeaker, brief speech segments.

A perception experiment was conducted to evaluate the proposal that speaker gender identification underlies the ability to estimate relative F0 height from multispeaker speech without cues typically present for speaker normalization. The Mandarin syllable sa was processed to generate fricative, vowel, and fricative-vowel stimuli. Both Mandarin and English listeners identified gender above chance from vowel and fricative-vowel stimuli. Fricative-vowel stimuli were identified more accurately than vowel stimuli, which were identified more accurately than fricative stimuli. Accuracy was comparable between Mandarin and English listeners, but reaction time showed distinct patterns. The perceptual evidence supports the idea that gender identification contributes to relative F0 height estimation.

The effects of frequency-place shift on consonant confusion in cochlear implant simulations.

The effects of frequency-place shift on consonant recognition and confusion matrices were examined. Frequency-place shift was manipulated using a noise-excited vocoder with 4 to 16 channels. In the vocoder processing, the location of the most apical carrier band varied from the matched condition (i.e., 28 mm from the base of the cochlear) to a basal shift (i.e., 22 mm from the base) in a step size of 1 mm. Ten normal-hearing subjects participated in the 20-alternative forced-choice test, where the consonants were presented in a /Ca/ context. Shift of 3 mm or more caused the consonant recognition scores to decrease significantly. The effects of spectral resolution disappeared when the amount of shift reached >or=3 mm. Information transmitted for voicing and place of articulation varied with spectral shift and spectral resolution, while information transmitted for manner was affected only by spectral shift but not spectral resolution. Spectral shift has shown specific effects on the confusion patterns of the consonants. The direction of errors reversed as spectral shift increased and the patterns of reversal were consistent across channel conditions. Overall, transmission of the consonant features can be accounted for by the acoustic features of the speech signal.

Identification of acoustically modified Mandarin tones by non-native listeners.

This study investigated identification of fragmented Mandarin tones by non-native listeners. Monosyllabic Mandarin words were digitally processed to generate intact, silent-center, center-only, and onset-only syllables. The syllables were recorded with two carrier phrases such that the offset of the carrier tone and the onset of the target tone were either continuous or discontinuous in fundamental frequency (F0). The syllables were presented with an original carrier phrase, excised from the carrier phrase, or excised and cross-spliced with another carrier phrase. Response accuracy and reaction time were measured, and tone confusion patterns were analyzed. Overall, tone identification varied as a function of modification and tone. Intact and center-only syllables were identified more accurately than silent-center and onset-only syllables. Tone 2 was consistently the most challenging tone to identify. Although the performance level of the third-year students approached that of native listeners reported in Lee, Tao, and Bond (2008), the non-native listeners did not show evidence of using coarticulatory information. Nonetheless, the continuity or discontinuity in F0 between the carrier and target tones did affect tone identification, suggesting the influence of context in non-native tone identification.

Perception of musical pitch and lexical tones by Mandarin-speaking musicians.

The relationship between music and language processing was explored in two perception experiments on the identification of musical notes and Mandarin tones. In the music task, Mandarin-speaking musicians were asked to identify musical notes of three timbres without a reference pitch. 72% of the musicians met the criterion for absolute pitch when an exact match was required, and 82% met the criterion when one-semitone errors were allowed. Accuracy of identification was negatively correlated with age of onset of musical training, and piano notes were identified more accurately than viola and pure tone stimuli. In the Mandarin task, the musicians were able to identify, beyond chance, brief Mandarin tone stimuli that were devoid of dynamic F0 information and cues commonly considered necessary for speaker normalization. Although F0 height detection was involved in both musical note and Mandarin tone identification, performances in the two tasks were not correlated. The putative link between absolute pitch and tone language experience was discussed.

Identifying isolated, multispeaker Mandarin tones from brief acoustic input: a perceptual and acoustic study.

Lexical tone identification relies primarily on the processing of F0. Since F0 range differs across individuals, the interpretation of F0 usually requires reference to specific speakers. This study examined whether multispeaker Mandarin tone stimuli could be identified without cues commonly considered necessary for speaker normalization. The sa syllables, produced by 16 speakers of each gender, were digitally processed such that only the fricative and the first six glottal periods remained in the stimuli, neutralizing the dynamic F0 contrasts among the tones. Each stimulus was presented once, in isolation, to 40 native listeners who had no prior exposure to the speakers' voices. Chi-square analyses showed that tone identification accuracy exceeded chance as did tone classification based on F0 height. Acoustic analyses showed contrasts between the high- and low-onset tones in F0, duration, and two voice quality measures (F1 bandwidth and spectral tilt). Correlation analyses showed that F0 covaried with the voice quality measures and that tone classification based on F0 height also correlated with these acoustic measures. Since the same acoustic measures consistently distinguished the female from the male stimuli, gender detection may be implicated in F0 height estimation when no context, dynamic F0, or familiarity with speaker voices is available.

Validity of eye movement methods and indices for capturing semantic (associative) priming effects.

PURPOSE: The purpose of this investigation was to evaluate the usefulness of eye movement methods and indices as a tool for studying priming effects by verifying whether eye movement indices capture semantic (associative) priming effects in a visual cross-format (written word to semantically related picture) priming paradigm. METHOD: In the stimuli development phase, words semantically associated to an array of pictures were generated based on 100 adults' association data for each picture. A total of 40 additional adult participants with normal language engaged in an eye movement experiment using the word-picture associations developed previously. The design consisted of each prime preceding a display showing 1 high-association (target) and 2 low-association (nontarget) images. Fixation durations, locations, and latencies were measured. RESULTS: Images semantically related to the prime showed greater fixation durations and shorter latencies compared to nontargets. Eye movement and traditional reaction time measures were found to correlate for some of the experimental conditions. CONCLUSIONS: Results showed that free-viewing eye movement measures, in which participants are not instructed to look at anything in particular, hold promise as valid indicators of priming effects. Further research in this area will help to advance language-processing theories in individuals with and without language impairment.

Identification of Mandarin tones by English-speaking musicians and nonmusicians.

This study examined Mandarin tone identification by 36 English-speaking musicians and 36 nonmusicians and musical note identification by the musicians. In the Mandarin task, participants were given a brief tutorial on Mandarin tones and identified the tones of the syllable sa produced by 32 speakers. The stimuli included intact syllables and acoustically modified syllables with limited F0 information. Acoustic analyses showed considerable overlap in F0 range among the tones due to the presence of multiple speakers. Despite no prior experience with Mandarin, the musicians identified intact tones at 68% and silent-center tones at 54% correct, both exceeding chance (25%). The musicians also outperformed the nonmusicians, who identified intact tones at 44% and silent-center tones at 36% correct. These results indicate musical training facilitated lexical tone identification, although the facilitation varied as a function of tone and the type of acoustic input. In the music task, the musicians listened to synthesized musical notes of three timbres and identified the notes without a reference pitch. Average identification accuracy was at chance level even when multiple semitone errors were allowed. Since none of the musicians possessed absolute pitch, the role of absolute pitch in Mandarin tone identification remains inconclusive.