Auditory and visual similarity of pitch contours.

It has been shown that visual display systems of intonation can be employed beneficially in teaching intonation to persons with deafness and in teaching the intonation of a foreign language. In this paper, the question is addressed whether important audible differences between two pitch contours correspond with visually conspicuous differences between displayed pitch contours. If visual feedback of intonation is to be effective in teaching situations, such correspondence must exist. In two experiments, phoneticians rated the dissimilarity of two pitch contours. In the first experiment they rated the two pitch contours auditorily (i.e., by listening to two resynthesized utterances). In the second, they rated the same two pitch contours visually (i.e., by looking at the two contours displayed on a computer screen). The results indicate why visual feedback may be very effective in intonation training if pitch contours are displayed in such a way that only auditorily relevant features are represented.

Measuring the perceptual similarity of pitch contours.

It has been shown that visual display systems of intonation can be employed beneficially in teaching intonation to persons with deafness and in teaching the intonation of a foreign language. In current training situations the correctness of a reproduced pitch contour is rated either by the teacher or automatically. In the latter case an algorithm mostly estimates the maximum deviation from an example contour. In game-like exercises, for instance, the pupil has to produce a pitch contour within the displayed floor and ceiling of a "tunnel" with a preadjusted height. In an experiment described in the companion paper, phoneticians had rated the dissimilarity of two pitch contours both auditorily, by listening to two resynthesized utterances, and visually, by looking at two pitch contours displayed on a computer screen. A test is reported in which these dissimilarity ratings were compared with automatic ratings obtained with this tunnel measure and with three other measures, the mean distance, the root-mean-square (RMS) distance, and the correlation coefficient. The most frequently used tunnel measure appeared to have the weakest correlation with the ratings by the phoneticians. In general, the automatic ratings obtained with the correlation coefficient showed the strongest correlation with the perceptual ratings. A disadvantage of this measure, however, may be that it normalizes for the range of the pitch contours. If range is important, as in intonation teaching to persons with deafness, the mean distance or the RMS distance are the best physical measures for automatic training of intonation.

Timing of pitch movements and accentuation of syllables in Dutch.

In this study, the relation between the timing of a rising or falling pitch movement and the syllable it accentuates is investigated. The five-syllable utterance /mamamamama/ was provided with a relatively fast rising or falling pitch movement. The timing of the movement was systematically varied and Dutch subjects were asked to indicate which syllable they perceived as accented. In order to find out where in the pitch movement the cue which induces the percept of accentuation is located, the duration of the pitch movement was varied. In order to find out which segments of the utterance this characteristic is linked to, the duration of the /m/ was varied. The results showed that the percept of accentuation is induced by a change in pitch at the start of the movement. The moment at which the course of pitch starts to change significantly determines which syllable is perceived as accented. If this moment lies some tens of milliseconds before the P-center, i.e., the perceptual moment of occurrence of the syllable, the preceding syllable is perceived as accented. For a rise, a high accent is perceived; for a fall, a low accent. If the pitch change occurs after this moment, the syllable with this P-center is perceived as accented. For the rise, a low accent is then perceived; for the fall, a high accent. This will be discussed in the light of earlier research on accentuation and of theoretical knowledge about pitch accents.

Prominence lent by rising and falling pitch movements: Testing two models.

Two experiments are reported in which a pitch-level difference (PLD) model for prominence perception [Hermes and Rump, J. Acoust. Soc. Am. 96, 83-92 (1994)] is subjected to further tests. The model holds that the contribution of pitch to the perceived degree of prominence is proportional to the difference in pitch level between the vocalic nuclei of the accented and the previous syllable. In experiment I, the influence of stretching and compressing the utterance in time was assessed. It was found that the predictions made by the model were not fully supported by the data. An alternative model was developed according to which pitch movements resynthesized in the same register lend equal prominence when pitch levels on the upper declination lines in the stimuli are equal. These two models gave different predictions when the lower declination lines are different. This was tested in experiment II. The results which are more or less between the predictions by the two models suggest that low pitch levels play a smaller role in prominence perception than high pitch levels do.

Vowel-onset detection by vowel-strength measurement, cochlear-nucleus simulation, and multilayer perceptrons.

An algorithm for detection of vowel onsets in fluent speech was presented by Hermes [j. Acoust. Soc. Am. 87, 866-873 (1990)]. Performance tests showed that detection was good for fluent speech, although the parameter settings had to be modified for application to well-articulated speech. One of the purposes of the algorithm was application to speech by deaf persons, for which it failed completely. In order to improve the algorithm and to make it more generally applicable, two alternative detection strategies have been explored in the present study. These strategies were (a) simulation of transient-chopper responses in the cochlear nucleus and (B) training of multilayer perceptrons. Two large databases of read speech have been used for performance comparison of the original algorithm and the new strategies. The strategy based on simulating cochlear-nucleus responses is found both to result in a higher false-alarm rate than the original algorithm and to be rather level dependent. On the other hand, the performance of a multilayer-perceptron network, trained on mel-scaled spectra, is comparable to the performance of the Hermes algorithm. In more general terms, the results suggest that temporal information on intensity and (rough) spectral envelope are important for human vowel-onset detection behavior. In formation on harmonicity can be used as a secondary source of information to avoid detection of mainly unvoiced, nonvowel onsets.

Teaching intonation to young deaf children with the intonation meter.

Incorrect production of intonation contours is a common phonatory problem in prelingually, profoundly deaf speakers. To help deaf speakers improve this, a visual display system for teaching intonation has been developed. In this system, called the Intonation Meter, visual feedback of intonation is given as a continuous representation of the pitch contour containing only the perceptually relevant aspects of the intonation pattern. This pitch-contour representation is supposed to facilitate the interpretation of the visual feedback of the pitch contour. A study was carried out, using a Single-Subject Design, in which subjects alternately received intonation training by means of regular methods and intonation training by means of regular methods in which also use was made of the Intonation Meter, to evaluate the effectiveness of the Intonation Meter for teaching intonation to young deaf children. Prelingually profoundly deaf children aged 6 to 7 years and 9 to 11 years participated in this study. The results showed that the 9 to 11 year old children showed most progress when the Intonation meter was used in intonation training whereas 6 to 7 year olds progressed well irrespective of whether or not the Intonation Meter was used, which is in accordance with the theory of a critical period for language learning. Alternatively, it is hypothesized that the cognitive requirements of the visual feedback might be to advanced for very young children to be helpful in learning to produce certain pitch contours.

The frequency scale of speech intonation.

In intonation research, prominence-lending pitch movements have either been described on a linear or on a logarithmic frequency scale. An experiment has been carried out to check whether pitch movements in speech intonation are perceived on one of these two scales or on a psychoacoustic scale representing the frequency selectivity of the auditory system. This last scale is intermediary between the other two scales. Subjects matched the excursion size of prominence-lending pitch movements in utterances resynthesized in different pitch registers. Their task was to adjust the excursion size in a comparison stimulus in such a way that it lent equal prominence to the corresponding syllable in a fixed test stimulus. The comparison stimulus and the test stimulus had pitches running parallel on either the logarithmic frequency scale, the psychoacoustic scale, or the linear frequency scale. In one-half of the experimental sessions, the test stimulus was presented in the low register, while the comparison stimulus was presented in the high register, and, conversely, for the other half of the sessions. The result is that, in all cases, stimuli are matched in such a way that the average excursion sizes in different registers are equal on the psychoacoustic scale.

Vowel-onset detection.

An algorithm is presented that correctly detects the large majority of vowel onsets in fluent speech. The algorithm is based on the simple assumption that vowel onsets are characterized by the appearance of rapidly increasing resonance peaks in the amplitude spectrum. Application to carefully articulated, isolated words results in a high number of false alarms, predominantly before consonants that can function as vowels in a different context such as another language or as a syllabic consonant. After applying some modifications in the setting of some parameters, this number of false alarms for isolated words can be reduced significantly, without the risk of a large number of missed detections. The temporal accuracy of the algorithm is better than 20 ms. This accuracy is determined with respect to the perceptual moment of occurrence of a vowel onset as determined by a phonetician.

Measurement of pitch by subharmonic summation.

In order to account for the phenomenon of virtual pitch, various theories assume implicitly or explicitly that each spectral component introduces a series of subharmonics. The spectral-compression method for pitch determination can be viewed as a direct implementation of this principle. The widespread application of this principle in pitch determination is, however, impeded by numerical problems with respect to accuracy and computational efficiency. A modified algorithm is described that solves these problems. Its performance is tested for normal speech and "telephone" speech, i.e., speech high-pass filtered at 300 Hz. The algorithm out-performs the harmonic-sieve method for pitch determination, while its computational requirements are about the same. The algorithm is described in terms of nonlinear system theory, i.c., subharmonic summation. It is argued that the favorable performance of the subharmonic-summation algorithm stems from its corresponding more closely with current pitch-perception theories than does the harmonic sieve.

Spectro-temporal characteristics of single units in the auditory midbrain of the lightly anaesthetised grass frog (Rana temporaria L.) investigated with tonal stimuli.

Responses were obtained from 112 auditory neurons in the midbrain of the grass frog in response to sequences of tones. Their spectro-temporal sensitivities (STS) were determined by a second-order cross-correlation technique. For the majority of units the shape of their action potentials, the degree of timelock to the stimulus and the recording sites were obtained. Two stages of information processing could be distinguished. One was characterized by short latencies (less than 30 ms), strong timelock to the stimulus and many of these units had axon-like action potential waveforms. They were localised in the ventral part of the principal nucleus from the torus semicircularis and in the transition region between laminar and principal nucleus. The other stage comprised units, found all over the torus, with longer latencies, and a weaker timelock to the stimulus. Several units which were predominantly found in the central part of the torus, especially the magnocellular nucleus, showed a broad or multiple STS. Within the principal nucleus a weak tonotopy was found, the dorsoposterior part being sensitive to lower frequencies, the ventroanterior part to the higher frequencies. Binaural-interaction properties are discussed with respect to the eardrum coupling through the mouth cavity. An organisational plan for the torus semicircularis is proposed.

Spectro-temporal characteristics of single units in the auditory midbrain of the lightly anaesthetised grass frog (Rana temporaria L) investigated with noise stimuli.

About 30% of the auditory units in the midbrain of the lightly anaesthetised grass frog respond in a sustained way to stationary pseudorandom noise. This response is described by the spectro-temporal receptive field (STRF), the regions in the spectro-temporal domain where the average second-order functional of those parts of the stimulus ensemble that precede the action potentials differ from the average second-order functional of the stimulus ensemble. By means of the STRF frequency selectivity, postactivation suppression and lateral suppression can quantitatively be studied under one and the same experimental condition. Auditory units that respond to stationary noise are localised in those parts of the torus where fibres enter from the olivary nucleus. They are characterised by relatively short latencies to tones and probably represent the first information-processing stage in the torus semicircularis.

Spectro-temporal characterization of auditory neurons: redundant or necessary.

For neurons in the auditory midbrain of the grass frog the use of a combined spectro-temporal characterization has been evaluated against the separate characterizations of frequency-sensitivity and temporal response properties. By factoring the joint density function of stimulus intensity, I (f, t), preceding a spike, into two marginal density functions I1(f) and I2(t) one may under the assumption of statistical independence reconstruct the joint density by multiplication: I1(f).I2(t). The reconstructed I(f, t) is compared to the original I(f, t) for 83 neurons: in 23% thereof the I(f, t) appeared to be vastly different from I(f, t). These units appeared to be located dominantly in the ventral parts of the auditory midbrain and had a latency exceeding 30 ms. On the basis of the action-potential wave forms the absence of non-separable I(f, t) in the incoming nerve fiber population is concluded. A spectro-temporal characterization of auditory neurons seems mandatory for investigations in and central from the auditory midbrain.