Processes underlying adaptation to tempo changes in sensorimotor synchronization.

In synchronizing finger taps with an auditory sequence, a small sudden tempo ("step") change in the sequence tends to be followed by rapid adaptation of the tapping period but slow adaptation of the relative phase of the taps, whereas a larger step change leads to initial period overshoot followed by rapid adaptation of both period and phase [M.H. Thaut, R.A. Miller, L.M. Schauer, Biological Cybernetics 79 (1998a) 241-250]. Experiment 1 replicated these findings and showed that the transition between the two patterns of adaptation occurs near the perceptual detection threshold for a tempo change. A reasonable explanation of these data was provided by a dual-process model of internal error correction [J. Mates, Biological Cybernetics 70 (1994a) 463-473, 70 (1994b) 475-484], with the added assumption that one process (period correction) depends on conscious awareness of a tempo change whereas the other (phase correction) does not. This assumption received support in Experiment 2, where a synchronization-continuation tapping task was used in combination with perceptual judgments to probe into the process of period correction following step changes. The results led to the conclusion that rapid adaptation of the tapping period to a small, undetected tempo change is in fact due to rapid internal phase correction, whereas slow adaptation of the relative phase of the taps is due to slow internal period correction.

Phase correction, phase resetting, and phase shifts after subliminal timing perturbations in sensorimotor synchronization.

Recent studies of synchronized finger tapping have shown that perceptually subliminal phase shifts in an auditory sequence are rapidly compensated for in the motor activity (B. H. Repp, 2000a). Experiment 1 used a continuation-tapping task to confirm that this compensation is indeed a phase correction, not an adjustment of the central timekeeper period. Experiments 2-5 revealed that this phase correction occurs even when there is no ordinary sensorimotor asynchrony--when the finger taps are in antiphase or arbitrary phase relative to the auditory sequence (Experiments 2 and 3) or when the tap coinciding with the sequence phase shift is withheld (Experiments 4 and 5). The phase correction observed in the latter conditions was instantaneous, which suggests that phase resetting occurs when the motor activity is discontinuous. A prolonged phase shift suggestive of overcompensation was observed in some conditions, which poses a challenge to pure phase correction models.

Compensation for subliminal timing perturbations in perceptual-motor synchronization.

It is sometimes assumed that limits of temporal discrimination established in psychophysical tasks constrain the timing information available for the control of action. Results from the five perceptual-motor synchronization experiments presented here argue against this assumption. Experiment 1 demonstrates that subliminal (0.8-2%) local changes in interval duration in an otherwise isochronous auditory sequence are rapidly compensated for in the timing of synchronized finger tapping. If this compensation is based on perception of the highly variable synchronization error (SE) rather than of the local change in stimulus period, then it could be based solely on SEs that exceed the temporal order threshold. However, that hypothesis is ruled out by additional analyses of Exp. 1 and the results of Exp. 2, a combined synchronization and temporal order judgment task. Experiments 3-5 further show that three factors that affect the detectability of local deviations from stimulus isochrony do not inhibit effective compensation for such deviations in synchronized tapping. Experiment 5, a combined synchronization and detection task, shows directly that compensation for timing perturbations does not depend on explicit detection. Overall, the results suggest that the automatic processes involved in the temporal control of action have access to more accurate timing information than do the conscious decision processes of auditory temporal judgment.

A microcosm of musical expression. III. Contributions of timing and dynamics to the aesthetic impression of pianists' performances of the initial measures of Chopin's Etude in E major.

Four judges repeatedly assessed the overall aesthetic quality of more than 100 recorded performances of the opening of Chopin's Etude in E major on a 10-point scale. The judgments, which exhibited reasonable reliability and modest intercorrelations, were entered into regression analyses with 16 independent variables derived from earlier objective analyses of the expressive timing and dynamics of the performances [Repp, J. Acoust. Soc. Am. 104, 1085-1100 (1998); 105, 1972-1988 (1999)]. Only between 9% and 18% of the variance in the judges' ratings was accounted for. By contrast, timing variables accounted for 53% of the variance in one judge's ratings of synthesized performances that varied in timing only and mimicked the timing patterns of the original performances. These results indicate, first, that the aesthetic impression of the original recordings rested primarily on aspects other than those measured (such as texture, tone, or aspects of timing and dynamics that eluded the earlier analyses) and, second, that very different patterns of timing and dynamics are aesthetically acceptable for the same music, provided that other, aesthetically more crucial performance aspects are present.

Detecting deviations from metronomic timing in music: effects of perceptual structure on the mental timekeeper.

The detectability of a deviation from metronomic timing--of a small local increment in interonset interval (IOI) duration--in a musical excerpt is subject to positional biases, or "timing expectations," that are closely related to the expressive timing (sequence of IOI durations) typically produced by musicians in performance (Repp, 1992b, 1998c, 1998d). Experiment 1 replicated this finding with some changes in procedure and showed that the perception-performance correlation is not the result of formal musical training or availability of a musical score. Experiments 2 and 3 used a synchronization task to examine the hypothesis that participants' perceptual timing expectations are due to systematic modulations in the period of a mental timekeeper that also controls perceptual-motor coordination. Indeed, there was systematic variation in the asynchronies between taps and metronomically timed musical event onsets, and this variation was correlated both with the variations in IOI increment detectability (Experiment 1) and with the typical expressive timing pattern in performance. When the music contained local IOI increments (Experiment 2), they were almost perfectly compensated for on the next tap, regardless of their detectability in Experiment 1, which suggests a perceptual-motor feedback mechanism that is sensitive to subthreshold timing deviations. Overall, the results suggest that aspects of perceived musical structure influence the predictions of mental timekeeping mechanisms, thereby creating a subliminal warping of experienced time.

A microcosm of musical expression: II. Quantitative analysis of pianists' dynamics in the initial measures of Chopin's Etude in E major.

Patterns of expressive dynamics were measured in bars 1-5 of 115 commercially recorded performances of Chopin's Etude in E major, op. 10, No. 3. The grand average pattern (or dynamic profile) was representative of many performances and highly similar to the average dynamic profile of a group of advanced student performances, which suggests a widely shared central norm of expressive dynamics. The individual dynamic profiles were subjected to principal components analysis, which yielded Varimax-rotated components, each representing a different, nonstandard dynamic profile associated with a small subset of performances. Most performances had dynamic patterns resembling a mixture of several components, and no clustering of of performances into distinct groups was apparent. Some weak relationships of dynamic profiles with sociocultural variables were found, most notably a tendency of female pianists to exhibit a greater dynamic range in the melody. Within the melody, there were no significant relationships between expressive timing [Repp, J. Acoust. Soc. Am. 104, 1085-1100 (1998)] and expressive dynamics. These two important dimensions seemed to be controlled independently at this local level and thus offer the artist many degrees of freedom in giving a melody expressive shape.

A microcosm of musical expression. I. Quantitative analysis of pianists' timing in the initial measures of Chopin's Etude in E major.

Patterns of expressive timing were measured in bars 1-5 of 115 commercially recorded performances of Chopin's Etude in E major, op. 10, No. 3. These patterns were subjected to principal components analysis, which suggested at least four independent "timing strategies": (1) major ritards at the ends of melodic gestures; (2) acceleration within some of these gestures, without final ritards; (3) extreme lengthening of the initial downbeat; and (4) ritards between as well as within melodic gestures. Strategies 1 and 4 respond in different ways to the melodic-rhythmic grouping structure of the music, and strategy 3 merely represents a local emphasis. Strategy 2 is the one most difficult to rationalize; it does not seem to represent an alternative structural interpretation of the music but rather an alternative gestural shaping. Each individual pianist's timing pattern could be described as a weighted combination of these four strategies plus idiosyncratic variation. A wide variety of combinations was represented, and no two individual patterns were exactly the same. In addition, there was a wide range of basic tempi and of degrees of tempo modulation. There were no strong relationships between any of these variables and sociocultural characteristics of the artists, although some weak trends were observed.

Variations on a theme by chopin: relations between perception and production of timing in music.

A note interonset interval (IOI) increment in mechanically timed music is more difficult to detect where expressive lengthening typically occurs in artistic performance. Experiment 1 showed this in an excerpt from a Chopin etude and extended the task to IOI decrement detection. A simple measure of variation in perceptual bias was derived that correlated highly with the average timing pattern of pianists' performances, more so than with acoustic surface properties of the music. Similar results, but decreasing correlations, were obtained in each of four subsequent experiments in which the music was simplified in stages. Although local psychoacoustic effects on time perception cannot be ruled out completely, the results suggest that musical structure (melodic-rhythmic grouping in particular) has temporal implications that are reflected not only in musicians' motor behavior but also in listeners' time-keeping abilities.

Obligatory "expectations" of expressive timing induced by perception of musical structure.

Previous research has shown that, in a task requiring the detection of local deviations from mechanically precise timing in music, the relative detectability of deviations in different positions is closely related to the typical expressive timing pattern musicians produce when playing the music. This result suggests that listeners expect to hear music expressively timed and compensate for the absence of expressive timing. Three new detection experiments shed additional light on the nature of these timing expectations in musically trained listeners. Experiment 1 shows that repeated exposure to an atypically (but not unmusically) timed performance leaves listeners' timing expectations unaffected. Experiment 2 demonstrates that the expectations do not manifest themselves when listeners merely imagine the music in synchrony with a click track. Experiment 3, however, shows that the timing expectations are fully operational when the click track is superimposed on the music. These results reveal timing "expectations" to be an obligatory consequence of the ongoing auditory perception of musical structure.

The perceptual prominence of fundamental frequency peaks.

Five perception experiments were conducted that investigated how the perceived prominence of F0 maxima in accented syllables in Dutch is affected by the variation of F0 minima that is supposed to relate to variation in global pitch range. The purpose of the first two experiments was to test the predictions of a model in which the reference line is directly given by an interpolation between observable F0 minima. The results showed that the model was inadequate, and confirmed earlier research suggesting that the reference line is calculated in a less direct way. The next three experiments investigated the role of the F0 of the unaccented portions of speech at the beginning ("onset") and at the end ("offset") of the contour, and show that only the (low) onset is used to calibrate the reference line. The results also suggest that longer onsets affect the abstract reference more than do shorter onsets.

Acoustics, perception, and production of legato articulation on a computer-controlled grand piano.

In an attempt to replicate and extend previous results obtained on a digital piano [B. H. Repp, J. Acoust. Soc. Am. 97, 3862-3874 (1995)], the present study analyzed piano tone decay characteristics, musically trained listeners' interactive adjustments of key overlap times (KOTs) in tone sequences, and samples of pianists' legato playing in scales and arpeggi on a computer-controlled Yamaha Disklavier. On the whole, the results resembled the earlier findings: In both perception and production, KOTs tended to be longer for high rather than for low tones and for relatively consonant rather than for dissonant successive tones. KOTs also increased as tempo decreased in production, but there was no corresponding effect in perception (where a smaller range of tempi was used). Even though the decay times of the natural piano tones were about twice as long as those of the digital piano used earlier, the average KOTs were not shorter; on the contrary, they were longer in the perception task, while there was little difference in production. Perception of optimal legato does not seem to rest on an invariant criterion of acoustic tone overlap, and pianists do not seem to adjust their KOTs substantially when playing on different instruments. However, there were large individual differences in KOTs.

The effect of tempo on pedal timing in piano performance.

The temporal coordination of hand and foot actions in piano performance is an interesting instance of highly practiced, perceptually guided complex motor behavior. To gain some insight into the nature of this coordination, ten pianists were asked to play two excerpts from the piano literature that required repeated use of the damper pedal to connect successive chords. Each excerpt was played at three prescribed tempos on a Yamaha Disklavier and was recorded in MIDI format. The question of interest was whether and how changes in tempo would affect the timing of pedal releases and depressions within the periods defined by successive manual chord onsets. Theoretical possibilities ranged from absolute invariance (variable phase relationships) to relative invariance of pedal timing (constant phase relationships). The results show that, typically, the timing of pedal actions is neither absolutely nor relatively invariant: As the tempo increases, both pedal releases and depressions usually occur a little sooner and pedal changes (release-depression sequences) are executed a little more quickly, but these effects are proportionally smaller than the changes in manual (and pedal) period duration. Since this may be due to unequal changes in peripheral hand and foot kinematics with tempo, it remains possible that there is invariance of either kind at the level of central motor commands. However, it is the peripheral timing that produces the acoustic consequences musicians try to achieve.

Spectral envelope and context effects in the tritone paradox.

In previous studies of the 'tritone paradox' Deutsch has suggested that, when listeners are presented with pairs of octave-complex tones that are equal in average log frequency but differ in chroma by 6 semitones (a tritone), they perceive the direction of the chroma difference according to an individual pitch-class template. However, it has also been found that the perceived direction changes for many listeners when the spectral envelope of the tones is shifted along the frequency axis. Reanalysis of these data indicates a strong tendency to perceive the pitch class corresponding to the frequency on which the spectral envelope is centered as subjectively lowest. In experiment 1 this spectral-envelope effect was replicated with tone pairs presented in isolation, at the rate of one a day, which rules out artifacts of test format. In experiment 2, involving another context-free format, envelope center frequency was varied over a wide range and it was shown that some individuals are totally envelope dependent, whereas others rely more on pitch class, and yet others show mixed patterns. Experiment 3 demonstrated that listeners' judgments of tritone pairs can be swayed easily by preceding context. Finally, experiment 4 showed that strong envelope effects are also obtained with Deutsch's own tritone test (issued on CD). The subjective relative pitch height of octave-complex tones thus depends on several competing factors, only one of which is pitch class.

Patterns of note onset asynchronies in expressive piano performance.

This paper presents an analysis of the asynchronies among nominally simultaneous notes in ten graduate student pianists' performances of three compositions (Schumann's "Träumerei," Debussy's "La fille aux cheveux de lin," and Chopin's Prelude in D-flat major), each repeated twice and recorded in MIDI format on a Yamaha Disklavier. Note onset times were sensed from hammer motion shortly before hammer-string contact. A pervasive tendency was found for the highest-pitched notes (usually the principal melody) to lead lower-pitched notes, especially those played with the same hand. Inner notes of within-hand chords tended to lag behind outer notes. Strong correlations between average MIDI velocity difference and average lead time were found within each hand, as well as between hands for some of the pianists. Other pianists had a tendency to lead with the left hand, independent of MIDI velocity. These individual differences in between-hand coordination were stable across the three compositions and did not reflect handedness. The results suggest that within-hand asynchronies and melody leads are largely a consequence of dynamic differentiation of voices (i.e., an artifact of hammer travel time), whereas left-hand leads are an individual characteristic and, in part, a deliberate expressive strategy exhibited by some pianists.

The dynamics of expressive piano performance: Schumann's "Träumerei" revisited.

Ten graduate student pianists were recorded playing Robert Schumann's "Träumerei" three times on a Yamaha Disclavier. Their expressive dynamics were analyzed at the level of hammer (MIDI) velocities. Individual dynamic profiles were similar across repeated performances, more so for the right hand (soprano and alto voices) than for the left hand (tenor and bass voices). As expected, the soprano voice, which usually had the principal melody, was played with greater force than the other voices, which gained prominence only when they carried temporarily important melodic fragments. Independent of this voice differentiation, there was a tendency for velocity to increase with pitch, at least in the soprano and alto voices. While there was an overall tendency for velocities to increase with local tempo, there were salient local departures from this coupling. Individual differences in expressive dynamics were not striking and were only weakly related to individual differences in expressive timing.

Detectability of duration and intensity increments in melody tones: a partial connection between music perception and performance.

Two experiments demonstrate positional variation in the relative detectability of, respectively, local temporal and dynamic perturbations in an isochronous and isodynamic sequence of melody tones, played on a computer-controlled piano. This variation may reflect listeners' expectations of expressive performance microstructure (the top-down hypothesis), or it may be due to psychoacoustic (pitch-related) stimulus factors (the bottom-up hypothesis). Percent correct scores for increments in tone duration correlated significantly with the average timing profile of pianists' expressive performances of the music, as predicted specifically by the top-down hypothesis. For intensity increments, the analogous perception-performance correlation was weak and the bottom-up factors of relative pitch height and/or direction of pitch change accounted for some of the perceptual variation. Subjects' musical training increased overall detection accuracy but did not affect the positional variation in accuracy scores in either experiment. These results are consistent with the top-down hypothesis for timing, but they favor the bottom-up hypothesis for dynamics. The perception-performance correlation for timing may also be viewed as being due to complex stimulus properties such as tonal motion and tension/relaxation that influence performers and listeners in similar ways.

Expressive timing in Schumann's "Träumerei:" an analysis of performances by graduate student pianists.

Statistical analyses were conducted on the expressive timing patterns of performances of Schumann's "Träumerei" by ten graduate student pianists who played from the score on a Yamaha Disclavier after a brief rehearsal. A previous study of acoustic recordings of "Träumerei" by 24 famous pianists [B. H. Repp, J. Acoust. Soc. Am. 92, 2546-2568 (1992)] provided "expert" timing data for comparison. In terms of group average timing pattern, individual shaping of ritardandi, and within-performance consistency, the students turned out to be quite comparable to the experts. This demonstrates that precision in expressive timing does not require extensive study and practice of the music at hand, only general musical and technical competence. Subsequent principal components analyses on the students' timing patterns revealed that they were much more homogeneous than the experts'. Individual differences among student pianists seemed to represent mainly variations around a common performance standard (the first principal component), whereas expert performances exhibited a variety of underlying timing patterns, especially at a detailed level of analysis. Experienced concert artists evidently feel less constrained by a performance norm, which makes their performances more interesting and original, hence less typical. Since the norm may represent the most natural or prototypical timing pattern, relatively spontaneous performances by young professionals may be a better starting point for modeling expressive timing than distinguished artists' performances.

Acoustics, perception, and production of legato articulation on a digital piano.

This study investigated the perception and production of legato ("connected") articulation in repeatedly ascending and descending tone sequences on a digital piano (Roland RD-250s). Initial measurements of the synthetic tones revealed substantial decay times following key release. High tones decayed faster than low tones, as they did prior to key release, and long tones decayed sooner than short tones because of their more extensive prerelease decay. Musically trained subjects (including pianists) were asked to adjust the key overlap times (KOTs) of successive piano tones so that they sounded optimally, minimally, or maximally legato. The results supported two predictions based on the acoustic measurements: KOTs for successive tones judged to be optimally or maximally legato were greater for high than for low tones, and greater for long than for short tones, so that auditory overlap presumably remained more nearly constant. For minimal legato adjustments the effect of tone duration was reversed, however. Adjusted KOTs were also longer for relatively consonant tones (three semitones separation) than for dissonant tones (one semitone separation). Subsequently, KOTs were measured in skilled pianists' legato productions of tone sequences similar to those in the perceptual experiment. KOTs clearly increased with tone duration, an effect that was probably motoric in origin. There was no effect of tone height, suggesting that the pianists did not immediately adjust to differences in acoustic overlap. KOTs were slightly shorter for dissonant than for consonant tones. They also varied with position in the ascending-descending tone sequences, indicating that the pianists exerted strategic control over KOT as a continuous expressive dimension.(ABSTRACT TRUNCATED AT 250 WORDS)

Relational invariance of expressive microstructure across global tempo changes in music performance: an exploratory study.

This study addressed the question of whether the expressive microstructure of a music performance remains relationally invariant across moderate (musically acceptable) changes in tempo. Two pianists played Schumann's "Träumerei" three times at each of three tempi on a digital piano, and the performance data were recorded in MIDI format. In a perceptual test, musically trained listeners attempted to distinguish the original performances from performances that had been artificially speeded up or slowed down to the same overall duration. Accuracy in this task was barely above chance, suggesting that relational invariance was largely preserved. Subsequent analysis of the MIDI data confirmed that each pianist's characteristic timing patterns were highly similar across the three tempi, although there were statistically significant deviations from perfect relational invariance. The timing of (relatively slow) grace notes seemed relationally invariant, but selective examination of other detailed temporal features (chord asynchrony, tone overlap, pedal timing) revealed no systematic scaling with tempo. Finally, although the intensity profile seemed unaffected by tempo, a slight overall increase in intensity with tempo was observed. Effects of musical structure on expressive microstructure were large and pervasive at all levels, as were individual differences between the two pianists. For the specific composition and range of tempi considered here, these results suggest that major (cognitively controlled) temporal and dynamic features of a performance change roughly in proportion with tempo, whereas minor features tend to be governed by tempo-independent motoric constraints.