Sensorimotor synchronization: a review of recent research (2006-2012).

Sensorimotor synchronization (SMS) is the coordination of rhythmic movement with an external rhythm, ranging from finger tapping in time with a metronome to musical ensemble performance. An earlier review (Repp, 2005) covered tapping studies; two additional reviews (Repp, 2006a, b) focused on music performance and on rate limits of SMS, respectively. The present article supplements and extends these earlier reviews by surveying more recent research in what appears to be a burgeoning field. The article comprises four parts, dealing with (1) conventional tapping studies, (2) other forms of moving in synchrony with external rhythms (including dance and nonhuman animals' synchronization abilities), (3) interpersonal synchronization (including musical ensemble performance), and (4) the neuroscience of SMS. It is evident that much new knowledge about SMS has been acquired in the last 7 years.

Detecting perturbations in polyrhythms: effects of complexity and attentional strategies.

Jones et al. in Journal of Experimental Psychology Human Perception and Performance 21:293-307, 1995, showed that a temporal perturbation is easier to detect in a 3:2 polyrhythm than in a single-stream isochronous baseline condition if the two isochronous pulse streams forming the polyrhythm are perceptually integrated: integration creates shorter inter-onset interval (IOI) durations that facilitate perturbation detection. The present study examined whether this benefit of integration outweighs the potential costs imposed by the greater IOI heterogeneity and memory demands of more complex polyrhythms. In "Experiment 1", musically trained participants tried to detect perturbations in 3:5, 4:5, 6:5, and 7:5 polyrhythms having one of two different pitch separations between pulse streams, as well as in an isochronous baseline condition. "Experiment 2" included an additional 2:5 polyrhythm, additional pitch separations, and instructions to integrate or segregate the two pulse streams. In both experiments, perturbation detection scores for polyrhythms were below baseline, decreased as polyrhythm complexity increased, and tended to be lower at a smaller pitch separation, with little effect of instructions. Clearly, polyrhythm complexity was the main determinant of detection performance, which is attributed to the interval heterogeneity and/or memory demands of the pattern formed by the integrated pulse streams. In this task, perceptual integration was disadvantageous, but apparently could not be avoided.

Synchronization with competing visual and auditory rhythms: bouncing ball meets metronome.

Synchronization of finger taps with periodically flashing visual stimuli is known to be much more variable than synchronization with an auditory metronome. When one of these rhythms is the synchronization target and the other serves as a distracter at various temporal offsets, strong auditory dominance is observed. However, it has recently been shown that visuomotor synchronization improves substantially with moving stimuli such as a continuously bouncing ball. The present study pitted a bouncing ball against an auditory metronome in a target-distracter synchronization paradigm, with the participants being auditory experts (musicians) and visual experts (video gamers and ball players). Synchronization was still less variable with auditory than with visual target stimuli in both groups. For musicians, auditory stimuli tended to be more distracting than visual stimuli, whereas the opposite was the case for the visual experts. Overall, there was no main effect of distracter modality. Thus, a distracting spatiotemporal visual rhythm can be as effective as a distracting auditory rhythm in its capacity to perturb synchronous movement, but its effectiveness also depends on modality-specific expertise.

A general linear framework for the comparison and evaluation of models of sensorimotor synchronization.

Sensorimotor synchronization (SMS), the temporal coordination of a rhythmic movement with an external rhythm, has been studied most often in tasks that require tapping along with a metronome. Models of SMS use information about the timing of preceding stimuli and responses to predict when the next response will be made. This article compares the theoretical structure and empirical predictions of four two-parameter models proposed in the literature: Michon (Timing in temporal tracking, Van Gorcum, Assen, 1967), Hary and Moore (Br J Math Stat Psychol 40:109-124, 1987b), Mates (Biol Cybern 70:463-473, 1994a; Biol Cybern 70:475-484, 1994b), and Schulze et al. (Mus Percept 22:461-467, 2005). By embedding these models within a general linear framework, the mathematical equivalence of the Michon, Hary and Moore, and Schulze et al. models is demonstrated. The Mates model, which differs from the other three, is then tested empirically with new data from a tapping experiment in which the metronome alternated between two tempi. The Mates model predictions are found to be invalid for about one-third of the trials, suggesting that at least one of the model's underlying assumptions is incorrect. The other models cannot be refuted as easily, but they do not predict some features of the data very accurately. Comparison of the models' predictions in a training/test procedure did not yield any significant differences. The general linear framework introduced here may help in the formulation of new models that make better predictions.

Quantifying phase correction in sensorimotor synchronization: empirical comparison of three paradigms.

Tapping in synchrony with a metronome requires phase error correction, a process often described by a single-parameter autoregressive model. The parameter (α) is a measure of sensorimotor coupling strength. This study compares α estimates obtained from three experimental paradigms: synchronization with (1) a perfectly regular metronome (RM), (2) a perturbed metronome containing phase shifts (PS), and (3) an "adaptively timed" metronome (AT). Musically trained participants performed in each paradigm at four tempi, with baseline interval durations ranging from 400 to 1300 ms. Two estimation methods were applied to each data set. Results showed that all α estimates increased with interval duration. However, the PS paradigm yielded much larger α values than did the AT paradigm, with those from the RM paradigm falling in between. Positional analysis of the PS data revealed that α increased immediately following a phase shift and then decreased sharply. Unexpectedly, all PS α estimates were uncorrelated with the RM and AT estimates, which were strongly correlated. These results suggest that abruptly perturbed sequences engage a different mechanism of phase correction than do regular or continuously modulated sequences.

Anticipatory phase correction in sensorimotor synchronization.

Studies of phase correction in sensorimotor synchronization often introduce timing perturbations that are unpredictable with regard to direction, magnitude, and position in the stimulus sequence. If participants knew any or all of these parameters in advance, would they be able to anticipate perturbations and thus regain synchrony more quickly? In Experiment 1, we asked musically trained participants to tap in synchrony with short isochronous tone sequences containing a phase shift (PS) of -100, -40, 40, or 100 ms and provided advance information about its direction, position, or both (but not about its magnitude). The first two conditions had little effect, but in the third condition participants shifted their tap in anticipation of the PS, though only by about ±40 ms on average. The phase correction response to the residual PS was also enhanced. In Experiment 2, we provided complete advance information about PSs of various magnitudes either at the time of the immediately preceding tone ("late") or at the time of the tone one position back ("early") while also varying sequence tempo. Anticipatory phase correction was generally conservative and was impeded by fast tempo in the "late" condition. At fast tempi in both conditions, advancing a tap was more difficult than delaying a tap. The results indicate that temporal constraints on anticipatory phase correction resemble those on reactive phase correction. While the latter is usually automatic, this study shows that phase correction can also be controlled consciously for anticipatory purposes.

No temporal binding of action consequences to actions in a rhythmic context.

Temporal binding (TB) refers to a subjective contraction of the time that elapses between an action and a delayed sensory consequence of it. The TB effect has been demonstrated primarily in tasks in which a key press triggers a tone after a short delay and in which participants judge the timing of one or both of these events relative to a visual reference (e.g., a rotating clock hand). In the present Experiments 1 and 2, musicians listened instead to an auditory "clock" (a metronome) and occasionally made a tap that triggered a delayed tone. The task was to judge whether that test tone fell before, on, or after the midpoint of the interval between two metronome tones. In a passive control condition, participants judged test tones but did not tap. The hypothesis was that the test tone would be perceived as occurring earlier in the active than in the passive condition. However, there was no difference in perceptual judgments. Experiment 3 used a visual metronome as the reference but again obtained negative results, despite greater uncertainty of judgments. It is suggested that TB of action consequences to actions does not occur when the reference signal is rhythmic because such a context enables participants (musicians, at least) to perceive and judge the actual time of occurrence of the action-triggered tone.

Tapping in synchrony with a perturbed metronome: the phase correction response to small and large phase shifts as a function of tempo.

When tapping is paced by an auditory sequence containing small phase shift (PS) perturbations, the phase correction response (PCR) of the tap following a PS increases with the baseline interonset interval (IOI), leading eventually to overcorrection (B. H. Repp, 2008). Experiment 1 shows that this holds even for fixed-size PSs that become imperceptible as the IOI increases (here, from 400 to 1200 ms). Earlier research has also shown (but only for IOI=500 ms) that the PCR is proportionally smaller for large than for small PSs (B. H. Repp, 2002a, 2002b). Experiment 2 introduced large PSs and found smaller PCRs than in Experiment 1, at all of the same IOIs. In Experiments 3A and 3B, the author investigated whether the change in slope of the sigmoid function relating PCR and PS magnitudes occurs at a fixed absolute or relative PS magnitude across different IOIs (600, 1000, 1400 ms). The results suggest no clear answer; the exact shape of the function may depend on the range of PSs used in an experiment. Experiment 4 examined the PCR in the IOI range from 1000 to 2000 ms and found overcorrection throughout, but with the PCR increasing much more gradually than in Experiment 1. These results provide important new information about the phase correction process and pose challenges for models of sensorimotor synchronization, which presently cannot explain nonlinear PCR functions and overcorrection.

Measuring perceptual centers using the phase correction response.

The perceptual center (P-center) is fundamental to the timing of heterogeneous event sequences, including music and speech. Unfortunately, there is currently no comprehensive and reliable model of P-centers in acoustic events, so P-centers must instead be measured empirically. This study reviews existing measurement methods and evaluates two methods in detail-the rhythm adjustment method and a new method based on the phase correction response (PCR) in a synchronous tapping task. The two methods yielded consistent P-center estimates and showed no evidence of P-center context dependence. The PCR method appears promising because it is accurate and efficient and does not require explicit perceptual judgments. As a secondary result, the magnitude of the PCR is shown to vary systematically with the onset complexity of speech sounds, which presumably reflects the perceived clarity of a sound's P-center.

Comfortable synchronization of cyclic drawing movements with a metronome.

Continuous circle drawing is considered a paragon of emergent timing, whereas the timing of finger tapping is said to be event-based. Synchronization with a metronome, however, must to some extent be event-based for both types of movement. Because the target events in the movement trajectory are more poorly defined in circle drawing than in tapping, circle drawing shows more variable asynchronies with a metronome than does tapping. One factor that may have contributed to high variability in past studies is that circle size, drawing direction, and target point were prescribed and perhaps outside the comfort range. In the present study, participants were free to choose most comfortable settings of these parameters for two continuously drawn shapes, circles and infinity signs, while synchronizing with a regular or intermittently perturbed metronome at four different tempi. Results showed that preferred circle sizes were generally smaller than in previous studies but tended to increase as tempo decreased. Synchronization results were similar for circles and infinity signs, and similar to earlier results for circles drawn within a fixed template (Repp & Steinman, 2010). Comparison with tapping data still showed drawing to exhibit much greater variability and persistence of asynchronies as well as slower phase correction in response to phase shifts in the metronome. With comfort level ruled out as a factor, these differences can now be attributed more confidently to differences in event definition and/or movement dynamics.

Temporal evolution of the phase correction response in synchronization of taps with perturbed two-interval rhythms.

Human sensorimotor synchronization is flexible but subject to temporal constraints. Previous research has shown that musicians tend to lose synchrony with target tones in an isochronous sequence when the sequence rate exceeds 8-10 Hz, presumably because phase correction ceases to function. The present study investigated directly the time required for an immediate phase correction response (PCR). Musicians tapped in synchrony with cyclic two-interval (short-long) rhythms, using the two hands in alternation. Perturbations were applied to the long interval, and the compensatory shift of the next tap (the PCR) was measured following the short interval, whose duration was varied from 100 to 300 ms. The PCR was found to increase gradually within this range, being nearly absent at 100 ms. Similar results were obtained when participants tapped only with the second tone in each rhythmic group, which confirms that the PCR is based on the preceding tone rather than on the preceding tap-tone asynchrony, and also when the second tone was omitted in the pacing sequence, which indicates that the PCR occurs automatically even when there is no synchronization target for the critical tap. These results extend earlier findings regarding rate limits of synchronization and also provide further support for an event-based phase resetting account of the PCR.

Altered sense of agency in schizophrenia and the putative psychotic prodrome.

The mechanisms underlying distortions in sense of agency, i.e. the experience of controlling one's own actions and their consequences, in schizophrenia are not fully understood and have barely been investigated in patients classified as being in a putative psychotic prodrome. This study aims to expound the contribution of early and late illness-related processes. Thirty schizophrenia patients, 30 putatively prodromal patients and 30 healthy controls were instructed to reproduce a computer-generated series of drum sounds on a drum pad. While tapping, subjects heard either their self-produced tones or a computer-controlled reproduction of the drum tone series that used either exactly the same, an accelerated or decelerated tempo. Subjects had to determine the source of agency. Results show similar significant impairments in assigning the source of agency under ambiguous conditions in schizophrenia and putatively prodromal patients and an exaggerated self-attribution bias, both of which were significantly correlated with increased (ego-)psychopathology. Patient groups, however, benefited significantly more than controls from additional sensorimotor cues to agency. Sensorimotor input seems to be a compensatory mechanism involved in correctly attributing agency. We deduce that altered awareness of agency may hold promise as an additional risk factor for psychosis.

Perception-production relationships and phase correction in synchronization with two-interval rhythms.

Two experiments investigated the effects of interval duration ratio on perception of local timing perturbations, accuracy of rhythm production, and phase correction in musicians listening to or tapping in synchrony with cyclically repeated auditory two-interval rhythms. Ratios ranged from simple (1:2) to complex (7:11, 5:13), and from small (5:13 = 0.38) to large (6:7 = 0.86). Rhythm production and perception exhibited similar ratio-dependent biases: rhythms with small ratios were produced with increased ratios, and timing perturbations in these rhythms tended to be harder to detect when they locally increased the ratio than when they reduced it. The opposite held for rhythms with large ratios. This demonstrates a close relation between rhythm perception and production. Unexpectedly, however, the neutral "attractor" was not the simplest ratio (1:2 = 0.50) but a complex ratio near 4:7 (= 0.57). Phase correction in response to perturbations was generally rapid and did not show the ratio-dependent biases observed in rhythm perception and production. Thus, phase correction operates efficiently and autonomously even in synchronization with rhythms exhibiting complex interval ratios.

A filled duration illusion in music: Effects of metrical subdivision on the perception and production of beat tempo.

This study replicates and extends previous findings suggesting that metrical subdivision slows the perceived beat tempo (Repp, 2008). Here, musically trained participants produced the subdivisions themselves and were found to speed up, thus compensating for the perceived slowing. This was shown in a synchronization-continuation paradigm (Experiment 1) and in a reproduction task (Experiment 2a). Participants also judged the tempo of a subdivided sequence as being slower than that of a preceding simple beat sequence (Experiment 2b). Experiment 2 also included nonmusician participants, with similar results. Tempo measurements of famous pianists' recordings of two variation movements from Beethoven sonatas revealed a strong tendency to play the first variation (subdivided beats) faster than the theme (mostly simple beats). A similar tendency was found in musicians' laboratory performances of a simple theme and variations, despite instruc-tions to keep the tempo constant (Experiment 3a). When playing melodic sequences in which only one of three beats per measure was subdivided, musicians tended to play these beats faster and to perceive them as longer than adjacent beats, and they played the whole sequence faster than a sequence without any subdivisions (Experiments 3b and 3c). The results amply demonstrate a filled duration illusion in rhythm perception and music performance: Intervals containing events seem longer than empty intervals and thus must be shortened to be perceived as equal in duration.

Do metrical accents create illusory phenomenal accents?

In music that is perceived as metrically structured, events coinciding with the main beat are called metrically accented. Are these accents purely cognitive, or do they perhaps represent illusory increases in perceived loudness or duration, caused by heightened attention to main beats? In four separate tasks, musicians tried to detect a small actual increase or decrease in the loudness or duration of a single note in melodies comprising 12 notes. Musical notation prescribed a meter (6/8) implying a main beat coinciding with every third note. Effects of metrical accentuation on detection performance were found in all four tasks. However, they reflected primarily an increase in sensitivity to physical changes in main beat positions, likely to be due to enhanced attention. There was no evidence of biases indicating illusory phenomenal accents in those positions. By contrast, and independent of metrical structure, pitch accents due to pitch contour pivots were often mistaken for increases in loudness.

Simultaneous event-based and emergent timing: synchronization, continuation, and phase correction.

It has been claimed that rhythmic tapping and circle drawing represent fundamentally different timing processes (event-based and emergent, respectively) and also that circle drawing is difficult to synchronize with a metronome and exhibits little phase correction. In the present study, musically trained participants tapped with their left hands, drew circles with their right (dominant) hands, and also performed both tasks simultaneously. In Experiment 1, they synchronized with a metronome and then continued on their own, whereas in Experiment 2, they synchronized with a metronome containing phase perturbations. Circle drawing generally exhibited reliable synchronization, although with greater variability than tapping, and also showed a clear phase-correction response that evolved gradually during the cycle immediately following a perturbation. When carried out simultaneously in synchrony, with or without a metronome, the two tasks affected each other in some ways but retained their distinctive timing characteristics. This shows that event-based and emergent timing can coexist in a dual-task situation. Furthermore, the authors argue that the two timing modes usually coexist in each individual task, although one mode is often dominant.

Sensorimotor synchronization and perception of timing: effects of music training and task experience.

To assess individual differences in basic synchronization skills and in perceptual sensitivity to timing deviations, brief tests made up of isochronous auditory sequences containing phase shifts or tempo changes were administered to 31 college students (most of them with little or no music training) and nine highly trained musicians (graduate students of music performance). Musicians showed smaller asynchronies, lower tapping variability, and greater perceptual sensitivity than college students, on average. They also showed faster phase correction following a tempo change in the pacing sequence. Unexpectedly, however, phase correction following a simple phase shift was unusually quick in both groups, especially in college students. It emerged that some of the musicians, who had previous experience with laboratory synchronization tasks, showed a much slower corrective response to phase shifts than did the other musicians. When these others were retested after having gained some task experience, their phase correction was slower than previously. These results show (1) that instantaneous phase correction in response to phase perturbations is more common than was previously believed, and suggest that (2) gradual phase correction is not a shortcoming but reflects a reduction in the strength of sensorimotor coupling afforded by practice.

Self versus other in piano performance: detectability of timing perturbations depends on personal playing style.

Differences between recorded repetitions of one's own movements are detected more readily than are differences between repetitions of others' movements, suggesting improved visual discrimination due to heightened resonance in the observer's action system and/or relatively accurate internal action simulation (Daprati et al. in Conscious Cogn 16:178-188, 2007). In Experiment 1, we attempted to replicate this finding in the auditory modality. Pianists were recorded playing musical excerpts three times and later judged whether pairs of recordings were the same take or different takes of the same excerpt. They were no better at distinguishing different takes of their own playing than those of other pianists' playing, even though discrimination and self-recognition were well above chance. In Experiment 2, the same pianists tried to detect small local timing deviations that had been introduced artificially. They were better at detecting such deviations in their own performances than in those of another pianist, but only if the deviations were placed at points of a pre-existing self-other difference in local timing. In that case, pianists' ability to predict their own characteristic action pattern did aid their perception of temporal irregularity. These results do not support the perceptual sharpening hypothesis of Daprati et al. in the musical domain, but they do suggest that pianists listening to performances generate idiosyncratic temporal expectations, probably through internal action simulation.

Context sensitivity and invariance in perception of octave-ambiguous tones.

Three experiments investigated the influence of unambiguous (UA) context tones on the perception of octave-ambiguous (OA) tones. In Experiment 1, pairs of OA tones spanning a tritone interval were preceded by pairs of UA tones instantiating a rising or falling interval between the same pitch classes. Despite the inherent ambiguity of OA tritone pairs, most participants showed little or no priming when judging the OA tritone as rising or falling. In Experiments 2 and 3, participants compared the pitch heights of single OA and UA tones representing either the same pitch class or being a tritone apart. These judgments were strongly influenced by the pitch range of the UA tones, but only slightly by the spectral center of the OA tones. Thus, the perceived pitch height of single OA tones is context sensitive, but the perceived relative pitch height of two OA tones, as described in previous research on the "tritone paradox," is largely invariant in UA tone contexts.

Top-down control of rhythm perception modulates early auditory responses.

Our perceptions are shaped by both extrinsic stimuli and intrinsic interpretation. The perceptual experience of a simple rhythm, for example, depends upon its metrical interpretation (where one hears the beat). Such interpretation can be altered at will, providing a model to study the interaction of endogenous and exogenous influences in the cognitive organization of perception. Using magnetoencephalography (MEG), we measured brain responses evoked by a repeating, rhythmically ambiguous phrase (two tones followed by a rest). In separate trials listeners were instructed to impose different metrical organizations on the rhythm by mentally placing the downbeat on either the first or the second tone. Since the stimulus was invariant, differences in brain activity between the two conditions should relate to endogenous metrical interpretation. Metrical interpretation influenced early evoked neural responses to tones, specifically in the upper beta range (20-30 Hz). Beta response was stronger (by 64% on average) when a tone was imagined to be the beat, compared to when it was not. A second experiment established that the beta increase closely resembles that due to physical accents, and thus may represent the genesis of a subjective accent. The results demonstrate endogenous modulation of early auditory responses, and suggest a unique role for the beta band in linking of endogenous and exogenous processing. Given the suggested role of beta in motor processing and long-range intracortical coordination, it is hypothesized that the motor system influences metrical interpretation of sound, even in the absence of overt movement.