Interperformer coordination in piano-singing duo performances: phrase structure and empathy impact.

Musicians' body motion plays a fundamental role in ensemble playing, by supporting sound production, communication, and expressivity. This research investigates how Western classical musicians' head motion during ensemble performances relates to a piece's phrase structure and musicians' empathic perspective taking (EPT) profile. Twenty-four advanced piano and singing students took part in the study, and their EPT score was pre-assessed using the Interpersonal Reactivity Index. High and low EPT duos were formed, and musicians were paired with a co-performer from the same and the other EPT group. Musicians rehearsed Fauré's Automne and Schumann's Die Kartenlegerin, and performed the pieces one time before and three times after rehearsal. Motion capture data of the musicians' front head, audio, and MIDI recordings of the performances were collected and analysed. Similarity in musicians' head motion and tendency to lead/lag their co-performer were computed by extracting, respectively, power and phase difference of the cross-wavelet transforms of the velocity curves of each paired marker. Results demonstrate that the power of interperformer coordination corresponds to the piece's phrase levels and that singer's EPT can impact the leader-follower relationships between musicians, depending on piece and take number. In the Fauré piece, the higher the singer's EPT score, the higher the tendency for the singer to lead and pianist to follow in take 3, and the lower the tendency for the singer to lead and pianist to follow in take 2. These results contribute to a further understanding of the mechanisms underpinning social interactions, by revealing the complexity of the association between empathy and body motion in ensembles in promoting and diffusing leadership between musicians.

Body motion of choral singers.

Recent investigations on music performances have shown the relevance of singers' body motion for pedagogical as well as performance purposes. However, little is known about how the perception of voice-matching or task complexity affects choristers' body motion during ensemble singing. This study focussed on the body motion of choral singers who perform in duo along with a pre-recorded tune presented over a loudspeaker. Specifically, we examined the effects of the perception of voice-matching, operationalized in terms of sound spectral envelope, and task complexity on choristers' body motion. Fifteen singers with advanced choral experience first manipulated the spectral components of a pre-recorded short tune composed for the study, by choosing the settings they felt most and least together with. Then, they performed the tune in unison (i.e., singing the same melody simultaneously) and in canon (i.e., singing the same melody but at a temporal delay) with the chosen filter settings. Motion data of the choristers' upper body and audio of the repeated performances were collected and analyzed. Results show that the settings perceived as least together relate to extreme differences between the spectral components of the sound. The singers' wrists and torso motion was more periodic, their upper body posture was more open, and their bodies were more distant from the music stand when singing in unison than in canon. These findings suggest that unison singing promotes an expressive-periodic motion of the upper body.

Judgment of togetherness in performances by musical duos.

Musicians experience varying degrees of togetherness with their co-performers when playing in ensembles. However, little is known about how togetherness is experienced by audiences and how interpersonal dynamics in body motion and sound support the judgment of togetherness. This research investigates audience sensitivity to audio and visual markers of interperformer coordination and expressivity in ensembles, in relation to modality of stimulus presentation and audience music background. A set of duo ensemble performances, comprising motion capture recordings of the musicians' upper bodies and instruments, were presented to participants with varying music background, including novices and semi-professional musicians. Participants were required to: (i) watch and listen, (ii) only watch, and (iii) only listen to the selected recordings, whilst providing dynamic ratings of how much togetherness between musicians they perceived. Results demonstrate that sound intensity and similarity in right arm motion (quantified using cross-wavelet transform analysis) were significant predictors of rated togetherness in novices, whilst sound synchronization and chest motion coordination predicted togetherness responses in semi-professional musicians. These results suggest the relevance of the quality of body motion coordination and of certain features of the audio outputs in the audience perception of togetherness. This research contributes to a better understanding of the perceptual mechanisms supporting socio-cognitive judgments of joint action activities.

Eye gaze as a means of giving and seeking information during musical interaction.

During skilled music ensemble performance, a multi-layered network of interaction processes allows musicians to negotiate common interpretations of ambiguously-notated music in real-time. This study investigated the conditions that encourage visual interaction during duo performance. Duos recorded performances of a new piece before and after a period of rehearsal. Mobile eye tracking and motion capture were used in combination to map uni- and bidirectional eye gaze patterns. Musicians watched each other more during temporally-unstable passages than during regularly-timed passages. They also watched each other more after rehearsal than before. Duo musicians may seek visual interaction with each other primarily, but not exclusively, when coordination is threatened by temporal instability. Visual interaction increases as musicians become familiar with the piece, suggesting that they visually monitor each other once a shared interpretation of the piece is established. Visual monitoring of co-performers' movements and attention may facilitate feelings of engagement and high-level creative collaboration.

Beating time: How ensemble musicians' cueing gestures communicate beat position and tempo.

Ensemble musicians typically exchange visual cues to coordinate piece entrances. "Cueing-in" gestures indicate when to begin playing and at what tempo. This study investigated how timing information is encoded in musicians' cueing-in gestures. Gesture acceleration patterns were expected to indicate beat position, while gesture periodicity, duration, and peak gesture velocity were expected to indicate tempo. Same-instrument ensembles (e.g., piano-piano) were expected to synchronize more successfully than mixed-instrument ensembles (e.g., piano-violin). Duos performed short passages as their head and (for violinists) bowing hand movements were tracked with accelerometers and Kinect sensors. Performers alternated between leader/follower roles; leaders heard a tempo via headphones and cued their partner in nonverbally. Violin duos synchronized more successfully than either piano duos or piano-violin duos, possibly because violinists were more experienced in ensemble playing than pianists. Peak acceleration indicated beat position in leaders' head-nodding gestures. Gesture duration and periodicity in leaders' head and bowing hand gestures indicated tempo. The results show that the spatio-temporal characteristics of cueing-in gestures guide beat perception, enabling synchronization with visual gestures that follow a range of spatial trajectories.

Communication for coordination: gesture kinematics and conventionality affect synchronization success in piano duos.

Ensemble musicians often exchange visual cues in the form of body gestures (e.g., rhythmic head nods) to help coordinate piece entrances. These cues must communicate beats clearly, especially if the piece requires interperformer synchronization of the first chord. This study aimed to (1) replicate prior findings suggesting that points of peak acceleration in head gestures communicate beat position and (2) identify the kinematic features of head gestures that encourage successful synchronization. It was expected that increased precision of the alignment between leaders' head gestures and first note onsets, increased gesture smoothness, magnitude, and prototypicality, and increased leader ensemble/conducting experience would improve gesture synchronizability. Audio/MIDI and motion capture recordings were made of piano duos performing short musical passages under assigned leader/follower conditions. The leader of each trial listened to a particular tempo over headphones, then cued their partner in at the given tempo, without speaking. A subset of motion capture recordings were then presented as point-light videos with corresponding audio to a sample of musicians who tapped in synchrony with the beat. Musicians were found to align their first taps with the period of deceleration following acceleration peaks in leaders' head gestures, suggesting that acceleration patterns communicate beat position. Musicians' synchronization with leaders' first onsets improved as cueing gesture smoothness and magnitude increased and prototypicality decreased. Synchronization was also more successful with more experienced leaders' gestures. These results might be applied to interactive systems using gesture recognition or reproduction for music-making tasks (e.g., intelligent accompaniment systems).

The Tight-interlocked Rhythm Section: Production and Perception of Synchronisation in Jazz Trio Performance.

This study investigates the production and perception of timing, synchronisation and dynamics in jazz trio performances. In a production experiment, six trio combinations of one saxophonist, two bassists, and three drummers were recorded while they performed three popular jazz songs. Onset timing and dynamics of each performer were extracted and analysed. Results showed that the tempo was significantly influenced by the timing of the drummers and all performers showed higher temporal precision on the backbeats. The drummers demonstrated individual swing-ratios, accentuations of beats and intrapersonal asynchronies between simultaneous hi-hat and ride cymbal onsets, which resulted in a hi-hat played 2-26 ms ahead of the pulse of the music. In a subsequent perception test, participants ([Formula: see text]) rated 12 excerpts of the jazz recordings. They selected their preferred version from a pool of stimuli containing the original version, but also manipulations with artificially increased or reduced asynchronies. Stimuli with reduced asynchronies smaller than 19 ms were preferred by the listeners over the original or the fully quantised timing. This suggests that listeners endorse a 'tight-interlocked' jazz rhythm section, with asynchronies smaller than the perceptual threshold (temporal masking), but with natural timing variabilities that makes it distinguishable from a computer-generated playback.

Finger Forces in Clarinet Playing.

Clarinettists close and open multiple tone holes to alter the pitch of the tones. Their fingering technique must be fast, precise, and coordinated with the tongue articulation. In this empirical study, finger force profiles and tongue techniques of clarinet students (N = 17) and professional clarinettists (N = 6) were investigated under controlled performance conditions. First, in an expressive-performance task, eight selected excerpts from the first Weber Concerto were performed. These excerpts were chosen to fit in a 2 × 2 × 2 design (register: low-high; tempo: slow-fast, dynamics: soft-loud). There was an additional condition controlled by the experimenter, which determined the expression levels (low-high) of the performers. Second, a technical-exercise task, an isochronous 23-tone melody was designed that required different effectors to produce the sequence (finger-only, tongue-only, combined tongue-finger actions). The melody was performed in three tempo conditions (slow, medium, fast) in a synchronization-continuation paradigm. Participants played on a sensor-equipped Viennese clarinet, which tracked finger forces and reed oscillations simultaneously. From the data, average finger force (F mean ) and peak force (F max ) were calculated. The overall finger forces were low (F mean = 1.17 N, F max = 3.05 N) compared to those on other musical instruments (e.g., guitar). Participants applied the largest finger forces during the high expression level performance conditions (F mean = 1.21 N). For the technical exercise task, timing and articulation information were extracted from the reed signal. Here, the timing precision of the fingers deteriorated the timing precision of the tongue for combined tongue-finger actions, especially for faster tempi. Although individual finger force profiles were overlapping, the group of professional players applied less finger force overall (F mean = 0.54 N). Such sensor instruments provide useful insights into player-instrument interactions and can also be used in the future to give feedback to students in various learning and practising situations.

When they listen and when they watch: Pianists' use of nonverbal audio and visual cues during duet performance.

Nonverbal auditory and visual communication helps ensemble musicians predict each other's intentions and coordinate their actions. When structural characteristics of the music make predicting co-performers' intentions difficult (e.g., following long pauses or during ritardandi), reliance on incoming auditory and visual signals may change. This study tested whether attention to visual cues during piano-piano and piano-violin duet performance increases in such situations. Pianists performed the secondo part to three duets, synchronizing with recordings of violinists or pianists playing the primo parts. Secondos' access to incoming audio and visual signals and to their own auditory feedback was manipulated. Synchronization was most successful when primo audio was available, deteriorating when primo audio was removed and only cues from primo visual signals were available. Visual cues were used effectively following long pauses in the music, however, even in the absence of primo audio. Synchronization was unaffected by the removal of secondos' own auditory feedback. Differences were observed in how successfully piano-piano and piano-violin duos synchronized, but these effects of instrument pairing were not consistent across pieces. Pianists' success at synchronizing with violinists and other pianists is likely moderated by piece characteristics and individual differences in the clarity of cueing gestures used.

Perception of touch quality in piano tones.

Both timbre and dynamics of isolated piano tones are determined exclusively by the speed with which the hammer hits the strings. This physical view has been challenged by pianists who emphasize the importance of the way the keyboard is touched. This article presents empirical evidence from two perception experiments showing that touch-dependent sound components make sounds with identical hammer velocities but produced with different touch forms clearly distinguishable. The first experiment focused on finger-key sounds: musicians could identify pressed and struck touches. When the finger-key sounds were removed from the sounds, the effect vanished, suggesting that these sounds were the primary identification cue. The second experiment looked at key-keyframe sounds that occur when the key reaches key-bottom. Key-bottom impact was identified from key motion measured by a computer-controlled piano. Musicians were able to discriminate between piano tones that contain a key-bottom sound from those that do not. However, this effect might be attributable to sounds associated with the mechanical components of the piano action. In addition to the demonstrated acoustical effects of different touch forms, visual and tactile modalities may play important roles during piano performance that influence the production and perception of musical expression on the piano.

Context-specific effects of musical expertise on audiovisual integration.

Ensemble musicians exchange auditory and visual signals that can facilitate interpersonal synchronization. Musical expertise improves how precisely auditory and visual signals are perceptually integrated and increases sensitivity to asynchrony between them. Whether expertise improves sensitivity to audiovisual asynchrony in all instrumental contexts or only in those using sound-producing gestures that are within an observer's own motor repertoire is unclear. This study tested the hypothesis that musicians are more sensitive to audiovisual asynchrony in performances featuring their own instrument than in performances featuring other instruments. Short clips were extracted from audio-video recordings of clarinet, piano, and violin performances and presented to highly-skilled clarinetists, pianists, and violinists. Clips either maintained the audiovisual synchrony present in the original recording or were modified so that the video led or lagged behind the audio. Participants indicated whether the audio and video channels in each clip were synchronized. The range of asynchronies most often endorsed as synchronized was assessed as a measure of participants' sensitivities to audiovisual asynchrony. A positive relationship was observed between musical training and sensitivity, with data pooled across stimuli. While participants across expertise groups detected asynchronies most readily in piano stimuli and least readily in violin stimuli, pianists showed significantly better performance for piano stimuli than for either clarinet or violin. These findings suggest that, to an extent, the effects of expertise on audiovisual integration can be instrument-specific; however, the nature of the sound-producing gestures that are observed has a substantial effect on how readily asynchrony is detected as well.

Production and perception of legato, portato, and staccato articulation in saxophone playing.

This paper investigates the production and perception of different articulation techniques on the saxophone. In a production experiment, two melodies were recorded that required different effectors to play the tones (tongue-only actions, finger-only actions, combined tongue and finger actions) at three different tempi. A sensor saxophone reed was developed to monitor tongue-reed interactions during performance. In the slow tempo condition, combined tongue-finger actions showed improved timing, compared to the timing of the tongue alone. This observation supports the multiple timer hypothesis where the tongue's timekeeper benefits from a coupling to the timekeeper of the fingers. In the fast tempo condition, finger-only actions were less precise than tongue-only actions and timing precision of combined tongue-finger actions showed the higher timing variability, close to the level of finger-only actions. This suggests that the finger actions have a dominant influence on the overall timing of saxophone performance. In a listening experiment we investigated whether motor expertise in music performance influences the perception of articulation techniques in saxophone performance. Participants with different backgrounds in music making (saxophonists, musicians not playing the saxophone, and non-musicians) attended an AB-X listening test. They had to discriminate between saxophone phrases played with different articulation techniques (legato, portato, staccato). Participants across all three groups discriminated the sound of staccato articulation well from the sound of portato articulation and legato articulation. Errors occurred across all groups of listeners when legato articulation (no tonguing) and portato articulation (soft tonguing) had to be discriminated. Saxophonists' results were superior compared to the results of the other two groups, suggesting that expertise in saxophone playing facilitated the discrimination task.

Temporal control and hand movement efficiency in skilled music performance.

Skilled piano performance requires considerable movement control to accomplish the high levels of timing and force precision common among professional musicians, who acquire piano technique over decades of practice. Finger movement efficiency in particular is an important factor when pianists perform at very fast tempi. We document the finger movement kinematics of highly skilled pianists as they performed a five-finger melody at very fast tempi. A three-dimensional motion-capture system tracked the movements of finger joints, the hand, and the forearm of twelve pianists who performed on a digital piano at successively faster tempi (7-16 tones/s) until they decided to stop. Joint angle trajectories computed for all adjacent finger phalanges, the hand, and the forearm (wrist angle) indicated that the metacarpophalangeal joint contributed most to the vertical fingertip motion while the proximal and distal interphalangeal joints moved slightly opposite to the movement goal (finger extension). An efficiency measure of the combined finger joint angles corresponded to the temporal accuracy and precision of the pianists' performances: Pianists with more efficient keystroke movements showed higher precision in timing and force measures. Keystroke efficiency and individual joint contributions remained stable across tempo conditions. Individual differences among pianists supported the view that keystroke efficiency is required for successful fast performance.

Tactile feedback and timing accuracy in piano performance.

Sequential actions such as playing a piano or tapping in synchrony to an external signal put high cognitive and motor demands on producers, including the generation of precise timing at a wide variety of rates. Tactile information from the fingertips has been shown to contribute to the control of timing in finger tapping tasks. We addressed the hypothesis that reduction of timing errors is related to tactile afferent information in pianists' finger movements during performance. Twelve pianists performed melodies at four rates in a synchronization-continuation paradigm. The pianists' finger motion trajectories toward the piano keys, recorded with a motion capture system, contained different types and amounts of kinematic landmarks at different performance rates. One landmark, a finger-key (FK) landmark, can occur when the finger makes initial contact with the key surface and changes its acceleration abruptly. Overall, there were more FK landmarks in the pianists' keystrokes, as the performance rate increased. The pianists were divided into two groups: those with low percentages of FK in the medium rates that increased with increasing performance rate and those with persistently high FK percentages. Low-FK pianists showed a positive relationship between increased tactile feedback from the current keystroke and increased temporal accuracy in the upcoming keystroke. These findings suggest that sensory information available at finger-key contact enhances the timing accuracy of finger movements in piano performance.

Touch and temporal behavior of grand piano actions.

This study investigated the temporal behavior of grand piano actions from different manufacturers under different touch conditions and dynamic levels. An experimental setup consisting of accelerometers and a calibrated microphone was used to capture key and hammer movements, as well as the sound signal. Five selected keys were played by pianists with two types of touch ("pressed touch" versus "struck touch") over the entire dynamic range. Discrete measurements were extracted from the accelerometer data for each of the over 2300 recorded tones (e.g., finger-key, hammer-string, and key bottom contact times, maximum hammer velocity). Travel times of the hammer (from finger-key to hammer-string) as a function of maximum hammer velocity varied clearly between the two types of touch, but only slightly between pianos. A travel time approximation used in earlier work [Goebl W., (2001). J. Acoust. Soc. Am. 110, 563-572] derived from a computer-controlled piano was verified. Constant temporal behavior over type of touch and low compression properties of the parts of the action (reflected in key bottom contact times) were hypothesized to be indicators for instrumental quality.

Measurement and reproduction accuracy of computer-controlled grand pianos.

The recording and reproducing capabilities of a Yamaha Disklavier grand piano and a Bösendorfer SE290 computer-controlled grand piano were tested, with the goal of examining their reliability for performance research. An experimental setup consisting of accelerometers and a calibrated microphone was used to capture key and hammer movements, as well as the acoustic signal. Five selected keys were played by pianists with two types of touch ("staccato" and "legato"). Timing and dynamic differences between the original performance, the corresponding MIDI file recorded by the computer-controlled pianos, and its reproduction were analyzed. The two devices performed quite differently with respect to timing and dynamic accuracy. The Disklavier's onset capturing was slightly more precise (+/- 10 ms) than its reproduction (-20 to +30 ms); the Bösendorfer performed generally better, but its timing accuracy was slightly less precise for recording (-10 to 3 ms) than for reproduction (+/- 2 ms). Both devices exhibited a systematic (linear) error in recording over time. In the dynamic dimension, the Bösendorfer showed higher consistency over the whole dynamic range, while the Disklavier performed well only in a wide middle range. Neither device was able to capture or reproduce different types of touch.