Temporal Coordination in Piano Duet Networked Music Performance (NMP): Interactions Between Acoustic Transmission Latency and Musical Role Asymmetries.

Today's audio, visual, and internet technologies allow people to interact despite physical distances, for casual conversation, group workouts, or musical performance. Musical ensemble performance is unique because interaction integrity critically depends on the timing between each performer's actions and when their acoustic outcomes arrive. Acoustic transmission latency (ATL) between players is substantially longer for networked music performance (NMP) compared to traditional in-person spaces where musicians can easily adapt. Previous work has shown that longer ATLs slow the average tempo in ensemble performance, and that asymmetric co-actor roles and empathy-related traits affect coordination patterns in joint action. Thus, we are interested in how musicians collectively adapt to a given latency and how such adaptation patterns vary with their task-related and person-related asymmetries. Here, we examined how two pianists performed duets while hearing each other's auditory outcomes with an ATL of 10, 20, or 40 ms. To test the hypotheses regarding task-related asymmetries, we designed duets such that pianists had: (1) a starting or joining role and (2) a similar or dissimilar musical part compared to their co-performer, with respect to pitch range and melodic contour. Results replicated previous clapping-duet findings showing that longer ATLs are associated with greater temporal asynchrony between partners and increased average tempo slowing. While co-performer asynchronies were not affected by performer role or part similarity, at the longer ATLs starting performers displayed slower tempos and smaller tempo variability than joining performers. This asymmetry of stability vs. flexibility between starters and joiners may sustain coordination, consistent with recent joint action findings. Our data also suggest that relative independence in musical parts may mitigate ATL-related challenges. Additionally, there may be a relationship between co-performer differences in empathy-related personality traits such as locus of control and coordination during performance under the influence of ATL. Incorporating the emergent coordinative dynamics between performers could help further innovation of music technologies and composition techniques for NMP.

Delayed feedback embedded in perception-action coordination cycles results in anticipation behavior during synchronized rhythmic action: A dynamical systems approach.

Dancing and playing music require people to coordinate actions with auditory rhythms. In laboratory perception-action coordination tasks, people are asked to synchronize taps with a metronome. When synchronizing with a metronome, people tend to anticipate stimulus onsets, tapping slightly before the stimulus. The anticipation tendency increases with longer stimulus periods of up to 3500ms, but is less pronounced in trained individuals like musicians compared to non-musicians. Furthermore, external factors influence the timing of tapping. These factors include the presence of auditory feedback from one's own taps, the presence of a partner performing coordinated joint tapping, and transmission latencies (TLs) between coordinating partners. Phenomena like the anticipation tendency can be explained by delay-coupled systems, which may be inherent to the sensorimotor system during perception-action coordination. Here we tested whether a dynamical systems model based on this hypothesis reproduces observed patterns of human synchronization. We simulated behavior with a model consisting of an oscillator receiving its own delayed activity as input. Three simulation experiments were conducted using previously-published behavioral data from 1) simple tapping, 2) two-person alternating beat-tapping, and 3) two-person alternating rhythm-clapping in the presence of a range of constant auditory TLs. In Experiment 1, our model replicated the larger anticipation observed for longer stimulus intervals and adjusting the amplitude of the delayed feedback reproduced the difference between musicians and non-musicians. In Experiment 2, by connecting two models we replicated the smaller anticipation observed in human joint tapping with bi-directional auditory feedback compared to joint tapping without feedback. In Experiment 3, we varied TLs between two models alternately receiving signals from one another. Results showed reciprocal lags at points of alternation, consistent with behavioral patterns. Overall, our model explains various anticipatory behaviors, and has potential to inform theories of adaptive human synchronization.

Detecting silent seizures by their sound.

OBJECTIVE: The traditional approach to interpreting electroencephalograms (EEGs) requires physicians with formal training to visually assess the waveforms. This approach can be less practical in critical settings where a trained EEG specialist is not readily available to review the EEG and diagnose ongoing subclinical seizures, such as nonconvulsive status epilepticus. METHODS: We have developed a novel method by which EEG data are converted to sound in real time by letting the underlying electrophysiological signal modulate a voice tone that is in the audible range. Here, we explored whether individuals without any prior EEG training could listen to 15-second sonified EEG and determine whether the EEG represents seizures or nonseizure conditions. We selected 84 EEG samples to represent seizures (n = 7), seizure-like activity (n = 25), or nonperiodic, nonrhythmic activity (normal or focal/generalized slowing, n = 52). EEGs from single channels in the left and right hemispheres were then converted to sound files. After a 4-minute training video, medical students (n = 34) and nurses (n = 30) were asked to designate each audio sample as "seizure" or "nonseizure." We then compared their performance with that of EEG-trained neurologists (n = 12) and medical students (n = 29) who also diagnosed the same EEGs on visual display. RESULTS: Nonexperts listening to single-channel sonified EEGs detected seizures with remarkable sensitivity (students, 98% ± 5%; nurses, 95% ± 14%) compared to experts or nonexperts reviewing the same EEGs on visual display (neurologists, 88% ± 11%; students, 76% ± 19%). If the EEGs contained seizures or seizure-like activity, nonexperts listening to sonified EEGs rated them as seizures with high specificity (students, 85% ± 9%; nurses, 82% ± 12%) compared to experts or nonexperts viewing the EEGs visually (neurologists, 90% ± 7%; students, 65% ± 20%). SIGNIFICANCE: Our study confirms that individuals without EEG training can detect ongoing seizures or seizure-like rhythmic periodic patterns by listening to sonified EEG. Although sonification of EEG cannot replace the traditional approaches to EEG interpretation, it provides a meaningful triage tool for fast assessment of patients with suspected subclinical seizures.

Effect of temporal separation on synchronization in rhythmic performance.

A variety of short time delays inserted between pairs of subjects were found to affect their ability to synchronize a musical task. The subjects performed a clapping rhythm together from separate sound-isolated rooms via headphones and without visual contact. One-way time delays between pairs were manipulated electronically in the range of 3 to 78 ms. We are interested in quantifying the envelope of time delay within which two individuals produce synchronous performances. The results indicate that there are distinct regimes of mutually coupled behavior, and that 'natural time delay'--delay within the narrow range associated with travel times across spatial arrangements of groups and ensembles--supports the most stable performance. Conditions outside of this envelope, with time delays both below and above it, create characteristic interaction dynamics in the mutually coupled actions of the duo. Trials at extremely short delays (corresponding to unnaturally close proximity) had a tendency to accelerate from anticipation. Synchronization lagged at longer delays (larger than usual physical distances) and produced an increasingly severe deceleration and then deterioration of performed rhythms. The study has implications for music collaboration over the Internet and suggests that stable rhythmic performance can be achieved by 'wired ensembles' across distances of thousands of kilometers.

Neural dynamics of event segmentation in music: converging evidence for dissociable ventral and dorsal networks.

The real world presents our sensory systems with a continuous stream of undifferentiated information. Segmentation of this stream at event boundaries is necessary for object identification and feature extraction. Here, we investigate the neural dynamics of event segmentation in entire musical symphonies under natural listening conditions. We isolated time-dependent sequences of brain responses in a 10 s window surrounding transitions between movements of symphonic works. A strikingly right-lateralized network of brain regions showed peak response during the movement transitions when, paradoxically, there was no physical stimulus. Model-dependent and model-free analysis techniques provided converging evidence for activity in two distinct functional networks at the movement transition: a ventral fronto-temporal network associated with detecting salient events, followed in time by a dorsal fronto-parietal network associated with maintaining attention and updating working memory. Our study provides direct experimental evidence for dissociable and causally linked ventral and dorsal networks during event segmentation of ecologically valid auditory stimuli.