Perceptual coupling in human dyads: Kinematics does not affect interpersonal synchronization.

The minimal, essential condition for individuals to interact is that they exchange information via at least one sensory channel. Once informational coupling is established, it enables basic forms of coordinated behavior to spontaneously emerge from the interaction. Our previous study revealed different coordination dynamics in dyads engaged in a joint finger-tapping task based on visual versus auditory coupling. This observation led us to propose the 'modality-dependent hypothesis', which posits that coordination dynamics are influenced by the sensory modality mediating informational coupling. However, recognizing that different modalities have inherent differences in accessing spatiotemporal features of perceived movement, we formulated the alternative 'kinematic hypothesis'. This hypothesis posits that differences in dynamics would vanish given equivalent kinematic information across modalities. The study involved forty (N = 40) participants, grouped into twenty (N = 20) dyads, who engaged in a joint finger-tapping task. This task was conducted under varying conditions of visual and auditory coupling, with manipulations in the access to kinematic information, categorized as discrete and continuous. Contrary to our initial predictions, the results strongly supported the 'modality-dependent hypothesis'. We observed that visual and auditory coupling consistently yielded distinct attractor dynamics, regardless of the access to kinematic information. Furthermore, all conditions of auditory coupling resulted in higher levels of synchronization than their visual counterparts. These findings suggest that the differences in interpersonal synchronization are predominantly influenced by the sensory modality, rather than the continuity of kinematic information. Our study highlights the significance of sensorimotor interactions in interpersonal synchronization and addresses the potential of sonification strategies in supporting motor training and rehabilitation.

Embodied perspective-taking enhances interpersonal synchronization: A body-swap study.

Humans exhibit a strong tendency to synchronize movements with each other, with visual perspective potentially influencing interpersonal synchronization. By manipulating the visual scenes of participants engaged in a joint finger-tapping task, we examined the effects of 1st person and 2nd person visual perspectives on their coordination dynamics. We hypothesized that perceiving the partner's movements from their 1st person perspective would enhance spontaneous interpersonal synchronization, potentially mediated by the embodiment of the partner's hand. We observed significant differences in attractor dynamics across visual perspectives. Specifically, participants in 1st person coupling were unable to maintain de-coupled trajectories as effectively as in 2nd person coupling. Our findings suggest that visual perspective influences coordination dynamics in dyadic interactions, engaging error-correction mechanisms in individual brains as they integrate the partner's hand into their body representation. Our results have the potential to inform the development of applications for motor training and rehabilitation.

Concert experiences in virtual reality environments.

Spurred by recent advances in digital technologies, virtual concerts have become established modes for event attendance and represent a rapidly growing segment of the music industry. Yet, up to now, general experience of virtual concert attendees remains largely underexplored. Here, we focus on a subcategory in this domain: music concerts in virtual reality (VR). Our approach is situated within the theoretical framework of embodied music cognition and entailed investigation through a survey study. Responses of seventy-four VR concert attendees were collected, consisting of demographics, motivations, experiences, and future perspectives. In contrast to previous research, which generally identified social connectedness as a main motivator for concert attendance, our sample regarded it as one of the least important incentives. On the other hand, in line with previous studies, 'seeing specific artists perform' and 'uniqueness of the experience', were pivotal. The latter was mostly fueled by the possibility to experience/interact with visuals and environments considered as unconceivable in the real world. Furthermore, 70% of our sample regarded VR concerts as 'the future of the music industry', mainly relating to the accessibility of such events. Positive evaluations of VR concert experiences, as well as future perspectives regarding the medium, were significantly influenced by the level of experienced immersivity. To our knowledge, this is the first study to provide such an account. Supplementary Information: The online version contains supplementary material available at 10.1007/s10055-023-00814-y.

Cholinergic-related pupil activity reflects level of emotionality during motor performance.

Pupil size covaries with the diffusion rate of the cholinergic and noradrenergic neurons throughout the brain, which are essential to arousal. Recent findings suggest that slow pupil fluctuations during locomotion are an index of sustained activity in cholinergic axons, whereas phasic dilations are related to the activity of noradrenergic axons. Here, we investigated movement induced arousal (i.e., by singing and swaying to music), hypothesising that actively engaging in musical behaviour will provoke stronger emotional engagement in participants and lead to different qualitative patterns of tonic and phasic pupil activity. A challenge in the analysis of pupil data is the turbulent behaviour of pupil diameter due to exogenous ocular activity commonly encountered during motor tasks and the high variability typically found between individuals. To address this, we developed an algorithm that adaptively estimates and removes pupil responses to ocular events, as well as a functional data methodology, derived from Pfaffs' generalised arousal, that provides a new statistical dimension on how pupil data can be interpreted according to putative neuromodulatory signalling. We found that actively engaging in singing enhanced slow cholinergic-related pupil dilations and having the opportunity to move your body while performing amplified the effect of singing on pupil activity. Phasic pupil oscillations during motor execution attenuated in time, which is often interpreted as a measure of sense of agency over movement.

The dynamics of musical participation.

In this paper we argue that our comprehension of musical participation-the complex network of interactive dynamics involved in collaborative musical experience-can benefit from an analysis inspired by the existing frameworks of dynamical systems theory and coordination dynamics. These approaches can offer novel theoretical tools to help music researchers describe a number of central aspects of joint musical experience in greater detail, such as prediction, adaptivity, social cohesion, reciprocity, and reward. While most musicians involved in collective forms of musicking already have some familiarity with these terms and their associated experiences, we currently lack an analytical vocabulary to approach them in a more targeted way. To fill this gap, we adopt insights from these frameworks to suggest that musical participation may be advantageously characterized as an open, non-equilibrium, dynamical system. In particular, we suggest that research informed by dynamical systems theory might stimulate new interdisciplinary scholarship at the crossroads of musicology, psychology, philosophy, and cognitive (neuro)science, pointing toward new understandings of the core features of musical participation.

A Methodological Framework for Assessing Social Presence in Music Interactions in Virtual Reality.

Virtual reality (VR) brings radical new possibilities to the empirical study of social music cognition and interaction. In the present article, we consider the role of VR as a research tool, based on its potential to create a sense of "social presence": the illusory feeling of being, and socially interacting, inside a virtual environment. This makes VR promising for bridging ecological validity ("research in the wild") and experimental control ("research in the lab") in empirical music research. A critical assumption however is the actual ability of VR to simulate real-life social interactions, either via human-embodied avatars or computer-controlled agents. The mediation of social musical interactions via VR is particularly challenging due to their embodied, complex, and emotionally delicate nature. In this article, we introduce a methodological framework to operationalize social presence by a combination of factors across interrelated layers, relating to the performance output, embodied co-regulation, and subjective experiences. This framework provides the basis for the proposal of a pragmatic approach to determine the level of social presence in virtual musical interactions, by comparing the outcomes across the multiple layers with the outcomes of corresponding real-life musical interactions. We applied and tested this pragmatic approach via a case-study of piano duet performances of the piece Piano Phase composed by Steve Reich. This case-study indicated that a piano duet performed in VR, in which the real-time interaction between pianists is mediated by embodied avatars, might lead to a strong feeling of social presence, as reflected in the measures of performance output, embodied co-regulation, and subjective experience. In contrast, although a piano duet in VR between an actual pianist and a computer-controlled agent led to a relatively successful performance output, it was inadequate in terms of both embodied co-regulation and subjective experience.

Livestream Experiments: The Role of COVID-19, Agency, Presence, and Social Context in Facilitating Social Connectedness.

Musical life became disrupted in 2020 due to the COVID-19 pandemic. Many musicians and venues turned to online alternatives, such as livestreaming. In this study, three livestreamed concerts were organized to examine separate, yet interconnected concepts-agency, presence, and social context-to ascertain which components of livestreamed concerts facilitate social connectedness. Hierarchical Bayesian modeling was conducted on 83 complete responses to examine the effects of the manipulations on feelings of social connectedness with the artist and the audience. Results showed that in concert 1, where half of the participants were allowed to vote for the final song to be played, this option did not result in the experience of more agency. Instead, if their preferred song was played (regardless of voting ability) participants experienced greater connectedness to the artist. In concert 2, participants who attended the concert with virtual reality headsets experienced greater feelings of physical presence, as well as greater feelings of connectedness with the artist, than those that viewed a normal YouTube livestream. In concert 3, attendance through Zoom led to greater experience of social presence, but predicted less connectedness with the artist, compared to a normal YouTube livestream. Crucially, a greater negative impact of COVID-19 (e.g., loneliness) predicted feelings of connectedness with the artist, possibly because participants fulfilled their social needs with this parasocial interaction. Examining data from all concerts suggested that physical presence was a predictor of connectedness with both the artist and the audience, while social presence only predicted connectedness with the audience. Correlational analyses revealed that reductions in loneliness and isolation were associated with feelings of shared agency, physical and social presence, and connectedness to the audience. Overall, the findings suggest that in order to reduce feelings of loneliness and increase connectedness, concert organizers and musicians could tune elements of their livestreams to facilitate feelings of physical and social presence.

A Breathing Sonification System to Reduce Stress During the COVID-19 Pandemic.

Since sound and music are powerful forces and drivers of human behavior and physiology, we propose the use of sonification to activate healthy breathing patterns in participants to induce relaxation. Sonification is often used in the context of biofeedback as it can represent an informational, non-invasive and real-time stimulus to monitor, motivate or modify human behavior. The first goal of this study is the proposal and evaluation of a distance-based biofeedback system using a tempo- and phase-aligned sonification strategy to adapt breathing patterns and induce states of relaxation. A second goal is the evaluation of several sonification stimuli on 18 participants that were recruited online and of which we analyzed psychometric and behavioral data using, respectively questionnaires and respiration rate and ratio. Sonification stimuli consisted of filtered noise mimicking a breathing sound, nature environmental sounds and a musical phrase. Preliminary results indicated the nature stimulus as most pleasant and as leading to the most prominent decrease of respiration rate. The noise sonification had the most beneficial effect on respiration ratio. While further research is needed to generalize these findings, this study and its methodological underpinnings suggest the potential of the proposed biofeedback system to perform ecologically valid experiments at participants' homes during the COVID-19 pandemic.

The influence of performing gesture type on interpersonal musical timing, and the role of visual contact and tempo.

Bodily gestures play an important role in the communication of expressive intentions between humans. Music ensemble performance, as an outstanding example of nonverbal human communication, offers an exemplary context to study and understand the gestural control and communication of these expressive intentions. An important mechanism in music ensemble performance is the anticipation and control of interpersonal timing. When performing, musicians are involved in a complex system of mutual adaptation which is not completely understood so far. In this study, we investigated the role of performers' gestures in the mediation process of interpersonal timing in a dyad performance. Therefore, we designed an experiment in which we controlled for the use of hand and arm movements in a musical task, in which dyads were asked to synchronously tap out a melody. Next to their comfortable/natural way of tapping, we instructed participants to either perform pronounced expressive hand and arm gestures in between successive taps, or to restrict from any overt body movement. In addition, we looked at effects of visual contact (yes/no) and tempo (slow: 50 beats per minute; fast: 100 beats per minute). The results show that performers' gestures improve interpersonal musical timing, in terms of the consistency and accuracy of onset asynchronies, and of the variability of produced inter-onset intervals. Interestingly, we found that the use of expressive gestures, in regard to comfortable/natural movements, add to these positive timing effects, but only when there is visual contact and at the slow tempo. In addition, we found that the type of gestures employed by musicians may modulate leader-follower dynamics. Together, these findings are explained by human anticipation mechanisms facilitated by gesturing, shedding new light on the principles underlying human communication of expressive intentions, through music.

Detrended fluctuation analysis of gait dynamics when entraining to music and metronomes at different tempi in persons with multiple sclerosis.

In persons with multiple sclerosis (PwMS), synchronizing walking to auditory stimuli such as to music and metronomes have been shown to be feasible, and positive clinical effects have been reported on step frequency and perception of fatigue. Yet, the dynamic interaction during the process of synchronization, such as the coupling of the steps to the beat intervals in music and metronomes, and at different tempi remain unknown. Understanding these interactions are clinically relevant, as it reflects the pattern of step intervals over time, known as gait dynamics. 28 PwMS and 29 healthy controls were instructed to walk to music and metronomes at 6 tempi (0-10% in increments of 2%). Detrended fluctuation analysis was applied to calculate the fractal statistical properties of the gait time-series to quantify gait dynamics by the outcome measure alpha. The results showed no group differences, but significantly higher alpha when walking to music compared to metronomes, and when walking to both stimuli at tempi + 8, + 10% compared to lower tempi. These observations suggest that the precision and adaptation gain differ during the coupling of the steps to beats in music compared to metronomes (continuous compared to discrete auditory structures) and at different tempi (different inter-beat-intervals).

Timing Markers of Interaction Quality During Semi-Hocket Singing.

Music is believed to work as a bio-social tool enabling groups of people to establish joint action and group bonding experiences. However, little is known about the quality of the group members' interaction needed to bring about these effects. To investigate the role of interaction quality, and its effect on joint action and bonding experience, we asked dyads (two singers) to perform music in medieval "hocket" style, in order to engage their co-regulatory activity. The music contained three relative inter-onset-interval (IOI) classes: quarter note, dotted quarter note and eight note, marking time intervals between successive onsets (generated by both singers). We hypothesized that singers co-regulated their activity by minimizing prediction errors in view of stable IOI-classes. Prediction errors were measured using a dynamic Bayesian inference approach that allows us to identify three different types of error called fluctuation (micro-timing errors measured in milliseconds), narration (omission errors or misattribution of an IOI to a wrong IOI class), and collapse errors (macro-timing errors that cause the breakdown of a performance). These three types of errors were correlated with the singers' estimated quality of the performance and the experienced sense of joint agency. We let the singers perform either while moving or standing still, under the hypothesis that the moving condition would have reduced timing errors and increased We-agency as opposed to Shared-agency (the former portraying a condition in which the performers blend into one another, the latter portraying a joint, but distinct, control of the performance). The results show that estimated quality correlates with fluctuation and narration errors, while agency correlates (to a lesser degree) with narration errors. Somewhat unexpectedly, there was a minor effect of movement, and it was beneficial only for good performers. Joint agency resulted in a "shared," rather than a "we," sense of joint agency. The methodology and findings open up promising avenues for future research on social embodied music interaction.

The sonic instructor: A music-based biofeedback system for improving weightlifting technique.

In this study, we assumed that correct functional movements for weightlifting can be learned with the help of a music-based biofeedback system. We compared musical feedback with verbal feedback from experienced trainers using two independent groups. The focus was on one specific movement called deadlift. Physical parameters under considerations were the spine (i.e. loss of midline stability resulting in flexion) and the forward displacement of the barbell during the repetitions relative to the mid-foot. We recruited 31 recreational weight lifters (21-42 years of age). Results revealed that both feedback types are effective in improving the movements for deadlift. No significant differences were found across the two feedback types, neither in terms of movement, nor in terms of clarity and motivation. The results suggest that the proposed feedback system is a valid tool for technology-aided training and self-training practices.

Continuous 12 min walking to music, metronomes and in silence: Auditory-motor coupling and its effects on perceived fatigue, motivation and gait in persons with multiple sclerosis.

BACKGROUND: In Persons with Multiple Sclerosis (PwMS), coupling walking to beats/pulses in short bursts is reported to be beneficial for cadence and perceived fatigue. However it is yet to be investigated if coupling and its effects can be sustained for longer durations, required for task-oriented training strategy in PwMS. AIMS: To investigate if PwMS compared to healthy controls (HC) sustain synchronization for 12 min when walking to music and metronome, and its effects on perceived physical and cognitive fatigue, motivation and gait compared to walking in silence. METHODS: Participants walked for 12 min in three conditions (music, metronome and silence). The tempo of the auditory conditions was individualized. Auditory-motor coupling and spatio-temporal gait parameters were measured during walking. The visual analogue scale was used for perceived fatigue, and the Likert scale for motivation. RESULTS: 27 PwMS and 28 HC participated. All participants synchronized to both stimuli, yet PwMS synchronized better to music. Overall, participants had lower cadence, speed and stride length when over time all conditions, with an exception of HC, with increasing cadence during the music condition. PwMS perceived less cognitive fatigue, no difference in perceived physical fatigue and a higher motivation walking to music compared to metronomes and silence. CONCLUSION: 12 min of uninterrupted walking was possible in PwMS in all conditions, while better synchronization, low perception of cognitive fatigue and high motivation occurred with music compared to other conditions. Coupling walking to music could offer novel paradigms for motor task-oriented training in PwMS.

Walking to Music and Metronome at Various Tempi in Persons With Multiple Sclerosis: A Basis for Rehabilitation.

Background. Mobility dysfunctions are prevalent in persons with multiple sclerosis (PwMS), thus novel rehabilitation mechanisms are needed toward functional training. The effect of auditory cueing is well-known in Parkinson's disease, yet the application of different types of auditory stimuli at different tempi has not been investigated yet. Objectives. Investigating if PwMS, compared with healthy controls (HC), can synchronize their gait to music and metronomes at different tempi during walking and the effects of the stimuli on perceived fatigue and gait. Additionally, exploring if cognitive impairment would be a factor on the results. Methods. The experimental session consisted of 2 blocks, music and metronomes. Per block, participants walked 3 minutes per tempi, with instructions to synchronize their steps to the beat. The tempi were 0%, +2%, +4% +6%, +8%, +10% of preferred walking cadence (PWC). Results. A total of 28 PwMS and 29 HC participated. On average, participants were able to synchronize at all tempi to music and metronome. Higher synchronization was obtained for metronomes compared with music. The highest synchronization for music was found between +2% and +8% of PWC yet pwMS perceived less physical and cognitive fatigue walking to music compared with metronomes. Cognitive impaired PwMS (n = 9) were not able to synchronize at tempi higher than +6%. Conclusion. Auditory-motor coupling and synchronization was feasible in HC and PwMS with motor and cognitive impairments. PwMS walked at higher tempi than their preferred walking cadence, and lower fatigue perception with music. Coupling walking to music could be a promising functional walking training strategy.

3Mo: A Model for Music-Based Biofeedback.

In the domain of sports and motor rehabilitation, it is of major importance to regulate and control physiological processes and physical motion in most optimal ways. For that purpose, real-time auditory feedback of physiological and physical information based on sound signals, often termed "sonification," has been proven particularly useful. However, the use of music in biofeedback systems has been much less explored. In the current article, we assert that the use of music, and musical principles, can have a major added value, on top of mere sound signals, to the benefit of psychological and physical optimization of sports and motor rehabilitation tasks. In this article, we present the 3Mo model to describe three main functions of music that contribute to these benefits. These functions relate the power of music to Motivate, and to Monitor and Modify physiological and physical processes. The model brings together concepts and theories related to human sensorimotor interaction with music, and specifies the underlying psychological and physiological principles. This 3Mo model is intended to provide a conceptual framework that guides future research on musical biofeedback systems in the domain of sports and motor rehabilitation.

Sensorimotor Grounding of Musical Embodiment and the Role of Prediction: A Review.

In a previous article, we reviewed empirical evidence demonstrating action-based effects on music perception to substantiate the musical embodiment thesis (Maes et al., 2014). Evidence was largely based on studies demonstrating that music perception automatically engages motor processes, or that body states/movements influence music perception. Here, we argue that more rigorous evidence is needed before any decisive conclusion in favor of a "radical" musical embodiment thesis can be posited. In the current article, we provide a focused review of recent research to collect further evidence for the "radical" embodiment thesis that music perception is a dynamic process firmly rooted in the natural disposition of sounds and the human auditory and motor system. Though, we emphasize that, on top of these natural dispositions, long-term processes operate, rooted in repeated sensorimotor experiences and leading to learning, prediction, and error minimization. This approach sheds new light on the development of musical repertoires, and may refine our understanding of action-based effects on music perception as discussed in our previous article (Maes et al., 2014). Additionally, we discuss two of our recent empirical studies demonstrating that music performance relies on similar principles of sensorimotor dynamics and predictive processing.

Auditory and motor contributions to the timing of melodies under cognitive load.

Current theoretical models and empirical research suggest that sensorimotor control and feedback processes may guide time perception and production. In the current study, we investigated the role of motor control and auditory feedback in an interval-production task performed under heightened cognitive load. We hypothesized that general associative learning mechanisms enable the calibration of time against patterns of dynamic change in motor control processes and auditory feedback information. In Experiment 1, we applied a dual-task interference paradigm consisting of a finger-tapping (continuation) task in combination with a working memory task. Participants (nonmusicians) had to either perform or avoid arm movements between successive key presses (continuous vs. discrete). Auditory feedback from a key press (a piano tone) filled either the complete duration of the target interval or only a small part (long vs. short). Results suggested that both continuous movement control and long piano feedback tones contributed to regular timing production. In Experiment 2, we gradually adjusted the duration of the long auditory feedback tones throughout the duration of a trial. The results showed that a gradual shortening of tones throughout time increased the rate at which participants performed tone onsets. Overall, our findings suggest that the human perceptual-motor system may be important in guiding temporal behavior under cognitive load.

The impact of cognitive load on operatic singers' timing performance.

In the present paper, we report the results of an empirical study on the effects of cognitive load on operatic singing. The main aim of the study was to investigate to what extent a working memory task affected the timing of operatic singers' performance. Thereby, we focused on singers' tendency to speed up, or slow down their performance of musical phrases and pauses. Twelve professional operatic singers were asked to perform an operatic aria three times; once without an additional working memory task, once with a concurrent working memory task (counting shapes on a computer screen), and once with a relatively more difficult working memory task (more shapes to be counted appearing one after another). The results show that, in general, singers speeded up their performance under heightened cognitive load. Interestingly, this effect was more pronounced in pauses-more in particular longer pauses-compared to musical phrases. We discuss the role of sensorimotor control and feedback processes in musical timing to explain these findings.

The role of working memory in the temporal control of discrete and continuous movements.

Music performance requires precise control of limb movements in order to achieve temporal precision of performed tone onsets. Previous findings suggest that processes recruited for the temporal control of rhythmic body movements, such as those required in music performance, depend on the movement type (discrete vs. continuous) and the rate of the produced interonset intervals (sub-second vs. supra-second). Using a dual-task paradigm, the current study addressed these factors in the temporal control of cellists' bowing movements. Cellists performed melodies in a synchronization-continuation timing task at a specified fast (intertone interval = 700 ms) or slow (intertone interval = 1,100 ms) tempo with either discrete (staccato) or continuous (legato) bowing movements. A secondary working memory task involved a concurrent digit-switch counting task. Analyses of the produced tone durations showed that the working memory load significantly impaired temporal regularity when the melodies were performed with discrete bowing movements at the slower tempo. In addition, discrete movements led to more errors on the working memory task. These findings suggest that continuous body movements provide temporal control information to performers under high cognitive load conditions.

Expressive body movement responses to music are coherent, consistent, and low dimensional.

Embodied music cognition stresses the role of the human body as mediator for the encoding and decoding of musical expression. In this paper, we set up a low dimensional functional model that accounts for 70% of the variability in the expressive body movement responses to music. With the functional principal component analysis, we modeled individual body movements as a linear combination of a group average and a number of eigenfunctions. The group average and the eigenfunctions are common to all subjects and make up what we call the commonalities. An individual performance is then characterized by a set of scores (the individualities), one score per eigenfunction. The model is based on experimental data which finds high levels of coherence/consistency between participants when grouped according to musical education. This shows an ontogenetic effect. Participants without formal musical education focus on the torso for the expression of basic musical structure (tempo). Musically trained participants decode additional structural elements in the music and focus on body parts having more degrees of freedom (such as the hands). Our results confirm earlier studies that different body parts move differently along with the music.