Effects of virtual reality working memory task difficulty on the passive processing of irrelevant auditory stimuli.

The virtual reality (VR) environment is claimed to be highly immersive. Participants may thus be potentially unaware of their real, external world. The present study presented irrelevant auditory stimuli while participants were engaged in an easy or difficult visual working memory (WM) task within the VR environment. The difficult WM task should be immersive and require many cognitive resources, thus few will be available for the processing of task-irrelevant auditory stimuli. Sixteen young adults wore a 3D head-mounted VR device. In the easy WM task, the stimuli were nameable objects. In the difficult WM task, the stimuli were abstract objects that could not be easily named. A novel paradigm using event-related potentials (ERPs) was implemented to examine the feasibility of quantifying the extent of processing of task-irrelevant stimuli occurring outside of the VR environment. Auditory stimuli irrelevant to the WM task were presented concurrently at every 1.5 or 12 s in separate conditions. Performance on the WM task varied with task difficulty, with accuracy significantly lower during the difficult task. The auditory ERPs consisted of N1 and a later P2/P3a deflection which were larger when the auditory stimuli were presented slowly. ERPs were unaffected by task difficulty, but significant correlations were found. N1 and P2/P3a amplitudes were smallest when performance on the Easy WM task was highest. It is possible that even the easy WM task was so immersive and required many processing resources that few were available for the co-processing of the task-irrelevant auditory stimuli.

An Immersive and Interactive Platform for Cognitive Assessment and Rehabilitation (bWell): Design and Iterative Development Process.

BACKGROUND: Immersive technologies like virtual reality can enable clinical care that meaningfully aligns with real-world deficits in cognitive functioning. However, options in immersive 3D environments are limited, partly because of the unique challenges presented by the development of a clinical care platform. These challenges include selecting clinically relevant features, enabling tasks that capture the full breadth of deficits, ensuring longevity in a rapidly changing technology landscape, and performing the extensive technical and clinical validation required for digital interventions. Complicating development, is the need to integrate recommendations from domain experts at all stages. OBJECTIVE: The Cognitive Health Technologies team at the National Research Council Canada aims to overcome these challenges with an iterative process for the development of bWell, a cognitive care platform providing multisensory cognitive tasks for adoption by treatment providers. METHODS: The team harnessed the affordances of immersive technologies while taking an interdisciplinary research and developmental approach, obtaining active input from domain experts with iterative deliveries of the platform. The process made use of technology readiness levels, agile software development, and human-centered design to advance four main activities: identification of basic requirements and key differentiators, prototype design and foundational research to implement components, testing and validation in lab settings, and recruitment of external clinical partners. RESULTS: bWell was implemented according to the findings from the design process. The main features of bWell include multimodal (fully, semi, or nonimmersive) and multiplatform (extended reality, mobile, and PC) implementation, configurable exercises that pair standardized assessment with adaptive and gamified variants for therapy, a therapist-facing user interface for task administration and dosing, and automated activity data logging. bWell has been designed to serve as a broadly applicable toolkit, targeting general aspects of cognition that are commonly impacted across many disorders, rather than focusing on 1 disorder or a specific cognitive domain. It comprises 8 exercises targeting different domains: states of attention (Egg), visual working memory (Theater), relaxation (Tent), inhibition and cognitive control (Mole), multitasking (Lab), self-regulation (Butterfly), sustained attention (Stroll), and visual search (Cloud). The prototype was tested and validated with healthy adults in a laboratory environment. In addition, a cognitive care network (5 sites across Canada and 1 in Japan) was established, enabling access to domain expertise and providing iterative input throughout the development process. CONCLUSIONS: Implementing an interdisciplinary and iterative approach considering technology maturity brought important considerations for the development of bWell. Altogether, this harnesses the affordances of immersive technology and design for a broad range of applications, and for use in both cognitive assessment and rehabilitation. The technology has attained a maturity level of prototype implementation with preliminary validation carried out in laboratory settings, with next steps to perform the validation required for its eventual adoption as a clinical tool.

Effector-independent brain network for auditory-motor integration: fMRI evidence from singing and cello playing.

Many everyday tasks share high-level sensory goals but differ in the movements used to accomplish them. One example of this is musical pitch regulation, where the same notes can be produced using the vocal system or a musical instrument controlled by the hands. Cello playing has previously been shown to rely on brain structures within the singing network for performance of single notes, except in areas related to primary motor control, suggesting that the brain networks for auditory feedback processing and sensorimotor integration may be shared (Segado et al. 2018). However, research has shown that singers and cellists alike can continue singing/playing in tune even in the absence of auditory feedback (Chen et al. 2013, Kleber et al. 2013), so different paradigms are required to test feedback monitoring and control mechanisms. In singing, auditory pitch feedback perturbation paradigms have been used to show that singers engage a network of brain regions including anterior cingulate cortex (ACC), anterior insula (aINS), and intraparietal sulcus (IPS) when compensating for altered pitch feedback, and posterior superior temporal gyrus (pSTG) and supramarginal gyrus (SMG) when ignoring it (Zarate et al. 2005, 2008). To determine whether the brain networks for cello playing and singing directly overlap in these sensory-motor integration areas, in the present study expert cellists were asked to compensate for or ignore introduced pitch perturbations when singing/playing during fMRI scanning. We found that cellists were able to sing/play target tones, and compensate for and ignore introduced feedback perturbations equally well. Brain activity overlapped for singing and playing in IPS and SMG when compensating, and pSTG and dPMC when ignoring; differences between singing/playing across all three conditions were most prominent in M1, centered on the relevant motor effectors (hand, larynx). These findings support the hypothesis that pitch regulation during cello playing relies on structures within the singing network and suggests that differences arise primarily at the level of forward motor control.

Partially Overlapping Brain Networks for Singing and Cello Playing.

This research uses an MR-Compatible cello to compare functional brain activation during singing and cello playing within the same individuals to determine the extent to which arbitrary auditory-motor associations, like those required to play the cello, co-opt functional brain networks that evolved for singing. Musical instrument playing and singing both require highly specific associations between sounds and movements. Because these are both used to produce musical sounds, it is often assumed in the literature that their neural underpinnings are highly similar. However, singing is an evolutionarily old human trait, and the auditory-motor associations used for singing are also used for speech and non-speech vocalizations. This sets it apart from the arbitrary auditory-motor associations required to play musical instruments. The pitch range of the cello is similar to that of the human voice, but cello playing is completely independent of the vocal apparatus, and can therefore be used to dissociate the auditory-vocal network from that of the auditory-motor network. While in the MR-Scanner, 11 expert cellists listened to and subsequently produced individual tones either by singing or cello playing. All participants were able to sing and play the target tones in tune (<50C deviation from target). We found that brain activity during cello playing directly overlaps with brain activity during singing in many areas within the auditory-vocal network. These include primary motor, dorsal pre-motor, and supplementary motor cortices (M1, dPMC, SMA),the primary and periprimary auditory cortices within the superior temporal gyrus (STG) including Heschl's gyrus, anterior insula (aINS), anterior cingulate cortex (ACC), and intraparietal sulcus (IPS), and Cerebellum but, notably, exclude the periaqueductal gray (PAG) and basal ganglia (Putamen). Second, we found that activity within the overlapping areas is positively correlated with, and therefore likely contributing to, both singing and playing in tune determined with performance measures. Third, we found that activity in auditory areas is functionally connected with activity in dorsal motor and pre-motor areas, and that the connectivity between them is positively correlated with good performance on this task. This functional connectivity suggests that the brain areas are working together to contribute to task performance and not just coincidently active. Last, our findings showed that cello playing may directly co-opt vocal areas (including larynx area of motor cortex), especially if musical training begins before age 7.

Neural network retuning and neural predictors of learning success associated with cello training.

The auditory and motor neural systems are closely intertwined, enabling people to carry out tasks such as playing a musical instrument whose mapping between action and sound is extremely sophisticated. While the dorsal auditory stream has been shown to mediate these audio-motor transformations, little is known about how such mapping emerges with training. Here, we use longitudinal training on a cello as a model for brain plasticity during the acquisition of specific complex skills, including continuous and many-to-one audio-motor mapping, and we investigate individual differences in learning. We trained participants with no musical background to play on a specially designed MRI-compatible cello and scanned them before and after 1 and 4 wk of training. Activation of the auditory-to-motor dorsal cortical stream emerged rapidly during the training and was similarly activated during passive listening and cello performance of trained melodies. This network activation was independent of performance accuracy and therefore appears to be a prerequisite of music playing. In contrast, greater recruitment of regions involved in auditory encoding and motor control over the training was related to better musical proficiency. Additionally, pre-supplementary motor area activity and its connectivity with the auditory cortex during passive listening before training was predictive of final training success, revealing the integrative function of this network in auditory-motor information processing. Together, these results clarify the critical role of the dorsal stream and its interaction with auditory areas in complex audio-motor learning.