High or low expectations: Expected intensity of action outcome is embedded in action kinetics.

Goal-directed actions are performed in order to attain certain sensory consequences in the world. However, expected attributes of these consequences can affect the kinetics of the action. In a set of three studies (n = 120), we examined how expected attributes of stimulus outcome (intensity) shape the kinetics of the triggering action (applied force), even when the action kinetic and attribute are independent. We show that during action execution (button presses), the expected intensity of sensory outcome affects the applied force of the stimulus-producing action in an inverse fashion. Thus, participants applied more force when the expected intensity of the outcome was low (vs. high intensity outcome). In the absence of expectations or when actions were performed in response to the sensory event, no intensity-dependent force modulations were found. Thus, expectations of stimulus intensity and causality play an important role in shaping action kinetics. Finally, we examined the relationship between kinetics and perception and found no influence of applied force level on perceptual detection of low intensity (near-threshold) outcome stimuli, suggesting no causal link between the two. Taken together, our results demonstrate that action kinetics are embedded with high-level context such as the expectation of consequence intensity and the causal relationship with environmental cues.

Temporal hierarchy of observed goal-directed actions.

During social interactions, we continuously integrate current and previous information over varying timescales to infer other people's action intentions. Motor cognition theories argue for a hierarchical organization of goal-directed actions based on temporal scales. Accordingly, transient motor primitives are represented at lower levels of the hierarchy, a combination of primitives building motor sequences at subordinate levels, and more stable overarching action goals at superordinate levels. A neural topography of hierarchal timescales for information accumulation was previously shown in the visual and auditory domains. However, whether such a temporal hierarchy can also account for observed goal-directed action representations in motor pathways remains to be determined. Thus, the current study examined the neural architecture underlying the processing of observed goal-directed actions using inter-subject correlation (ISC) of fMRI activity. Observers (n = 24) viewed sequential hand movements presented in their intact order or piecewise scrambled at three timescales pertaining to goal-directed action evolution (Primitives: ± 1.5 s, Sub-Goals: ± 4 s, and High-Goals: ± 10 s). The results revealed differential intrinsic temporal capacities for integrating goal-directed action information across brain areas engaged in action observation. Longer timescales (> ± 10 s) were found in the posterior parietal and dorsal premotor compared to the ventral premotor (± 4 s) and anterior parietal (± 1.5 s) cortex. Moreover, our results revealed a hemispheric bias with more extended timescales in the right MT+, primary somatosensory, and early visual cortices compared to their homotopic regions in the left hemisphere. Our findings corroborate a hierarchical neural mapping of observed actions based on temporal scales of goals and provide further support for a ubiquitous time-dependent neural organization of information processing across multiple modalities.

The power of multivariate approach in identifying EEG correlates of interlimb coupling.

Interlimb coupling refers to the interaction between movements of one limb and movements of other limbs. Understanding mechanisms underlying this effect is important to real life because it reflects the level of interdependence between the limbs that plays a role in daily activities including tool use, cooking, or playing musical instruments. Interlimb coupling involves multiple brain regions working together, including coordination of neural activity in sensory and motor regions across the two hemispheres. Traditional neuroscience research took a univariate approach to identify neural features that correspond to behavioural coupling measures. Yet, this approach reduces the complexity of the neural activity during interlimb tasks to one value. In this brief research report, we argue that identifying neural correlates of interlimb coupling would benefit from a multivariate approach in which full patterns from multiple sources are used to predict behavioural coupling. We demonstrate the feasibility of this approach in an exploratory EEG study where participants (n = 10) completed 240 trials of a well-established drawing paradigm that involves interlimb coupling. Using artificial neural network (ANN), we show that multivariate representation of the EEG signal significantly captures the interlimb coupling during bimanual drawing whereas univariate analyses failed to identify such correlates. Our findings demonstrate that analysing distributed patterns of multiple EEG channels is more sensitive than single-value techniques in uncovering subtle differences between multiple neural signals. Using such techniques can improve identification of neural correlates of complex motor behaviours.

Playing with your ears: Audio-motor skill learning is sensitive to the lateral relationship between trained hand and ear.

A salient feature of motor and sensory circuits in the brain is their contralateral hemispheric bias-a feature that might play a role in integration and learning of sensorimotor skills. In the current behavioral study, we examined whether the lateral configuration between sound-producing hand and feedback-receiving ear affects performance and learning of an audio-motor skill. Right-handed participants (n = 117) trained to play a piano sequence using their right or left hand while auditory feedback was presented monaurally, either to the right or left ear. Participants receiving auditory feedback to the contralateral ear during training performed better than participants receiving ipsilateral feedback (with respect to the training hand). Furthermore, in the Left-Hand training groups, the contralateral training advantage persisted in a generalization task. Our results demonstrate that audio-motor learning is sensitive to the lateral configuration between motor and sensory circuits and suggest that integration of neural activity across hemispheres facilitates such learning.

Same action, different meaning: neural substrates of action semantic meaning.

Voluntary actions are shaped by desired goals and internal intentions. Multiple factors, including the planning of subsequent actions and the expectation of sensory outcome, were shown to modulate kinetics and neural activity patterns associated with similar goal-directed actions. Notably, in many real-world tasks, actions can also vary across the semantic meaning they convey, although little is known about how semantic meaning modulates associated neurobehavioral measures. Here, we examined how behavioral and functional magnetic resonance imaging measures are modulated when subjects execute similar actions (button presses) for two different semantic meanings-to answer "yes" or "no" to a binary question. Our findings reveal that, when subjects answer using their right hand, the two semantic meanings are differentiated based on voxel patterns in the frontoparietal cortex and lateral-occipital complex bilaterally. When using their left hand, similar regions were found, albeit only with a more liberal threshold. Although subjects were faster to answer "yes" versus "no" when using their right hand, the neural differences cannot be explained by these kinetic differences. To the best of our knowledge, this is the first evidence showing that semantic meaning is embedded in the neural representation of actions, independent of alternative modulating factors such as kinetic and sensory features.

Motor learning in hemi-Parkinson using VR-manipulated sensory feedback.

AIMS: Modalities for rehabilitation of the neurologically affected upper-limb (UL) are generally of limited benefit. The majority of patients seriously affected by UL paresis remain with severe motor disability, despite all rehabilitation efforts. Consequently, extensive clinical research is dedicated to develop novel strategies aimed to improve the functional outcome of the affected UL. We have developed a novel virtual-reality training tool that exploits the voluntary control of one hand and provides real-time movement-based manipulated sensory feedback as if the other hand is the one that moves. The aim of this study was to expand our previous results, obtained in healthy subjects, to examine the utility of this training setup in the context of neuro-rehabilitation. METHODS: We tested the training setup in patient LA, a young man with significant unilateral UL dysfunction stemming from hemi-parkinsonism. LA underwent daily intervention in which he intensively trained the non-affected upper limb, while receiving online sensory feedback that created an illusory perception of control over the affected limb. Neural changes were assessed using functional magnetic resonance imaging (fMRI) scans before and after training. RESULTS: Training-induced behavioral gains were accompanied by enhanced activation in the pre-frontal cortex and a widespread increase in resting-state functional connectivity. DISCUSSION: Our combination of cutting edge technologies, insights gained from basic motor neuroscience in healthy subjects and well-known clinical treatments, hold promise for the pursuit of finding novel and more efficient rehabilitation schemes for patients suffering from hemiplegia.Implications for rehabilitationAssistive devices used in hospitals to support patients with hemiparesis require expensive equipment and trained personnel - constraining the amount of training that a given patient can receive. The setup we describe is simple and can be easily used at home with the assistance of an untrained caregiver/family member. Once installed at the patient's home, the setup is lightweight, mobile, and can be used with minimal maintenance . Building on advances in machine learning, our software can be adapted to personal use at homes. Our findings can be translated into practice with relatively few adjustments, and our experimental design may be used as an important adjuvant to standard clinical care for upper limb hemiparesis.

Real-time processes in the development of action planning.

Across species and ages, planning multi-step actions is a hallmark of intelligence and critical for survival. Traditionally, researchers adopt a "top-down" approach to action planning by focusing on the ability to create an internal representation of the world that guides the next step in a multi-step action. However, a top-down approach does not inform on underlying mechanisms, so researchers can only speculate about how and why improvements in planning occur. The current study takes a "bottom-up" approach by testing developmental changes in the real-time, moment-to-moment interplay among perceptual, neural, and motor components of action planning using simultaneous video, motion-tracking, head-mounted eye tracking, and electroencephalography (EEG). Preschoolers (n = 32) and adults (n = 22) grasped a hammer with their dominant hand to pound a peg when the hammer handle pointed in different directions. When the handle pointed toward their non-dominant hand, younger children ("nonadaptive planners") used a habitual overhand grip that interfered with wielding the hammer, whereas adults and older children ("adaptive planners") used an adaptive underhand grip. Adaptive and nonadaptive children differed in when and where they directed their gaze to obtain visual information, neural activation of the motor system before reaching, and straightness of their reach trajectories. Nonadaptive children immediately used a habitual overhand grip before gathering visual information, leaving insufficient time to form a plan before acting. Our novel bottom-up approach transcends mere speculation by providing converging evidence that the development of action planning depends on a real-time "tug of war" between habits and information gathering and processing.

Children do not distinguish efficient from inefficient actions during observation.

Observation is a powerful way to learn efficient actions from others. However, the role of observers' motor skill in assessing efficiency of others is unknown. Preschoolers are notoriously poor at performing multi-step actions like grasping the handle of a tool. Preschoolers (N = 22) and adults (N = 22) watched video-recorded actors perform efficient and inefficient tool use. Eye tracking showed that preschoolers and adults looked equally long at the videos, but adults looked longer than children at how actors grasped the tool. Deep learning analyses of participants' eye gaze distinguished efficient from inefficient grasps for adults, but not for children. Moreover, only adults showed differential action-related pupil dilation and neural activity (suppressed oscillation power in the mu frequency) while observing efficient vs. inefficient grasps. Thus, children observe multi-step actions without "seeing" whether the initial step is efficient. Findings suggest that observer's own motor efficiency determines whether they can perceive action efficiency in others.

Action-locked Neural Responses in Auditory Cortex to Self-generated Sounds.

Sensory perception is a product of interactions between the internal state of an organism and the physical attributes of a stimulus. It has been shown across the animal kingdom that perception and sensory-evoked physiological responses are modulated depending on whether or not the stimulus is the consequence of voluntary actions. These phenomena are often attributed to motor signals sent to relevant sensory regions that convey information about upcoming sensory consequences. However, the neurophysiological signature of action-locked modulations in sensory cortex, and their relationship with perception, is still unclear. In the current study, we recorded neurophysiological (using Magnetoencephalography) and behavioral responses from 16 healthy subjects performing an auditory detection task of faint tones. Tones were either generated by subjects' voluntary button presses or occurred predictably following a visual cue. By introducing a constant temporal delay between button press/cue and tone delivery, and applying source-level analysis, we decoupled action-locked and auditory-locked activity in auditory cortex. We show action-locked evoked-responses in auditory cortex following sound-triggering actions and preceding sound onset. Such evoked-responses were not found for button-presses that were not coupled with sounds, or sounds delivered following a predictive visual cue. Our results provide evidence for efferent signals in human auditory cortex that are locked to voluntary actions coupled with future auditory consequences.

Better-than-chance classification for signal detection.

The estimated accuracy of a classifier is a random quantity with variability. A common practice in supervised machine learning, is thus to test if the estimated accuracy is significantly better than chance level. This method of signal detection is particularly popular in neuroimaging and genetics. We provide evidence that using a classifier's accuracy as a test statistic can be an underpowered strategy for finding differences between populations, compared to a bona fide statistical test. It is also computationally more demanding than a statistical test. Via simulation, we compare test statistics that are based on classification accuracy, to others based on multivariate test statistics. We find that the probability of detecting differences between two distributions is lower for accuracy-based statistics. We examine several candidate causes for the low power of accuracy-tests. These causes include: the discrete nature of the accuracy-test statistic, the type of signal accuracy-tests are designed to detect, their inefficient use of the data, and their suboptimal regularization. When the purpose of the analysis is the evaluation of a particular classifier, not signal detection, we suggest several improvements to increase power. In particular, to replace V-fold cross-validation with the Leave-One-Out Bootstrap.

Single-cell activity in human STG during perception of phonemes is organized according to manner of articulation.

One of the central tasks of the human auditory system is to extract sound features from incoming acoustic signals that are most critical for speech perception. Specifically, phonological features and phonemes are the building blocks for more complex linguistic entities, such as syllables, words and sentences. Previous ECoG and EEG studies showed that various regions in the superior temporal gyrus (STG) exhibit selective responses to specific phonological features. However, electrical activity recorded by ECoG or EEG grids reflects average responses of large neuronal populations and is therefore limited in providing insights into activity patterns of single neurons. Here, we recorded spiking activity from 45 units in the STG from six neurosurgical patients who performed a listening task with phoneme stimuli. Fourteen units showed significant responsiveness to the stimuli. Using a Naïve-Bayes model, we find that single-cell responses to phonemes are governed by manner-of-articulation features and are organized according to sonority with two main clusters for sonorants and obstruents. We further find that 'neural similarity' (i.e. the similarity of evoked spiking activity between pairs of phonemes) is comparable to the 'perceptual similarity' (i.e. to what extent two phonemes are judged as sounding similar) based on perceptual confusion, assessed behaviorally in healthy subjects. Thus, phonemes that were perceptually similar also had similar neural responses. Taken together, our findings indicate that manner-of-articulation is the dominant organization dimension of phoneme representations at the single-cell level, suggesting a remarkable consistency across levels of analyses, from the single neuron level to that of large neuronal populations and behavior.

Voluntary Actions Modulate Perception and Neural Representation of Action-Consequences in a Hand-Dependent Manner.

Evoked neural activity in sensory regions and perception of sensory stimuli are modulated when the stimuli are the consequence of voluntary movement, as opposed to an external source. It has been suggested that such modulations are due to motor commands that are sent to relevant sensory regions during voluntary movement. However, given the anatomical-functional laterality bias of the motor system, it is plausible that the pattern of such behavioral and neural modulations will also exhibit a similar bias, depending on the effector triggering the stimulus (e.g., right/left hand). Here, we examined this issue in the visual domain using behavioral and neural measures (fMRI). Healthy participants judged the relative brightness of identical visual stimuli that were either self-triggered (using right/left hand button presses), or triggered by the computer. Stimuli were presented either in the right or left visual field. Despite identical physical properties of the visual consequences, we found stronger perceptual modulations when the triggering hand was ipsi- (rather than contra-) lateral to the stimulated visual field. Additionally, fMRI responses in visual cortices differentiated between stimuli triggered by right/left hand. Our findings support a model in which voluntary actions induce sensory modulations that follow the anatomical-functional bias of the motor system.

Motor output, neural states and auditory perception.

Behavior is a complex product of interactions between sensory influx arising from the environment and the neural state of the organism. Therefore, identical sensory input can elicit different behavioral responses. Research in recent years has demonstrated that perception is modulated when an organism is engaged in active behavior - suggesting that neural activity in motor pathways is one factor governing the neural state of networks engaged in sensory processing. In the current manuscript, we focus on the auditory modality and propose a mechanism by which activity in motor cortex changes the neural state in auditory cortex through global inhibition. In turn, such global inhibition reduces auditory net population activity, sharpens auditory frequency tuning curves, shifts the auditory oscillatory state and increases the signal-to-noise ratio of auditory evoked neural activity. These changes can result in either attenuated or enhanced behavioral responses depending on the environmental context. We base our model on animal and human literature and suggest that these motor-induced shifts in sensory states may explain reported phenomena and apparent discrepancies in the literature of motor-sensory interactions, such as sensory attenuation or sensory enhancement.

A novel tool for time-locking study plans to results.

Often researchers wish to mark an objective line between study plans that were specified before data acquisition and decisions that were made following data exploration. Contrary to common perception, registering study plans to an online platform prior to data collection does not by itself provide such an objective distinction, even when the registration is time-stamped. Here, we adapt a method from the field of cryptography to allow encoding of study plans and predictions within random aspects of the data acquisition process. Doing so introduces a causal link between the preregistered content and objective attributes of the acquired data, such as the timing and location of brain activations. This guarantees that the preregistered plans and predictions are indeed specified prior to data collection. Our time-locking system does not depend on any external party and can be performed entirely in-lab. We provide code for easy implementation and a detailed example from the field of functional Magnetic Resonance Imaging (fMRI).

Predicted sensory consequences of voluntary actions modulate amplitude of preceding readiness potentials.

Self-generated, voluntary actions, are preceded by a slow negativity in the scalp electroencephalography (EEG) signal recorded from frontal regions (termed 'readiness potential'; RP). This signal, and its lateralized subcomponent (LRP), is mainly regarded as preparatory motor activity associated with the forthcoming voluntary motor act. However, it is not clear whether this neural signature is associated with preparatory motor activity, expectation of its associated sensory consequences, or both. Here we recorded EEG data from 14 healthy subjects while they performed self-paced button presses with their right index and middle fingers. Button-presses with one finger triggered a sound (motor+sound condition), while button-presses with the other finger did not (motor-only condition). Additionally, subjects listened to externally-generated sounds delivered in expected timings (sound-only condition). We found that the RP amplitude (locked to time of button press) was significantly more negative in the motor+sound compared with motor-only conditions. Importantly, no signal negativity was observed prior to expected sound delivery in the sound-only condition. Thus, the differences in RP amplitude between motor+sound and motor-only conditions are beyond differences in mere expectation of a forthcoming auditory sound. Our results suggest that information regarding expected auditory consequences is represented in the RP preceding voluntary action execution.

Suppression of EEG mu rhythm during action observation corresponds with subsequent changes in behavior.

Movement is intrinsically linked to perception such that observing an action induces in the observer behavioral changes during execution of similar actions. Electroencephalogram (EEG) studies have revealed that at the group level, action observation suppresses oscillatory power in mu (8-12 Hz) and beta (15-25 Hz) bands over the sensorimotor cortex - a phenomenon associated with increased excitability of cortical neurons. However, it is unclear whether differences in suppression level across individuals is linked with individual differences in subsequent behavioral changes. Here 32 subjects performed self-paced finger tapping with their right hand before and after observation of a video displaying finger-tapping at either 2 or 4 Hz. Behaviorally, subjects' rate of self-pace tapping increased following observation, with higher increases following 4 Hz observation. The level of EEG power suppression in the low frequency range (low mu; 8-10 Hz) during observation corresponded to subsequent behavioral changes in tapping rate across individuals. Our results demonstrate that observing actions implicitly shifts subsequent execution rates, and that individual differences in the level of this implicit shift can be explained by activity in the sensorimotor cortex during observation.

Perception as a Route for Motor Skill Learning: Perspectives from Neuroscience.

Learning a motor skill requires physical practice that engages neural networks involved in movement. These networks have also been found to be engaged during perception of sensory signals associated with actions. Nonetheless, despite extensive evidence for the existence of such sensory-evoked neural activity in motor pathways, much less is known about their contribution to learning and actual changes in behavior. Primate studies usually involve an overlearned task while studies in humans have largely focused on characterizing activity of the action observation network (AON) in the context of action understanding, theory of mind, and social interactions. Relatively few studies examined neural plasticity induced by perception and its role in transfer of motor knowledge. Here, we review this body of literature and point to future directions for the development of alternative, physiologically grounded ways in which sensory signals could be harnessed to improve motor skills.

Behavioral and neural effects of congruency of visual feedback during short-term motor learning.

Visual feedback can facilitate or interfere with movement execution. Here, we describe behavioral and neural mechanisms by which the congruency of visual feedback during physical practice of a motor skill modulates subsequent performance gains. 18 healthy subjects learned to execute rapid sequences of right hand finger movements during fMRI scans either with or without visual feedback. Feedback consisted of a real-time, movement-based display of virtual hands that was either congruent (right virtual hand movement), or incongruent (left virtual hand movement yoked to the executing right hand). At the group level, right hand performance gains following training with congruent visual feedback were significantly higher relative to training without visual feedback. Conversely, performance gains following training with incongruent visual feedback were significantly lower. Interestingly, across individual subjects these opposite effects correlated. Activation in the Supplementary Motor Area (SMA) during training corresponded to individual differences in subsequent performance gains. Furthermore, functional coupling of SMA with visual cortices predicted individual differences in behavior. Our results demonstrate that some individuals are more sensitive than others to congruency of visual feedback during short-term motor learning and that neural activation in SMA correlates with such inter-individual differences.

Using Virtual Reality to Transfer Motor Skill Knowledge from One Hand to Another.

As far as acquiring motor skills is concerned, training by voluntary physical movement is superior to all other forms of training (e.g. training by observation or passive movement of trainee's hands by a robotic device). This obviously presents a major challenge in the rehabilitation of a paretic limb since voluntary control of physical movement is limited. Here, we describe a novel training scheme we have developed that has the potential to circumvent this major challenge. We exploited the voluntary control of one hand and provided real-time movement-based manipulated sensory feedback as if the other hand is moving. Visual manipulation through virtual reality (VR) was combined with a device that yokes left-hand fingers to passively follow right-hand voluntary finger movements. In healthy subjects, we demonstrate enhanced within-session performance gains of a limb in the absence of voluntary physical training. Results in healthy subjects suggest that training with the unique VR setup might also be beneficial for patients with upper limb hemiparesis by exploiting the voluntary control of their healthy hand to improve rehabilitation of their affected hand.

Sensitivity to perception level differentiates two subnetworks within the mirror neuron system.

Mirror neurons are a subset of brain cells that discharge during action execution and passive observation of similar actions. An open question concerns the functional role of their ability to match observed and executed actions. Since understanding of goals requires conscious perception of actions, we expect that mirror neurons potentially involved in action goal coding, will be modulated by changes in action perception level. Here, we manipulated perception level of action videos depicting short hand movements and measured the corresponding fMRI BOLD responses in mirror regions. Our results show that activity levels within a network of regions, including the sensorimotor cortex, primary motor cortex, dorsal premotor cortex and posterior superior temporal sulcus, are sensitive to changes in action perception level, whereas activity levels in the inferior frontal gyrus, ventral premotor cortex, supplementary motor area and superior parietal lobule are invariant to such changes. In addition, this parcellation to two sub-networks manifest as smaller functional distances within each group of regions during task and resting state. Our results point to functional differences between regions within the mirror neurons system which may have implications with respect to their possible role in action understanding.