Inferring control objectives in a virtual balancing task in humans and monkeys.

Natural behaviors have redundancy, which implies that humans and animals can achieve their goals with different strategies. Given only observations of behavior, is it possible to infer the control objective that the subject is employing? This challenge is particularly acute in animal behavior because we cannot ask or instruct the subject to use a particular strategy. This study presents a three-pronged approach to infer an animal's control objective from behavior. First, both humans and monkeys performed a virtual balancing task for which different control strategies could be utilized. Under matched experimental conditions, corresponding behaviors were observed in humans and monkeys. Second, a generative model was developed that represented two main control objectives to achieve the task goal. Model simulations were used to identify aspects of behavior that could distinguish which control objective was being used. Third, these behavioral signatures allowed us to infer the control objective used by human subjects who had been instructed to use one control objective or the other. Based on this validation, we could then infer objectives from animal subjects. Being able to positively identify a subject's control objective from observed behavior can provide a powerful tool to neurophysiologists as they seek the neural mechanisms of sensorimotor coordination.

Motor cortex retains and reorients neural dynamics during motor imagery.

The most prominent characteristic of motor cortex is its activation during movement execution, but it is also active when we simply imagine movements in the absence of actual motor output. Despite decades of behavioural and imaging studies, it is unknown how the specific activity patterns and temporal dynamics in motor cortex during covert motor imagery relate to those during motor execution. Here we recorded intracortical activity from the motor cortex of two people who retain some residual wrist function following incomplete spinal cord injury as they performed both actual and imagined isometric wrist extensions. We found that we could decompose the population activity into three orthogonal subspaces, where one was similarly active during both action and imagery, and the others were active only during a single task type-action or imagery. Although they inhabited orthogonal neural dimensions, the action-unique and imagery-unique subspaces contained a strikingly similar set of dynamic features. Our results suggest that during motor imagery, motor cortex maintains the same overall population dynamics as during execution by reorienting the components related to motor output and/or feedback into a unique, output-null imagery subspace.

Inferring control objectives in a virtual balancing task in humans and monkeys.

Natural behaviors have redundancy, which implies that humans and animals can achieve their goals with different control objectives. Given only observations of behavior, is it possible to infer the control strategy that the subject is employing? This challenge is particularly acute in animal behavior because we cannot ask or instruct the subject to use a particular control strategy. This study presents a threepronged approach to infer an animal's control strategy from behavior. First, both humans and monkeys performed a virtual balancing task for which different control objectives could be utilized. Under matched experimental conditions, corresponding behaviors were observed in humans and monkeys. Second, a generative model was developed that represented two main control strategies to achieve the task goal. Model simulations were used to identify aspects of behavior that could distinguish which control objective was being used. Third, these behavioral signatures allowed us to infer the control objective used by human subjects who had been instructed to use one control objective or the other. Based on this validation, we could then infer strategies from animal subjects. Being able to positively identify a subject's control objective from behavior can provide a powerful tool to neurophysiologists as they seek the neural mechanisms of sensorimotor coordination.

Motor cortex retains and reorients neural dynamics during motor imagery.

The most prominent role of motor cortex is generating patterns of neural activity that lead to movement, but it is also active when we simply imagine movements in the absence of actual motor output. Despite decades of behavioral and imaging studies, it is unknown how the specific activity patterns and temporal dynamics within motor cortex during covert motor imagery relate to those during motor execution. Here we recorded intracortical activity from the motor cortex of two people with residual wrist function following incomplete spinal cord injury as they performed both actual and imagined isometric wrist extensions. We found that we could decompose the population-level activity into orthogonal subspaces such that one set of components was similarly active during both action and imagery, and others were only active during a single task typeâ€"action or imagery. Although they inhabited orthogonal neural dimensions, the action-unique and imagery-unique subspaces contained a strikingly similar set of dynamical features. Our results suggest that during motor imagery, motor cortex maintains the same overall population dynamics as during execution by recreating the missing components related to motor output and/or feedback within a unique imagery-only subspace.

A large-scale neural network training framework for generalized estimation of single-trial population dynamics.

Achieving state-of-the-art performance with deep neural population dynamics models requires extensive hyperparameter tuning for each dataset. AutoLFADS is a model-tuning framework that automatically produces high-performing autoencoding models on data from a variety of brain areas and tasks, without behavioral or task information. We demonstrate its broad applicability on several rhesus macaque datasets: from motor cortex during free-paced reaching, somatosensory cortex during reaching with perturbations, and dorsomedial frontal cortex during a cognitive timing task.

Small, correlated changes in synaptic connectivity may facilitate rapid motor learning.

Animals rapidly adapt their movements to external perturbations, a process paralleled by changes in neural activity in the motor cortex. Experimental studies suggest that these changes originate from altered inputs (Hinput) rather than from changes in local connectivity (Hlocal), as neural covariance is largely preserved during adaptation. Since measuring synaptic changes in vivo remains very challenging, we used a modular recurrent neural network to qualitatively test this interpretation. As expected, Hinput resulted in small activity changes and largely preserved covariance. Surprisingly given the presumed dependence of stable covariance on preserved circuit connectivity, Hlocal led to only slightly larger changes in activity and covariance, still within the range of experimental recordings. This similarity is due to Hlocal only requiring small, correlated connectivity changes for successful adaptation. Simulations of tasks that impose increasingly larger behavioural changes revealed a growing difference between Hinput and Hlocal, which could be exploited when designing future experiments.

Local field potentials reflect cortical population dynamics in a region-specific and frequency-dependent manner.

The spiking activity of populations of cortical neurons is well described by the dynamics of a small number of population-wide covariance patterns, whose activation we refer to as 'latent dynamics'. These latent dynamics are largely driven by the same correlated synaptic currents across the circuit that determine the generation of local field potentials (LFPs). Yet, the relationship between latent dynamics and LFPs remains largely unexplored. Here, we characterised this relationship for three different regions of primate sensorimotor cortex during reaching. The correlation between latent dynamics and LFPs was frequency-dependent and varied across regions. However, for any given region, this relationship remained stable throughout the behaviour: in each of primary motor and premotor cortices, the LFP-latent dynamics correlation profile was remarkably similar between movement planning and execution. These robust associations between LFPs and neural population latent dynamics help bridge the wealth of studies reporting neural correlates of behaviour using either type of recordings.

Cuneate nucleus: The somatosensory gateway to the brain.

Much remains unknown about the transformation of proprioceptive afferent input from the periphery to the cortex. Until recently, the only recordings from neurons in the cuneate nucleus (CN) were from anesthetized animals. We are beginning to learn more about how the sense of proprioception is transformed as it propagates centrally. Recent recordings from microelectrode arrays chronically implanted in CN have revealed that CN neurons with muscle-like properties have a greater sensitivity to active reaching movements than to passive limb displacement, and we find that these neurons have receptive fields that resemble single muscles. In this review, we focus on the varied uses of proprioceptive input and the possible role of CN in processing this information.

Machine Learning for Neural Decoding.

Despite rapid advances in machine learning tools, the majority of neural decoding approaches still use traditional methods. Modern machine learning tools, which are versatile and easy to use, have the potential to significantly improve decoding performance. This tutorial describes how to effectively apply these algorithms for typical decoding problems. We provide descriptions, best practices, and code for applying common machine learning methods, including neural networks and gradient boosting. We also provide detailed comparisons of the performance of various methods at the task of decoding spiking activity in motor cortex, somatosensory cortex, and hippocampus. Modern methods, particularly neural networks and ensembles, significantly outperform traditional approaches, such as Wiener and Kalman filters. Improving the performance of neural decoding algorithms allows neuroscientists to better understand the information contained in a neural population and can help to advance engineering applications such as brain-machine interfaces. Our code package is available at github.com/kordinglab/neural_decoding.

Area 2 of primary somatosensory cortex encodes kinematics of the whole arm.

Proprioception, the sense of body position, movement, and associated forces, remains poorly understood, despite its critical role in movement. Most studies of area 2, a proprioceptive area of somatosensory cortex, have simply compared neurons' activities to the movement of the hand through space. Using motion tracking, we sought to elaborate this relationship by characterizing how area 2 activity relates to whole arm movements. We found that a whole-arm model, unlike classic models, successfully predicted how features of neural activity changed as monkeys reached to targets in two workspaces. However, when we then evaluated this whole-arm model across active and passive movements, we found that many neurons did not consistently represent the whole arm over both conditions. These results suggest that 1) neural activity in area 2 includes representation of the whole arm during reaching and 2) many of these neurons represented limb state differently during active and passive movements.

Long-term stability of cortical population dynamics underlying consistent behavior.

Animals readily execute learned behaviors in a consistent manner over long periods of time, and yet no equally stable neural correlate has been demonstrated. How does the cortex achieve this stable control? Using the sensorimotor system as a model of cortical processing, we investigated the hypothesis that the dynamics of neural latent activity, which captures the dominant co-variation patterns within the neural population, must be preserved across time. We recorded from populations of neurons in premotor, primary motor and somatosensory cortices as monkeys performed a reaching task, for up to 2 years. Intriguingly, despite a steady turnover in the recorded neurons, the low-dimensional latent dynamics remained stable. The stability allowed reliable decoding of behavioral features for the entire timespan, while fixed decoders based directly on the recorded neural activity degraded substantially. We posit that stable latent cortical dynamics within the manifold are the fundamental building blocks underlying consistent behavioral execution.

Neural Networks for Modeling Neural Spiking in S1 Cortex.

Somatosensation is composed of two distinct modalities: touch, arising from sensors in the skin, and proprioception, resulting primarily from sensors in the muscles, combined with these same cutaneous sensors. In contrast to the wealth of information about touch, we know quite less about the nature of the signals giving rise to proprioception at the cortical level. Likewise, while there is considerable interest in developing encoding models of touch-related neurons for application to brain machine interfaces, much less emphasis has been placed on an analogous proprioceptive interface. Here we investigate the use of Artificial Neural Networks (ANNs) to model the relationship between the firing rates of single neurons in area 2, a largely proprioceptive region of somatosensory cortex (S1) and several types of kinematic variables related to arm movement. To gain a better understanding of how these kinematic variables interact to create the proprioceptive responses recorded in our datasets, we train ANNs under different conditions, each involving a different set of input and output variables. We explore the kinematic variables that provide the best network performance, and find that the addition of information about joint angles and/or muscle lengths significantly improves the prediction of neural firing rates. Our results thus provide new insight regarding the complex representations of the limb motion in S1: that the firing rates of neurons in area 2 may be more closely related to the activity of peripheral sensors than it is to extrinsic hand position. In addition, we conduct numerical experiments to determine the sensitivity of ANN models to various choices of training design and hyper-parameters. Our results provide a baseline and new tools for future research that utilizes machine learning to better describe and understand the activity of neurons in S1.

Modern Machine Learning as a Benchmark for Fitting Neural Responses.

Neuroscience has long focused on finding encoding models that effectively ask "what predicts neural spiking?" and generalized linear models (GLMs) are a typical approach. It is often unknown how much of explainable neural activity is captured, or missed, when fitting a model. Here we compared the predictive performance of simple models to three leading machine learning methods: feedforward neural networks, gradient boosted trees (using XGBoost), and stacked ensembles that combine the predictions of several methods. We predicted spike counts in macaque motor (M1) and somatosensory (S1) cortices from standard representations of reaching kinematics, and in rat hippocampal cells from open field location and orientation. Of these methods, XGBoost and the ensemble consistently produced more accurate spike rate predictions and were less sensitive to the preprocessing of features. These methods can thus be applied quickly to detect if feature sets relate to neural activity in a manner not captured by simpler methods. Encoding models built with a machine learning approach accurately predict spike rates and can offer meaningful benchmarks for simpler models.

Methodological considerations for a chronic neural interface with the cuneate nucleus of macaques.

While the response properties of neurons in the somatosensory nerves and anterior parietal cortex have been extensively studied, little is known about the encoding of tactile and proprioceptive information in the cuneate nucleus (CN) or external cuneate nucleus (ECN), the first recipients of upper limb somatosensory afferent signals. The major challenge in characterizing neural coding in CN/ECN has been to record from these tiny, difficult-to-access brain stem structures. Most previous investigations of CN response properties have been carried out in decerebrate or anesthetized animals, thereby eliminating the well-documented top-down signals from cortex, which likely exert a strong influence on CN responses. Seeking to fill this gap in our understanding of somatosensory processing, we describe an approach to chronically implanting arrays of electrodes in the upper limb representation in the brain stem in primates. First, we describe the topography of CN/ECN in rhesus macaques, including its somatotopic organization and the layout of its submodalities (touch and proprioception). Second, we describe the design of electrode arrays and the implantation strategy to obtain stable recordings. Third, we show sample responses of CN/ECN neurons in brain stem obtained from awake, behaving monkeys. With this method, we are in a position to characterize, for the first time, somatosensory representations in CN and ECN of primates.NEW & NOTEWORTHY In primates, the neural basis of touch and of our sense of limb posture and movements has been studied in the peripheral nerves and in somatosensory cortex, but coding in the cuneate and external cuneate nuclei, the first processing stage for these signals in the central nervous system, remains an enigma. We have developed a method to record from these nuclei, thereby paving the way to studying how sensory information from the limb is encoded there.

Musculoskeletal geometry accounts for apparent extrinsic representation of paw position in dorsal spinocerebellar tract.

Proprioception, the sense of limb position and motion, arises from individual muscle receptors. An important question is how and where in the neuroaxis our high level "extrinsic" sense of limb movement originates. In the 1990s, a series of papers detailed the properties of neurons in the dorsal spinocerebellar tract (DSCT) of the cat. Despite their direct projections from sensory receptors, it appeared that half of these neurons had consistent, high-level tuning to paw position rather than to joint angles (or muscle lengths). These results suggested that many DSCT neurons compute paw position from lower level sensory information. We examined the contribution of musculoskeletal geometry to this apparent extrinsic representation by simulating a three-joint hindlimb with mono- and biarticular muscles, each providing a muscle spindlelike signal, modulated by the muscle length. We simulated neurons driven by randomly weighted combinations of these signals and moved the paw to different positions under two joint-covariance conditions similar to the original experiments. Our results paralleled those experiments in a number of respects: 1) Many neurons were tuned to paw position relative to the hip under both conditions. 2) The distribution of tuning was strongly bimodal, with most neurons driven by whole-leg flexion or extension. 3) The change in tuning between conditions clustered around zero (median absolute change ~20°). These results indicate that, at least for these constraint conditions, extrinsic-like representation can be achieved simply through musculoskeletal geometry and convergent muscle length inputs. Consequently, they suggest a reinterpretation of the earlier results may be required.NEW & NOTEWORTHY A classic experiment concluding that many dorsal spinocerebellar tract neurons encode paw position rather than joint angles has been cited by many studies as evidence for high-level computation occurring within a single synapse of the sensors. However, our study provides evidence that such a computation is not required to explain the results. Using simulation, we replicated many of the original results with purely random connectivity, suggesting that a reinterpretation of the classic experiment is needed.

Epidermal electronics.

We report classes of electronic systems that achieve thicknesses, effective elastic moduli, bending stiffnesses, and areal mass densities matched to the epidermis. Unlike traditional wafer-based technologies, laminating such devices onto the skin leads to conformal contact and adequate adhesion based on van der Waals interactions alone, in a manner that is mechanically invisible to the user. We describe systems incorporating electrophysiological, temperature, and strain sensors, as well as transistors, light-emitting diodes, photodetectors, radio frequency inductors, capacitors, oscillators, and rectifying diodes. Solar cells and wireless coils provide options for power supply. We used this type of technology to measure electrical activity produced by the heart, brain, and skeletal muscles and show that the resulting data contain sufficient information for an unusual type of computer game controller.

Detection of granularity in dermoscopy images of malignant melanoma using color and texture features.

Granularity, also called peppering and multiple blue-grey dots, is defined as an accumulation of tiny, blue-grey granules in dermoscopy images. Granularity is most closely associated with a diagnosis of malignant melanoma. This study analyzes areas of granularity with color and texture measures to discriminate granularity in melanoma from similar areas in non-melanoma skin lesions. The granular areas in dermoscopy images of 74 melanomas and 14 melanomas in situ were identified and manually selected. For 200 non-melanoma dermoscopy images, those areas which most closely resembled granularity in color and texture were similarly selected. Ten texture and twenty-two color measures were studied. The texture measures consisted of the average and range of energy, inertia, correlation, inverse difference, and entropy. The color measures consisted of absolute and relative RGB averages, absolute and relative RGB chromaticity averages, absolute and relative G/B averages, CIE X, Y, Z, X/Y, X/Z and Y/Z averages, R variance, and luminance. These measures were calculated for each granular area of the melanomas and the comparable areas in the non-melanoma images. Receiver operating characteristic (ROC) curve analysis showed that the best separation of melanoma images from non-melanoma images by granular area features was obtained with a combination of color and texture measures. Comparison of ROC results showed greater separation of melanoma from benign lesions using relative color than using absolute color. Statistical analysis showed that the four most significant measures of granularity in melanoma are two color measures and two texture measures averaged over the spots: relative blue, relative green, texture correlation, and texture energy range. The best feature set, utilizing texture and relative color measures, achieved an accuracy of 96.4% based on area under the receiver operating characteristic curve.

Detection of basal cell carcinoma using color and histogram measures of semitranslucent areas.

BACKGROUND: Semitranslucency, defined as a smooth, jelly-like area with varied, near-skin-tone color, can indicate a diagnosis of basal cell carcinoma (BCC) with high specificity. This study sought to analyze potential areas of semitranslucency with histogram-derived texture and color measures to discriminate BCC from non-semitranslucent areas in non-BCC skin lesions. METHODS: For 210 dermoscopy images, the areas of semitranslucency in 42 BCCs and comparable areas of smoothness and color in 168 non-BCCs were selected manually. Six color measures and six texture measures were applied to the semitranslucent areas of the BCC and the comparable areas in the non-BCC images. RESULTS: Receiver operating characteristic (ROC) curve analysis showed that the texture measures alone provided greater separation of BCC from non-BCC than the color measures alone. Statistical analysis showed that the four most important measures of semitranslucency are three histogram measures: contrast, smoothness, and entropy, and one color measure: blue chromaticity. Smoothness is the single most important measure. The combined 12 measures achieved a diagnostic accuracy of 95.05% based on area under the ROC curve. CONCLUSION: Texture and color analysis measures, especially smoothness, may afford automatic detection of BCC images with semitranslucency.