Reinforcement Learning May Demystify the Limited Human Motor Learning Efficacy Due to Visual-Proprioceptive Mismatch.

Vision and proprioception have fundamental sensory mismatches in delivering locational information, and such mismatches are critical factors limiting the efficacy of motor learning. However, it is still not clear how and to what extent this mismatch limits motor learning outcomes. To further the understanding of the effect of sensory mismatch on motor learning outcomes, a reinforcement learning algorithm and the simplified biomechanical elbow joint model were employed to mimic the motor learning process in a computational environment. By applying a reinforcement learning algorithm to the motor learning of elbow joint flexion task, simulation results successfully explained how visual-proprioceptive mismatch limits motor learning outcomes in terms of motor control accuracy and task completion speed. The larger the perceived angular offset between the two sensory modalities, the lower the motor control accuracy. Also, the more similar the peak reward amplitude of the two sensory modalities, the lower the motor control accuracy. In addition, simulation results suggest that insufficient exploration rate limits task completion speed, and excessive exploration rate limits motor control accuracy. Such a speed-accuracy trade-off shows that a moderate exploration rate could serve as another important factor in motor learning.

Proportional sway-based electrotactile feedback improves lateral standing balance.

Introduction: Plantar cutaneous augmentation is a promising approach in balance rehabilitation by enhancing motion-dependent sensory feedback. The effect of plantar cutaneous augmentation on balance has been mainly investigated in its passive form (e.g., textured insole) or on lower-limb amputees. In this study, we tested the effect of plantar cutaneous augmentation on balance in its active form (i.e., electrical stimulation) for individuals with intact limbs. Methods: Ten healthy subjects participated in the study and were instructed to maintain their balance as long as possible on the balance board, with or without electrotactile feedback evoked on the medial side of the heel, synched with the lateral board sway. Electrotactile feedback was given in two different modes: 1) Discrete-mode E-stim as the stimulation on/off by a predefined threshold of lateral board sway and 2) Proportional-mode E-stim as the stimulation frequency proportional to the amount of lateral board sway. All subjects were distracted from the balancing task by the n-back counting task, to test subjects' balancing capability with minimal cognitive involvement. Results: Proportional-mode E-stim, along with the n-back counting task, increased the balance time from 1.86 ± 0.03 s to 1.98 ± 0.04 s (p = 0.010). However, discrete-mode E-stim did not change the balance time (p = 0.669). Proportional-mode E-stim also increased the time duration per each swayed state (p = 0.035) while discrete-mode E-stim did not (p = 0.053). Discussion: These results suggest that proportional-mode E-stim is more effective than discrete-mode E-stim on improving standing balance. It is perhaps because the proportional electrotactile feedback better mimics the natural tactile sensation of foot pressure than its discrete counterpart.

AdjointBackMapV2: Precise reconstruction of arbitrary CNN unit's activation via adjoint operators.

Adjoint operators have been found to be effective in the exploration of CNN's inner workings (Wan and Choe, 2022). However, the previous no-bias assumption restricted its generalization. We overcome the restriction via embedding input images into an extended normed space that includes bias in all CNN layers as part of the extended space and propose an adjoint-operator-based algorithm that maps high-level weights back to the extended input space for reconstructing an effective hypersurface. Such hypersurface can be computed for an arbitrary unit in the CNN, and we prove that this reconstructed hypersurface, when multiplied by the original input (through an inner product), will precisely replicate the output value of each unit. We show experimental results based on the CIFAR-10 and CIFAR-100 data sets where the proposed approach achieves near 0 activation value reconstruction error.

AdjointBackMap: Reconstructing effective decision hypersurfaces from CNN layers using adjoint operators.

There are several methods in the exploration of Convolutional Neural Networks' (CNNs') inner workings. However, in general, finding the inverse of the function performed by CNNs as a whole is an ill-posed problem. In this paper, we propose a method based on adjoint operators to reconstruct, given an arbitrary unit in the CNN (except for the first convolutional layer), its effective hypersurface in the input space. Since the reconstructed hyperplane (each point on the hypersurface) resides in the input space, we can easily visualize it. Our results show that the reconstructed hyperplane, when multiplied by the original input image, would give nearly the exact output value of that unit. We find that the CNN unit's decision process is largely conditioned on the input, and the corresponding reconstructed hypersurfaces are highly sensitive to adversarial noise, thus providing insights on why CNNs are susceptible to adversarial attack.

A Queryable Graph Representation of Vascular Connectivity in the Whole Mouse Brain.

We construct a graph representation for the topology and geometry of the vasculature presenting across the whole mouse brain (dataset: Knife-Edge Scanning Microscope Brain Atlas India Ink). We use our graph representation to calculate preliminary estimates of the average radius as 4:8 µm, total vascular volume as 1:1000 mm3, total vascular surface area as 6:5511 cm2, and total vascular length of 2866:6567 cm. We then isolate a posterior cerebral region, derive its graph representation, and then import that representation to a Neo4j graph database. We then detail how researchers can query this database online to isolate specific vascular networks for further analysis and reconstruction.

Data-Driven Synthetic Cerebrovascular Models For Validation Of Segmentation Algorithms.

We introduce a novel method to generate biologically grounded synthetic cerebrovasculature models in a datadriven fashion. First, the centerlines of vascular filaments embedded in an acquired imaging volume are obtained by a segmentation algorithm. That imaging volume is reconstructed from a graph encoding of the centerline (i.e., generating the model's ground truth) and the segmentation algorithm is applied to the resultant volume. As the location and characteristics of the vasculature embedded in this volume are known,theaccuracyofthesegmentationalgorithmcanbeassessed. Moreover, because the synthetic volume was reconstructed directly from biological data, an assessment is made on embedded filaments that are representative of the topologicalandgeometricalcharacteristicsofthedataset. Webelieve thatsuchmodels will provide the means necessary for the enhanced evaluation of vascular segmentation algorithms.

Tracing and analysis of the whole mouse brain vasculature with systematic cleaning to remove and consolidate erroneous images.

Whole mouse brain microvascular images at submicrometer scale can be obtained by Knife-Edge Scanning Microscopy (KESM). However, due to the large size of the image dataset and the noise from the serial sectioning process of the KESM, whole mouse brain vascular reconstruction and analysis with submicrometer resolution have not been achieved yet, while several previous studies demonstrated manually selected small noise-free portion of the KESM dataset. In addition to the KESM dataset, there have been studies for vessel reconstruction and analysis of the whole mouse brain at lower resolution or of partial brain regions at submicrometer resolution. However, to the best of our knowledge, there has been no study for vessel reconstruction and analysis of the whole mouse brain at submicrometer resolution. In this paper, we propose a framework for the 3D reconstruction and analysis of the whole KESM mouse brain vasculature dataset with rich vasculature information extracted at submicrometer resolution. The framework consists of two methods. The propose methods provide the systematic cleaning to remove and consolidate erroneous images automatically, which enables the full tracing and analysis of the whole KESM mouse brain dataset with richer vasculature information.

Towards An Open-Source Framework For The Analysis Of Cerebrovasculature Structure.

The use of graphs to analyze cerebrovascular networks is quite common in studies of the microcirculation. While we have learned a lot from studies utilizing graphs as a tool for the analysis of microvessels, most methodologies for these procedures have only been described in brief and most are not publicly accessible. In this work, we introduce the foundation for an anticipated open-source framework that we hope will streamline the analysis of cerebrovascular structure. We believe that a standardized and accessible framework for the analysis vascular filaments is not only needed, but is necessary, for studies charting the microcirculation on image volumes spanning several grains of tissue. We set forth the foundations for a comprehensive and complete framework in our current work.

A Digital Liquid State Machine With Biologically Inspired Learning and Its Application to Speech Recognition.

This paper presents a bioinspired digital liquid-state machine (LSM) for low-power very-large-scale-integration (VLSI)-based machine learning applications. To the best of the authors' knowledge, this is the first work that employs a bioinspired spike-based learning algorithm for the LSM. With the proposed online learning, the LSM extracts information from input patterns on the fly without needing intermediate data storage as required in offline learning methods such as ridge regression. The proposed learning rule is local such that each synaptic weight update is based only upon the firing activities of the corresponding presynaptic and postsynaptic neurons without incurring global communications across the neural network. Compared with the backpropagation-based learning, the locality of computation in the proposed approach lends itself to efficient parallel VLSI implementation. We use subsets of the TI46 speech corpus to benchmark the bioinspired digital LSM. To reduce the complexity of the spiking neural network model without performance degradation for speech recognition, we study the impacts of synaptic models on the fading memory of the reservoir and hence the network performance. Moreover, we examine the tradeoffs between synaptic weight resolution, reservoir size, and recognition performance and present techniques to further reduce the overhead of hardware implementation. Our simulation results show that in terms of isolated word recognition evaluated using the TI46 speech corpus, the proposed digital LSM rivals the state-of-the-art hidden Markov-model-based recognizer Sphinx-4 and outperforms all other reported recognizers including the ones that are based upon the LSM or neural networks.

Predictable internal brain dynamics in EEG and its relation to conscious states.

Consciousness is a complex and multi-faceted phenomenon defying scientific explanation. Part of the reason why this is the case is due to its subjective nature. In our previous computational experiments, to avoid such a subjective trap, we took a strategy to investigate objective necessary conditions of consciousness. Our basic hypothesis was that predictive internal dynamics serves as such a condition. This is in line with theories of consciousness that treat retention (memory), protention (anticipation), and primary impression as the tripartite temporal structure of consciousness. To test our hypothesis, we analyzed publicly available sleep and awake electroencephalogram (EEG) data. Our results show that EEG signals from awake or rapid eye movement (REM) sleep states have more predictable dynamics compared to those from slow-wave sleep (SWS). Since awakeness and REM sleep are associated with conscious states and SWS with unconscious or less consciousness states, these results support our hypothesis. The results suggest an intricate relationship among prediction, consciousness, and time, with potential applications to time perception and neurorobotics.

A microchip for quantitative analysis of CNS axon growth under localized biomolecular treatments.

Growth capability of neurons is an essential factor in axon regeneration. To better understand how microenvironments influence axon growth, methods that allow spatial control of cellular microenvironments and easy quantification of axon growth are critically needed. Here, we present a microchip capable of physically guiding the growth directions of axons while providing physical and fluidic isolation from neuronal somata/dendrites that enables localized biomolecular treatments and linear axon growth. The microchip allows axons to grow in straight lines inside the axon compartments even after the isolation; therefore, significantly facilitating the axon length quantification process. We further developed an image processing algorithm that automatically quantifies axon growth. The effect of localized extracellular matrix components and brain-derived neurotropic factor treatments on axon growth was investigated. Results show that biomolecules may have substantially different effects on axon growth depending on where they act. For example, while chondroitin sulfate proteoglycan causes axon retraction when added to the axons, it promotes axon growth when applied to the somata. The newly developed microchip overcomes limitations of conventional axon growth research methods that lack localized control of biomolecular environments and are often performed at a significantly lower cell density for only a short period of time due to difficulty in monitoring of axonal growth. This microchip may serve as a powerful tool for investigating factors that promote axon growth and regeneration.

Parameter learning for alpha integration.

In pattern recognition, data integration is an important issue, and when properly done, it can lead to improved performance. Also, data integration can be used to help model and understand multimodal processing in the brain. Amari proposed α-integration as a principled way of blending multiple positive measures (e.g., stochastic models in the form of probability distributions), enabling an optimal integration in the sense of minimizing the α-divergence. It also encompasses existing integration methods as its special case, for example, a weighted average and an exponential mixture. The parameter α determines integration characteristics, and the weight vector w assigns the degree of importance to each measure. In most work, however, α and w are given in advance rather than learned. In this letter, we present a parameter learning algorithm for learning α and ω from data when multiple integrated target values are available. Numerical experiments on synthetic as well as real-world data demonstrate the effectiveness of the proposed method.

Fast macro-scale transmission imaging of microvascular networks using KESM.

Accurate microvascular morphometric information has significant implications in several fields, including the quantification of angiogenesis in cancer research, understanding the immune response for neural prosthetics, and predicting the nature of blood flow as it relates to stroke. We report imaging of the whole mouse brain microvascular system at resolutions sufficient to perform accurate morphometry. Imaging was performed using Knife-Edge Scanning Microscopy (KESM) and is the first example of this technique that can be directly applied to clinical research. We are able to achieve ≈ 0.7µm resolution laterally with 1µm depth resolution using serial sectioning. No alignment was necessary and contrast was sufficient to allow segmentation and measurement of vessels.

Specimen preparation, imaging, and analysis protocols for knife-edge scanning microscopy.

Major advances in high-throughput, high-resolution, 3D microscopy techniques have enabled the acquisition of large volumes of neuroanatomical data at submicrometer resolution. One of the first such instruments producing whole-brain-scale data is the Knife-Edge Scanning Microscope (KESM), developed and hosted in the authors' lab. KESM has been used to section and image whole mouse brains at submicrometer resolution, revealing the intricate details of the neuronal networks (Golgi), vascular networks (India ink), and cell body distribution (Nissl). The use of KESM is not restricted to the mouse nor the brain. We have successfully imaged the octopus brain, mouse lung, and rat brain. We are currently working on whole zebra fish embryos. Data like these can greatly contribute to connectomics research; to microcirculation and hemodynamic research; and to stereology research by providing an exact ground-truth. In this article, we will describe the pipeline, including specimen preparation (fixing, staining, and embedding), KESM configuration and setup, sectioning and imaging with the KESM, image processing, data preparation, and data visualization and analysis. The emphasis will be on specimen preparation and visualization/analysis of obtained KESM data. We expect the detailed protocol presented in this article to help broaden the access to KESM and increase its utilization.

Multiscale exploration of mouse brain microstructures using the knife-edge scanning microscope brain atlas.

Connectomics is the study of the full connection matrix of the brain. Recent advances in high-throughput, high-resolution 3D microscopy methods have enabled the imaging of whole small animal brains at a sub-micrometer resolution, potentially opening the road to full-blown connectomics research. One of the first such instruments to achieve whole-brain-scale imaging at sub-micrometer resolution is the Knife-Edge Scanning Microscope (KESM). KESM whole-brain data sets now include Golgi (neuronal circuits), Nissl (soma distribution), and India ink (vascular networks). KESM data can contribute greatly to connectomics research, since they fill the gap between lower resolution, large volume imaging methods (such as diffusion MRI) and higher resolution, small volume methods (e.g., serial sectioning electron microscopy). Furthermore, KESM data are by their nature multiscale, ranging from the subcellular to the whole organ scale. Due to this, visualization alone is a huge challenge, before we even start worrying about quantitative connectivity analysis. To solve this issue, we developed a web-based neuroinformatics framework for efficient visualization and analysis of the multiscale KESM data sets. In this paper, we will first provide an overview of KESM, then discuss in detail the KESM data sets and the web-based neuroinformatics framework, which is called the KESM brain atlas (KESMBA). Finally, we will discuss the relevance of the KESMBA to connectomics research, and identify challenges and future directions.

Facilitating neural dynamics for delay compensation: a road to predictive neural dynamics?

Goal-directed behavior is a hallmark of cognition. An important prerequisite to goal-directed behavior is that of prediction. In order to establish a goal and devise a plan, one needs to see into the future and predict possible future events. Our earlier work has suggested that compensation mechanisms for neuronal transmission delay may have led to a preliminary form of prediction. In that work, facilitating neuronal dynamics was found to be effective in overcoming delay (the Facilitating Activation Network model, or FAN). The extrapolative property of the delay compensation mechanism can be considered as prediction for incoming signals (predicting the present based on the past). The previous FAN model turns out to have a limitation especially when longer delay needs to be compensated, which requires higher facilitation rates than FAN's normal range. We derived an improved facilitating dynamics at the neuronal level to overcome this limitation. In this paper, we tested our proposed approach in controllers for 2D pole balancing, where the new approach was shown to perform better than the previous FAN model. Next, we investigated the differential utilization of facilitating dynamics in sensory vs. motor neurons and found that motor neurons utilize the facilitating dynamics more than the sensory neurons. These findings are expected to help us better understand the role of facilitating dynamics in delay compensation, and its potential development into prediction, a necessary condition for goal-directed behavior.

Extrapolative delay compensation through facilitating synapses and its relation to the flash-lag effect.

Neural conduction delay is a serious issue for organisms that need to act in real time. Various forms of flash-lag effect (FLE) suggest that the nervous system may perform extrapolation to compensate for delay. For example, in motion FLE, the position of a moving object is perceived to be ahead of a brief flash when they are actually colocalized. However, the precise mechanism for extrapolation at a single-neuron level has not been fully investigated. Our hypothesis is that facilitating synapses, with their dynamic sensitivity to the rate of change in the input, can serve as a neural basis for extrapolation. To test this hypothesis, we constructed and tested models of facilitating dynamics. First, we derived a spiking neuron model of facilitating dynamics at a single-neuron level, and tested it in the luminance FLE domain. Second, the spiking neuron model was extended to include multiple neurons and spike-timing-dependent plasticity (STDP), and was tested with orientation FLE. The results showed a strong relationship between delay compensation, FLE, and facilitating synapses/STDP. The results are expected to shed new light on real time and predictive processing in the brain, at the single neuron level.

Dynamical pathway analysis.

BACKGROUND: Although a great deal is known about one gene or protein and its functions under different environmental conditions, little information is available about the complex behaviour of biological networks subject to different environmental perturbations. Observing differential expressions of one or more genes between normal and abnormal cells has been a mainstream method of discovering pertinent genes in diseases and therefore valuable drug targets. However, to date, no such method exists for elucidating and quantifying the differential dynamical behaviour of genetic regulatory networks, which can have greater impact on phenotypes than individual genes. RESULTS: We propose to redress the deficiency by formulating the functional study of biological networks as a control problem of dynamical systems. We developed mathematical methods to study the stability, the controllability, and the steady-state behaviour, as well as the transient responses of biological networks under different environmental perturbations. We applied our framework to three real-world datasets: the SOS DNA repair network in E. coli under different dosages of radiation, the GSH redox cycle in mice lung exposed to either poisonous air or normal air, and the MAPK pathway in mammalian cell lines exposed to three types of HIV type I Vpr, a wild type and two mutant types; and we found that the three genetic networks exhibited fundamentally different dynamical properties in normal and abnormal cells. CONCLUSION: Difference in stability, relative stability, degrees of controllability, and transient responses between normal and abnormal cells means considerable difference in dynamical behaviours and different functioning of cells. Therefore differential dynamical properties can be a valuable tool in biomedical research.