Brain Decoding Using fMRI Images for Multiple Subjects through Deep Learning.

Substantial information related to human cerebral conditions can be decoded through various noninvasive evaluating techniques like fMRI. Exploration of the neuronal activity of the human brain can divulge the thoughts of a person like what the subject is perceiving, thinking, or visualizing. Furthermore, deep learning techniques can be used to decode the multifaceted patterns of the brain in response to external stimuli. Existing techniques are capable of exploring and classifying the thoughts of the human subject acquired by the fMRI imaging data. fMRI images are the volumetric imaging scans which are highly dimensional as well as require a lot of time for training when fed as an input in the deep learning network. However, the hassle for more efficient learning of highly dimensional high-level features in less training time and accurate interpretation of the brain voxels with less misclassification error is needed. In this research, we propose an improved CNN technique where features will be functionally aligned. The optimal features will be selected after dimensionality reduction. The highly dimensional feature vector will be transformed into low dimensional space for dimensionality reduction through autoadjusted weights and combination of best activation functions. Furthermore, we solve the problem of increased training time by using Swish activation function, making it denser and increasing efficiency of the model in less training time. Finally, the experimental results are evaluated and compared with other classifiers which demonstrated the supremacy of the proposed model in terms of accuracy.

Aberrant Brain Functional Connectivity Strength and Effective Connectivity in Patients with Type 2 Diabetes Mellitus.

Alterations of brain functional connectivity in patients with type 2 diabetes mellitus (T2DM) have been reported by resting-state functional magnetic resonance imaging studies, but the underlying precise neuropathological mechanism remains unclear. This study is aimed at investigating the implicit alterations of functional connections in T2DM by integrating functional connectivity strength (FCS) and Granger causality analysis (GCA) and further exploring their associations with clinical characteristics. Sixty T2DM patients and thirty-three sex-, age-, and education-matched healthy controls (HC) were recruited. Global FCS analysis of resting-state functional magnetic resonance imaging was performed to explore seed regions with significant differences between the two groups; then, GCA was applied to detect directional effective connectivity (EC) between the seeds and other brain regions. Correlations of EC with clinical variables were further explored in T2DM patients. Compared with HC, T2DM patients showed lower FCS in the bilateral fusiform gyrus, right superior frontal gyrus (SFG), and right postcentral gyrus, but higher FCS in the right supplementary motor area (SMA). Moreover, altered directional EC was found between the left fusiform gyrus and bilateral lingual gyrus and right medial frontal gyrus (MFG), as well as between the right SFG and bilateral frontal regions. In addition, triglyceride, insulin, and plasma glucose levels were correlated with the abnormal EC of the left fusiform, while disease duration and cognitive function were associated with the abnormal EC of the right SFG in T2DM patients. These results suggest that T2DM patients show aberrant brain function connectivity strength and effective connectivity which is associated with the diabetes-related metabolic characteristics, disease duration, and cognitive function, providing further insights into the complex neural basis of diabetes.

Nonlinear visuoauditory integration in the mouse superior colliculus.

Sensory information from different modalities is processed in parallel, and then integrated in associative brain areas to improve object identification and the interpretation of sensory experiences. The Superior Colliculus (SC) is a midbrain structure that plays a critical role in integrating visual, auditory, and somatosensory input to assess saliency and promote action. Although the response properties of the individual SC neurons to visuoauditory stimuli have been characterized, little is known about the spatial and temporal dynamics of the integration at the population level. Here we recorded the response properties of SC neurons to spatially restricted visual and auditory stimuli using large-scale electrophysiology. We then created a general, population-level model that explains the spatial, temporal, and intensity requirements of stimuli needed for sensory integration. We found that the mouse SC contains topographically organized visual and auditory neurons that exhibit nonlinear multisensory integration. We show that nonlinear integration depends on properties of auditory but not visual stimuli. We also find that a heuristically derived nonlinear modulation function reveals conditions required for sensory integration that are consistent with previously proposed models of sensory integration such as spatial matching and the principle of inverse effectiveness.

Estimating the dimensionality of the manifold underlying multi-electrode neural recordings.

It is generally accepted that the number of neurons in a given brain area far exceeds the number of neurons needed to carry any specific function controlled by that area. For example, motor areas of the human brain contain tens of millions of neurons that control the activation of tens or at most hundreds of muscles. This massive redundancy implies the covariation of many neurons, which constrains the population activity to a low-dimensional manifold within the space of all possible patterns of neural activity. To gain a conceptual understanding of the complexity of the neural activity within a manifold, it is useful to estimate its dimensionality, which quantifies the number of degrees of freedom required to describe the observed population activity without significant information loss. While there are many algorithms for dimensionality estimation, we do not know which are well suited for analyzing neural activity. The objective of this study was to evaluate the efficacy of several representative algorithms for estimating the dimensionality of linearly and nonlinearly embedded data. We generated synthetic neural recordings with known intrinsic dimensionality and used them to test the algorithms' accuracy and robustness. We emulated some of the important challenges associated with experimental data by adding noise, altering the nature of the embedding of the low-dimensional manifold within the high-dimensional recordings, varying the dimensionality of the manifold, and limiting the amount of available data. We demonstrated that linear algorithms overestimate the dimensionality of nonlinear, noise-free data. In cases of high noise, most algorithms overestimated the dimensionality. We thus developed a denoising algorithm based on deep learning, the "Joint Autoencoder", which significantly improved subsequent dimensionality estimation. Critically, we found that all algorithms failed when the intrinsic dimensionality was high (above 20) or when the amount of data used for estimation was low. Based on the challenges we observed, we formulated a pipeline for estimating the dimensionality of experimental neural data.

Dynamic Electrode-to-Image (DETI) mapping reveals the human brain's spatiotemporal code of visual information.

A number of neuroimaging techniques have been employed to understand how visual information is transformed along the visual pathway. Although each technique has spatial and temporal limitations, they can each provide important insights into the visual code. While the BOLD signal of fMRI can be quite informative, the visual code is not static and this can be obscured by fMRI's poor temporal resolution. In this study, we leveraged the high temporal resolution of EEG to develop an encoding technique based on the distribution of responses generated by a population of real-world scenes. This approach maps neural signals to each pixel within a given image and reveals location-specific transformations of the visual code, providing a spatiotemporal signature for the image at each electrode. Our analyses of the mapping results revealed that scenes undergo a series of nonuniform transformations that prioritize different spatial frequencies at different regions of scenes over time. This mapping technique offers a potential avenue for future studies to explore how dynamic feedforward and recurrent processes inform and refine high-level representations of our visual world.

Topology-preserving smoothing of retinotopic maps.

Retinotopic mapping, i.e., the mapping between visual inputs on the retina and neuronal activations in cortical visual areas, is one of the central topics in visual neuroscience. For human observers, the mapping is obtained by analyzing functional magnetic resonance imaging (fMRI) signals of cortical responses to slowly moving visual stimuli on the retina. Although it is well known from neurophysiology that the mapping is topological (i.e., the topology of neighborhood connectivity is preserved) within each visual area, retinotopic maps derived from the state-of-the-art methods are often not topological because of the low signal-to-noise ratio and spatial resolution of fMRI. The violation of topological condition is most severe in cortical regions corresponding to the neighborhood of the fovea (e.g., < 1 degree eccentricity in the Human Connectome Project (HCP) dataset), significantly impeding accurate analysis of retinotopic maps. This study aims to directly model the topological condition and generate topology-preserving and smooth retinotopic maps. Specifically, we adopted the Beltrami coefficient, a metric of quasiconformal mapping, to define the topological condition, developed a mathematical model to quantify topological smoothing as a constrained optimization problem, and elaborated an efficient numerical method to solve the problem. The method was then applied to V1, V2, and V3 simultaneously in the HCP dataset. Experiments with both simulated and real retinotopy data demonstrated that the proposed method could generate topological and smooth retinotopic maps.

Unveiling functions of the visual cortex using task-specific deep neural networks.

The human visual cortex enables visual perception through a cascade of hierarchical computations in cortical regions with distinct functionalities. Here, we introduce an AI-driven approach to discover the functional mapping of the visual cortex. We related human brain responses to scene images measured with functional MRI (fMRI) systematically to a diverse set of deep neural networks (DNNs) optimized to perform different scene perception tasks. We found a structured mapping between DNN tasks and brain regions along the ventral and dorsal visual streams. Low-level visual tasks mapped onto early brain regions, 3-dimensional scene perception tasks mapped onto the dorsal stream, and semantic tasks mapped onto the ventral stream. This mapping was of high fidelity, with more than 60% of the explainable variance in nine key regions being explained. Together, our results provide a novel functional mapping of the human visual cortex and demonstrate the power of the computational approach.

A strategy for mapping biophysical to abstract neuronal network models applied to primary visual cortex.

A fundamental challenge for the theoretical study of neuronal networks is to make the link between complex biophysical models based directly on experimental data, to progressively simpler mathematical models that allow the derivation of general operating principles. We present a strategy that successively maps a relatively detailed biophysical population model, comprising conductance-based Hodgkin-Huxley type neuron models with connectivity rules derived from anatomical data, to various representations with fewer parameters, finishing with a firing rate network model that permits analysis. We apply this methodology to primary visual cortex of higher mammals, focusing on the functional property of stimulus orientation selectivity of receptive fields of individual neurons. The mapping produces compact expressions for the parameters of the abstract model that clearly identify the impact of specific electrophysiological and anatomical parameters on the analytical results, in particular as manifested by specific functional signatures of visual cortex, including input-output sharpening, conductance invariance, virtual rotation and the tilt after effect. Importantly, qualitative differences between model behaviours point out consequences of various simplifications. The strategy may be applied to other neuronal systems with appropriate modifications.

Behavioral correlates of cortical semantic representations modeled by word vectors.

The quantitative modeling of semantic representations in the brain plays a key role in understanding the neural basis of semantic processing. Previous studies have demonstrated that word vectors, which were originally developed for use in the field of natural language processing, provide a powerful tool for such quantitative modeling. However, whether semantic representations in the brain revealed by the word vector-based models actually capture our perception of semantic information remains unclear, as there has been no study explicitly examining the behavioral correlates of the modeled brain semantic representations. To address this issue, we compared the semantic structure of nouns and adjectives in the brain estimated from word vector-based brain models with that evaluated from human behavior. The brain models were constructed using voxelwise modeling to predict the functional magnetic resonance imaging (fMRI) response to natural movies from semantic contents in each movie scene through a word vector space. The semantic dissimilarity of brain word representations was then evaluated using the brain models. Meanwhile, data on human behavior reflecting the perception of semantic dissimilarity between words were collected in psychological experiments. We found a significant correlation between brain model- and behavior-derived semantic dissimilarities of words. This finding suggests that semantic representations in the brain modeled via word vectors appropriately capture our perception of word meanings.

The impact of emotionally valenced music on emotional state and EEG profile: Convergence of self-report and quantitative data.

Musical stimuli can induce a variety of emotions in individuals. We sought to determine whether different valenced music would induce EEG profile changes and self-reported emotional states in individuals following the viewing of a complex video with a concrete narrative and emotional ambivalence. We used a five-minute video titled "El Empleo", coupled with either joyful, fearful, or no music. EEG recordings were taken throughout the duration of the experiment and a self-reported questionnaire on emotional state was administered after viewing of the video. We found self-reported measures of happiness increased following viewing of the video paired with joyful music, while EEG data demonstrated that the following brain regions displayed significant changes in activity following both fearful and joyful music: the right inferior parietal lobule, left uncus, and left insula. Additionally, we found that anxiety self-report scores correlated negatively with average gamma activity within the insula within each group. The convergence of self-reported data and quantitative EEG data was consistent across 27 participants. These data indicate that different valenced music can alter EEG activity in emotion specific regions, reflected in participants perceived emotional state.

Reliability of task-specific neuronal activation assessed with functional PET, ASL and BOLD imaging.

Mapping the neuronal response during cognitive processing is of crucial importance to gain new insights into human brain function. BOLD imaging and ASL are established MRI methods in this endeavor. Recently, the novel approach of functional PET (fPET) was introduced, enabling absolute quantification of glucose metabolism at rest and during task execution in a single measurement. Here, we report test-retest reliability of fPET in direct comparison to BOLD imaging and ASL. Twenty healthy subjects underwent two PET/MRI measurements, providing estimates of glucose metabolism, cerebral blood flow (CBF) and blood oxygenation. A cognitive task was employed with different levels of difficulty requiring visual-motor coordination. Task-specific neuronal activation was robustly detected with all three imaging approaches. The highest reliability was obtained for glucose metabolism at rest. Although this dropped during task performance it was still comparable to that of CBF. In contrast, BOLD imaging yielded high performance only for qualitative spatial overlap of task effects but not for quantitative comparison. Hence, the combined assessment of fPET and ASL offers reliable and simultaneous absolute quantification of glucose metabolism and CBF at rest and task.

Mathematical Modeling of Brain Activity under Specific Auditory Stimulation.

Understanding the connection between different stimuli and the brain response represents a complex research area. However, the use of mathematical models for this purpose is relatively unexplored. The present study investigates the effects of three different auditory stimuli on cerebral biopotentials by means of mathematical functions. The effects of acoustic stimuli (S1, S2, and S3) on cerebral activity were evaluated by electroencephalographic (EEG) recording on 21 subjects for 20 minutes of stimulation, with a 5-minute period of silence before and after stimulation. For the construction of the mathematical models used for the study of the EEG rhythms, we used the Box-Jenkins methodology. Characteristic mathematical models were obtained for the main frequency bands and were expressed by 2 constant functions, 8 first-degree functions, a second-degree function, a fourth-degree function, 6 recursive functions, and 4 periodic functions. The values obtained for the variance estimator are low, demonstrating that the obtained models are correct. The resulting mathematical models allow us to objectively compare the EEG response to the three stimuli, both between the stimuli itself and between each stimulus and the period before stimulation.

Statistical power: Implications for planning MEG studies.

Statistical power is key for robust, replicable science. Here, we systematically explored how numbers of trials and subjects affect statistical power in MEG sensor-level data. More specifically, we simulated "experiments" using the MEG resting-state dataset of the Human Connectome Project (HCP). We divided the data in two conditions, injected a dipolar source at a known anatomical location in the "signal condition", but not in the "noise condition", and detected significant differences at sensor level with classical paired t-tests across subjects, using amplitude, squared amplitude, and global field power (GFP) measures. Group-level detectability of these simulated effects varied drastically with anatomical origin. We thus examined in detail which spatial properties of the sources affected detectability, looking specifically at the distance from closest sensor and orientation of the source, and at the variability of these parameters across subjects. In line with previous single-subject studies, we found that the most detectable effects originate from source locations that are closest to the sensors and oriented tangentially with respect to the head surface. In addition, cross-subject variability in orientation also affected group-level detectability, boosting detection in regions where this variability was small and hindering detection in regions where it was large. Incidentally, we observed a considerable covariation of source position, orientation, and their cross-subject variability in individual brain anatomical space, making it difficult to assess the impact of each of these variables independently of one another. We thus also performed simulations where we controlled spatial properties independently of individual anatomy. These additional simulations confirmed the strong impact of distance and orientation and further showed that orientation variability across subjects affects detectability, whereas position variability does not. Importantly, our study indicates that strict unequivocal recommendations as to the ideal number of trials and subjects for any experiment cannot be realistically provided for neurophysiological studies and should be adapted according to the brain regions under study.

Rude mechanicals in brain haemodynamics: non-neural actors that influence blood flow.

Fluctuations in blood oxygenation and flow are widely used to infer brain activity during resting-state functional magnetic resonance imaging (fMRI). However, there are strong systemic and vascular contributions to resting-state signals that are unrelated to ongoing neural activity. Importantly, these non-neural contributions to haemodynamic signals (or 'rude mechanicals') can be as large as or larger than the neurally evoked components. Here, we review the two broad classes of drivers of these signals. One is systemic and is tied to fluctuations in external drivers such as heart rate and breathing, and the robust autoregulatory mechanisms that try to maintain a constant milieu in the brain. The other class comprises local, active fluctuations that appear to be intrinsic to vascular tissue and are likely similar to active local fluctuations seen in vasculature all over the body. In this review, we describe these non-neural fluctuations and some of the tools developed to correct for them when interpreting fMRI recordings. However, we also emphasize the links between these vascular fluctuations and brain physiology and point to ways in which fMRI measurements can be used to exploit such links to gain valuable information about neurovascular health and about internal brain states. This article is part of the theme issue 'Key relationships between non-invasive functional neuroimaging and the underlying neuronal activity'.

Prefiltering based on experimental paradigm for analysis of fMRI complex brain networks.

Brain networks offers a new insight about connections between function and anatomical regions of human brain. We present results from brain networks built from functional magnetic resonance images during finger tapping paradigm. Pearson voxel-voxel correlation in time and frequency domains were performed for all subjects. Besides this standard framework we have implemented a new approach consisting in filtering the data with respect to the fMRI paradigm (finger tapping) in order to obtain a better understanding of the network involved in the execution of the task. The main topological graph measures have been compared in both cases: voxel-voxel correlation and voxel-paradigm filtering plus voxel-voxel correlation. With the standard voxel-voxel correlation a clearly free-scale network was obtained. On the other hand, when we prefiltered the paradigm we obtained two different kind of networks: 1) free-scale; 2) random-like. To our best knowledge, this behaviour is reported here for first time for brain networks. We suggest that paradigm signal prefiltering can provide more infomation about the brain networks.

A meta-analysis of functional magnetic resonance imaging studies of divergent thinking using activation likelihood estimation.

There are conflicting findings regarding brain regions and networks underpinning creativity, with divergent thinking tasks commonly used to study this. A handful of meta-analyses have attempted to synthesise findings on neural mechanisms of divergent thinking. With the rapid proliferation of research and recent developments in fMRI meta-analysis approaches, it is timely to reassess the regions activated during divergent thinking creativity tasks. Of particular interest is examining the evidence regarding large-scale brain networks proposed to be key in divergent thinking and extending this work to consider the role of the semantic control network. Studies utilising fMRI with healthy participants completing divergent thinking tasks were systematically identified, with 20 studies meeting the criteria. Activation Likelihood Estimation was then used to integrate the neuroimaging results across studies. This revealed four clusters: the left inferior parietal lobe; the left inferior frontal and precentral gyrus; the superior and medial frontal gyrus and the right cerebellum. These regions are key in the semantic network, important for flexible retrieval of stored knowledge, highlighting the role of this network in divergent thinking.

Evaluating the Acute Effect of Stereoscopic Recovery by Dichoptic Stimulation Using Electroencephalogram.

Amblyopia is a common developmental disorder in adolescents and children. Stereoscopic loss is a symptom of amblyopia that can seriously affect the quality of patient's life. Recent studies have shown that the push-pull perceptual learning protocol had a positive effect on stereoscopic recovery. In this study, we developed a stereoscopic training method using a polarized visualization system according to the push-pull protocol. Dichoptic stimulation for 36 anisometropic and amblyopic subjects and 33 children with normal visual acuity (VA) has been conducted. Electroencephalogram (EEG) was used to evaluate the neurophysiological changes before, during, and after stimulation. For the anisometropic and amblyopic subjects, the statistical analysis demonstrated significant differences (p < 0.01) in the beta rhythm at the middle temporal and occipital lobes, while the EEG from the normal VA subjects indicated no significant changes when comparing the results before and after training. We concluded that the dichoptic training in our study can activate the middle temporal visual area and visual cortex. The EEG changes can be used to evaluate the training effects. This study also found that the beta band EEG acquired during visual stimulation at the dorsal visual stream can be potentially used for predicting acute training effect. The results facilitated the optimization of the individual training plan.

Influence of Patient-Specific Head Modeling on EEG Source Imaging.

Electromagnetic source imaging (ESI) techniques have become one of the most common alternatives for understanding cognitive processes in the human brain and for guiding possible therapies for neurological diseases. However, ESI accuracy strongly depends on the forward model capabilities to accurately describe the subject's head anatomy from the available structural data. Attempting to improve the ESI performance, we enhance the brain structure model within the individual-defined forward problem formulation, combining the head geometry complexity of the modeled tissue compartments and the prior knowledge of the brain tissue morphology. We validate the proposed methodology using 25 subjects, from which a set of magnetic-resonance imaging scans is acquired, extracting the anatomical priors and an electroencephalography signal set needed for validating the ESI scenarios. Obtained results confirm that incorporating patient-specific head models enhances the performed accuracy and improves the localization of focal and deep sources.

Non-Invasive Functional-Brain-Imaging with an OPM-based Magnetoencephalography System.

A non-invasive functional-brain-imaging system based on optically-pumped-magnetometers (OPM) is presented. The OPM-based magnetoencephalography (MEG) system features 20 OPM channels conforming to the subject's scalp. We have conducted two MEG experiments on three subjects: assessment of somatosensory evoked magnetic field (SEF) and auditory evoked magnetic field (AEF) using our OPM-based MEG system and a commercial MEG system based on superconducting quantum interference devices (SQUIDs). We cross validated the robustness of our system by calculating the distance between the location of the equivalent current dipole (ECD) yielded by our OPM-based MEG system and the ECD location calculated by the commercial SQUID-based MEG system. We achieved sub-centimeter accuracy for both SEF and AEF responses in all three subjects. Due to the proximity (12 mm) of the OPM channels to the scalp, it is anticipated that future OPM-based MEG systems will offer enhanced spatial resolution as they will capture finer spatial features compared to traditional MEG systems employing SQUIDs.

Coordinate based meta-analysis of networks in neuroimaging studies.

Meta-analysis of summary results from published neuroimaging studies independently testing a common hypothesis is performed using coordinate based meta-analysis (CBMA), which tests for consistent activation (in the case of functional MRI studies) of the same anatomical regions. Using just the reported coordinates it is also possible to meta-analyse coactivated regions to reveal a network-like structure of coordinate clusters (network nodes) distributed at the coactivated locations and a measure of the coactivation strength (network edges), which is determined by the presence/absence of reported activation. Here a new coordinate-based method to estimate a network of coactivations is detailed, which utilises the Z score accompanying each reported. Coordinate based meta-analysis of networks (CBMAN) assumes that if the activation pattern reported by independent studies is truly consistent, then the relative magnitude of these Z scores might also be consistent. It is hypothesised that this is detectable as Z score covariance between coactivated regions provided the within study variances are small. Advantages of using the Z scores instead of coordinates to measure coactivation strength are that censoring by the significance thresholds can be considered, and that using a continuous measure rather than a dichotomous one can increase statistical power. CBMAN uses maximum likelihood estimation to fit multivariate normal distributions to the standardised Z scores, and the covariances are considered as edges of a network of coactivated clusters (nodes). Here it is validated by numerical simulation and demonstrated on real data used previously to demonstrate CBMA. Software to perform CBMAN is freely available.