A modular and adaptable analysis pipeline to compare slow cerebral rhythms across heterogeneous datasets.

Neuroscience is moving toward a more integrative discipline where understanding brain function requires consolidating the accumulated evidence seen across experiments, species, and measurement techniques. A remaining challenge on that path is integrating such heterogeneous data into analysis workflows such that consistent and comparable conclusions can be distilled as an experimental basis for models and theories. Here, we propose a solution in the context of slow-wave activity (<1 Hz), which occurs during unconscious brain states like sleep and general anesthesia and is observed across diverse experimental approaches. We address the issue of integrating and comparing heterogeneous data by conceptualizing a general pipeline design that is adaptable to a variety of inputs and applications. Furthermore, we present the Collaborative Brain Wave Analysis Pipeline (Cobrawap) as a concrete, reusable software implementation to perform broad, detailed, and rigorous comparisons of slow-wave characteristics across multiple, openly available electrocorticography (ECoG) and calcium imaging datasets.

Simulations approaching data: cortical slow waves in inferred models of the whole hemisphere of mouse.

The development of novel techniques to record wide-field brain activity enables estimation of data-driven models from thousands of recording channels and hence across large regions of cortex. These in turn improve our understanding of the modulation of brain states and the richness of traveling waves dynamics. Here, we infer data-driven models from high-resolution in-vivo recordings of mouse brain obtained from wide-field calcium imaging. We then assimilate experimental and simulated data through the characterization of the spatio-temporal features of cortical waves in experimental recordings. Inference is built in two steps: an inner loop that optimizes a mean-field model by likelihood maximization, and an outer loop that optimizes a periodic neuro-modulation via direct comparison of observables that characterize cortical slow waves. The model reproduces most of the features of the non-stationary and non-linear dynamics present in the high-resolution in-vivo recordings of the mouse brain. The proposed approach offers new methods of characterizing and understanding cortical waves for experimental and computational neuroscientists.

Evaluating the statistical similarity of neural network activity and connectivity via eigenvector angles.

Neural systems are networks, and strategic comparisons between multiple networks are a prevalent task in many research scenarios. In this study, we construct a statistical test for the comparison of matrices representing pairwise aspects of neural networks, in particular, the correlation between spiking activity and connectivity. The "eigenangle test" quantifies the similarity of two matrices by the angles between their ranked eigenvectors. We calibrate the behavior of the test for use with correlation matrices using stochastic models of correlated spiking activity and demonstrate how it compares to classical two-sample tests, such as the Kolmogorov-Smirnov distance, in the sense that it is able to evaluate also structural aspects of pairwise measures. Furthermore, the principle of the eigenangle test can be applied to compare the similarity of adjacency matrices of certain types of networks. Thus, the approach can be used to quantitatively explore the relationship between connectivity and activity with the same metric. By applying the eigenangle test to the comparison of connectivity matrices and correlation matrices of a random balanced network model before and after a specific synaptic rewiring intervention, we gauge the influence of connectivity features on the correlated activity. Potential applications of the eigenangle test include simulation experiments, model validation, and data analysis.

Rigorous Neural Network Simulations: A Model Substantiation Methodology for Increasing the Correctness of Simulation Results in the Absence of Experimental Validation Data.

The reproduction and replication of scientific results is an indispensable aspect of good scientific practice, enabling previous studies to be built upon and increasing our level of confidence in them. However, reproducibility and replicability are not sufficient: an incorrect result will be accurately reproduced if the same incorrect methods are used. For the field of simulations of complex neural networks, the causes of incorrect results vary from insufficient model implementations and data analysis methods, deficiencies in workmanship (e.g., simulation planning, setup, and execution) to errors induced by hardware constraints (e.g., limitations in numerical precision). In order to build credibility, methods such as verification and validation have been developed, but they are not yet well-established in the field of neural network modeling and simulation, partly due to ambiguity concerning the terminology. In this manuscript, we propose a terminology for model verification and validation in the field of neural network modeling and simulation. We outline a rigorous workflow derived from model verification and validation methodologies for increasing model credibility when it is not possible to validate against experimental data. We compare a published minimal spiking network model capable of exhibiting the development of polychronous groups, to its reproduction on the SpiNNaker neuromorphic system, where we consider the dynamics of several selected network states. As a result, by following a formalized process, we show that numerical accuracy is critically important, and even small deviations in the dynamics of individual neurons are expressed in the dynamics at network level.

Reproducible Neural Network Simulations: Statistical Methods for Model Validation on the Level of Network Activity Data.

Computational neuroscience relies on simulations of neural network models to bridge the gap between the theory of neural networks and the experimentally observed activity dynamics in the brain. The rigorous validation of simulation results against reference data is thus an indispensable part of any simulation workflow. Moreover, the availability of different simulation environments and levels of model description require also validation of model implementations against each other to evaluate their equivalence. Despite rapid advances in the formalized description of models, data, and analysis workflows, there is no accepted consensus regarding the terminology and practical implementation of validation workflows in the context of neural simulations. This situation prevents the generic, unbiased comparison between published models, which is a key element of enhancing reproducibility of computational research in neuroscience. In this study, we argue for the establishment of standardized statistical test metrics that enable the quantitative validation of network models on the level of the population dynamics. Despite the importance of validating the elementary components of a simulation, such as single cell dynamics, building networks from validated building blocks does not entail the validity of the simulation on the network scale. Therefore, we introduce a corresponding set of validation tests and present an example workflow that practically demonstrates the iterative model validation of a spiking neural network model against its reproduction on the SpiNNaker neuromorphic hardware system. We formally implement the workflow using a generic Python library that we introduce for validation tests on neural network activity data. Together with the companion study (Trensch et al., 2018), the work presents a consistent definition, formalization, and implementation of the verification and validation process for neural network simulations.