Echo state networks for the recognition of type 1 Brugada syndrome from conventional 12-LEAD ECG.

Artificial Intelligence (AI) applications and Machine Learning (ML) methods have gained much attention in recent years for their ability to automatically detect patterns in data without being explicitly taught rules. Specific features characterise the ECGs of patients with Brugada Syndrome (BrS); however, there is still ambiguity regarding the correct diagnosis of BrS and its differentiation from other pathologies. This work presents an application of Echo State Networks (ESN) in the Recurrent Neural Networks (RNN) class for diagnosing BrS from the ECG time series. 12-lead ECGs were obtained from patients with a definite clinical diagnosis of spontaneous BrS Type 1 pattern (Group A), patients who underwent provocative pharmacological testing to induce BrS type 1 pattern, which resulted in positive (Group B) or negative (Group C), and control subjects (Group D). One extracted beat in the V2 lead was used as input, and the dataset was used to train and evaluate the ESN model using a double cross-validation approach. ESN performance was compared with that of 4 cardiologists trained in electrophysiology. The model performance was assessed in the dataset, with a correct global diagnosis observed in 91.5 % of cases compared to clinicians (88.0 %). High specificity (94.5 %), sensitivity (87.0 %) and AUC (94.7 %) for BrS recognition by ESN were observed in Groups A + B vs. C + D. Our results show that this ML model can discriminate Type 1 BrS ECGs with high accuracy comparable to expert clinicians. Future availability of larger datasets may improve the model performance and increase the potential of the ESN as a clinical support system tool for daily clinical practice.

Exploiting the structure of biochemical pathways to investigate dynamical properties with neural networks for graphs.

MOTIVATION: Dynamical properties of biochemical pathways (BPs) help in understanding the functioning of living cells. Their in silico assessment requires simulating a dynamical system with a large number of parameters such as kinetic constants and species concentrations. Such simulations are based on numerical methods that can be time-expensive for large BPs. Moreover, parameters are often unknown and need to be estimated. RESULTS: We developed a framework for the prediction of dynamical properties of BPs directly from the structure of their graph representation. We represent BPs as Petri nets, which can be automatically generated, for instance, from standard SBML representations. The core of the framework is a neural network for graphs that extracts relevant information directly from the Petri net structure and exploits them to learn the association with the desired dynamical property. We show experimentally that the proposed approach reliably predicts a range of diverse dynamical properties (robustness, monotonicity, and sensitivity) while being faster than numerical methods at prediction time. In synergy with the neural network models, we propose a methodology based on Petri nets arc knock-out that allows the role of each molecule in the occurrence of a certain dynamical property to be better elucidated. The methodology also provides insights useful for interpreting the predictions made by the model. The results support the conjecture often considered in the context of systems biology that the BP structure plays a primary role in the assessment of its dynamical properties. AVAILABILITY AND IMPLEMENTATION: https://github.com/marcopodda/petri-bio (code), https://zenodo.org/record/7610382 (data).

A Deep Graph Network-Enhanced Sampling Approach to Efficiently Explore the Space of Reduced Representations of Proteins.

The limits of molecular dynamics (MD) simulations of macromolecules are steadily pushed forward by the relentless development of computer architectures and algorithms. The consequent explosion in the number and extent of MD trajectories induces the need for automated methods to rationalize the raw data and make quantitative sense of them. Recently, an algorithmic approach was introduced by some of us to identify the subset of a protein's atoms, or mapping, that enables the most informative description of the system. This method relies on the computation, for a given reduced representation, of the associated mapping entropy, that is, a measure of the information loss due to such simplification; albeit relatively straightforward, this calculation can be time-consuming. Here, we describe the implementation of a deep learning approach aimed at accelerating the calculation of the mapping entropy. We rely on Deep Graph Networks, which provide extreme flexibility in handling structured input data and whose predictions prove to be accurate and-remarkably efficient. The trained network produces a speedup factor as large as 105 with respect to the algorithmic computation of the mapping entropy, enabling the reconstruction of its landscape by means of the Wang-Landau sampling scheme. Applications of this method reach much further than this, as the proposed pipeline is easily transferable to the computation of arbitrary properties of a molecular structure.

A gentle introduction to deep learning for graphs.

The adaptive processing of graph data is a long-standing research topic that has been lately consolidated as a theme of major interest in the deep learning community. The snap increase in the amount and breadth of related research has come at the price of little systematization of knowledge and attention to earlier literature. This work is a tutorial introduction to the field of deep learning for graphs. It favors a consistent and progressive presentation of the main concepts and architectural aspects over an exposition of the most recent literature, for which the reader is referred to available surveys. The paper takes a top-down view of the problem, introducing a generalized formulation of graph representation learning based on a local and iterative approach to structured information processing. Moreover, it introduces the basic building blocks that can be combined to design novel and effective neural models for graphs. We complement the methodological exposition with a discussion of interesting research challenges and applications in the field.

A machine learning approach to estimating preterm infants survival: development of the Preterm Infants Survival Assessment (PISA) predictor.

Estimation of mortality risk of very preterm neonates is carried out in clinical and research settings. We aimed at elaborating a prediction tool using machine learning methods. We developed models on a cohort of 23747 neonates <30 weeks gestational age, or <1501 g birth weight, enrolled in the Italian Neonatal Network in 2008-2014 (development set), using 12 easily collected perinatal variables. We used a cohort from 2015-2016 (N = 5810) as a test set. Among several machine learning methods we chose artificial Neural Networks (NN). The resulting predictor was compared with logistic regression models. In the test cohort, NN had a slightly better discrimination than logistic regression (P < 0.002). The differences were greater in subgroups of neonates (at various gestational age or birth weight intervals, singletons). Using a cutoff of death probability of 0.5, logistic regression misclassified 67/5810 neonates (1.2 percent) more than NN. In conclusion our study - the largest published so far - shows that even in this very simplified scenario, using only limited information available up to 5 minutes after birth, a NN approach had a small but significant advantage over current approaches. The software implementing the predictor is made freely available to the community.

Design of deep echo state networks.

In this paper, we provide a novel approach to the architectural design of deep Recurrent Neural Networks using signal frequency analysis. In particular, focusing on the Reservoir Computing framework and inspired by the principles related to the inherent effect of layering, we address a fundamental open issue in deep learning, namely the question of how to establish the number of layers in recurrent architectures in the form of deep echo state networks (DeepESNs). The proposed method is first analyzed and refined on a controlled scenario and then it is experimentally assessed on challenging real-world tasks. The achieved results also show the ability of properly designed DeepESNs to outperform RC approaches on a speech recognition task, and to compete with the state-of-the-art in time-series prediction on polyphonic music tasks.

Generative Kernels for Tree-Structured Data.

This paper presents a family of methods for the design of adaptive kernels for tree-structured data that exploits the summarization properties of hidden states of hidden Markov models for trees. We introduce a compact and discriminative feature space based on the concept of hidden states multisets and we discuss different approaches to estimate such hidden state encoding. We show how it can be used to build an efficient and general tree kernel based on Jaccard similarity. Furthermore, we derive an unsupervised convolutional generative kernel using a topology induced on the Markov states by a tree topographic mapping. This paper provides an extensive empirical assessment on a variety of structured data learning tasks, comparing the predictive accuracy and computational efficiency of state-of-the-art generative, adaptive, and syntactical tree kernels. The results show that the proposed generative approach has a good tradeoff between computational complexity and predictive performance, in particular when considering the soft matching introduced by the topographic mapping.

Localizing Tortoise Nests by Neural Networks.

The goal of this research is to recognize the nest digging activity of tortoises using a device mounted atop the tortoise carapace. The device classifies tortoise movements in order to discriminate between nest digging, and non-digging activity (specifically walking and eating). Accelerometer data was collected from devices attached to the carapace of a number of tortoises during their two-month nesting period. Our system uses an accelerometer and an activity recognition system (ARS) which is modularly structured using an artificial neural network and an output filter. For the purpose of experiment and comparison, and with the aim of minimizing the computational cost, the artificial neural network has been modelled according to three different architectures based on the input delay neural network (IDNN). We show that the ARS can achieve very high accuracy on segments of data sequences, with an extremely small neural network that can be embedded in programmable low power devices. Given that digging is typically a long activity (up to two hours), the application of ARS on data segments can be repeated over time to set up a reliable and efficient system, called Tortoise@, for digging activity recognition.

Compositional generative mapping for tree-structured data--part I: bottom-up probabilistic modeling of trees.

We introduce a novel compositional (recursive) probabilistic model for trees that defines an approximated bottom-up generative process from the leaves to the root of a tree. The proposed model defines contextual state transitions from the joint configuration of the children to the parent nodes. We argue that the bottom-up context postulates different probabilistic assumptions with respect to a top-down approach, leading to different representational capabilities. We discuss classes of applications that are best suited to a bottom-up approach. In particular, the bottom-up context is shown to better correlate and model the co-occurrence of substructures among the child subtrees of internal nodes. A mixed memory approximation is introduced to factorize the joint children-to-parent state transition matrix as a mixture of pairwise transitions. The proposed approach is the first practical bottom-up generative model for tree-structured data that maintains the same computational class of its top-down counterpart. Comparative experimental analyses exploiting synthetic and real-world datasets show that the proposed model can deal with deep structures better than a top-down generative model. The model is also shown to better capture structural information from real-world data comprising trees with a large out-degree. The proposed bottom-up model can be used as a fundamental building block for the development of other new powerful models.

Architectural and Markovian factors of echo state networks.

Echo State Networks (ESNs) constitute an emerging approach for efficiently modeling Recurrent Neural Networks (RNNs). In this paper we investigate some of the main aspects that can be accounted for the success and limitations of this class of models. In particular, we propose complementary classes of factors related to contractivity and architecture of reservoirs and we study their relative relevance. First, we show the existence of a class of tasks for which ESN performance is independent of the architectural design. The effect of the Markovian factor, characterizing a significant class within these cases, is shown by introducing instances of easy/hard tasks for ESNs featured by contractivity of reservoir dynamics. In the complementary cases, for which architectural design is effective, we investigate and decompose the aspects of network design that allow a larger reservoir to progressively improve the predictive performance. In particular, we introduce four key architectural factors: input variability, multiple time-scales dynamics, non-linear interactions among units and regression in an augmented feature space. To investigate the quantitative effects of the different architectural factors within this class of tasks successfully approached by ESNs, variants of the basic ESN model are proposed and tested on instances of datasets of different nature and difficulty. Experimental evidences confirm the role of the Markovian factor and show that all the identified key architectural factors have a major role in determining ESN performances.

QSPR Analysis of Copolymers by Recursive Neural Networks: Prediction of the Glass Transition Temperature of (Meth)acrylic Random Copolymers.

The glass transition temperature (Tg ) of acrylic and methacrylic random copolymers was investigated by means of Quantitative Structure-Property Relationship (QSPR) methodology based on Recursive Neural Networks (RNN). This method can directly take molecular structures as input, in the form of labelled trees, without needing predefined descriptors. It was applied to three data sets containing up to 615 polymers (340 homopolymers and 275 copolymers). The adopted representation was able to account for the structure of the repeating unit as well as average macromolecular characteristics, such as stereoregularity and molar composition. The best result, obtained on a data set focused on copolymers, showed a Mean Average Residual (MAR) of 4.9 K, a standard error of prediction (S) of 6.1 K and a squared correlation coefficient (R(2) ) of 0.98 for the test set, with an optimal rate with respect to the training error. Through the treatment of homopolymers and copolymers both as separated and merged data sets, we also showed that the proposed approach is particularly suited for generalizing prediction of polymer properties to various types of chemical structures in a uniform setting.

Neural network for graphs: a contextual constructive approach.

This paper presents a new approach for learning in structured domains (SDs) using a constructive neural network for graphs (NN4G). The new model allows the extension of the input domain for supervised neural networks to a general class of graphs including both acyclic/cyclic, directed/undirected labeled graphs. In particular, the model can realize adaptive contextual transductions, learning the mapping from graphs for both classification and regression tasks. In contrast to previous neural networks for structures that had a recursive dynamics, NN4G is based on a constructive feedforward architecture with state variables that uses neurons with no feedback connections. The neurons are applied to the input graphs by a general traversal process that relaxes the constraints of previous approaches derived by the causality assumption over hierarchical input data. Moreover, the incremental approach eliminates the need to introduce cyclic dependencies in the definition of the system state variables. In the traversal process, the NN4G units exploit (local) contextual information of the graphs vertices. In spite of the simplicity of the approach, we show that, through the compositionality of the contextual information developed by the learning, the model can deal with contextual information that is incrementally extended according to the graphs topology. The effectiveness and the generality of the new approach are investigated by analyzing its theoretical properties and providing experimental results.

Evaluation of hierarchical structured representations for QSPR studies of small molecules and polymers by recursive neural networks.

This paper reports some recent results from the empirical evaluation of different types of structured molecular representations used in QSPR analysis through a recursive neural network (RNN) model, which allows for their direct use without the need for measuring or computing molecular descriptors. This RNN methodology has been applied to the prediction of the properties of small molecules and polymers. In particular, three different descriptions of cyclic moieties, namely group, template and cyclebreak have been proposed. The effectiveness of the proposed method in dealing with different representations of chemical structures, either specifically designed or of more general use, has been demonstrated by its application to data sets encompassing various types of cyclic structures. For each class of experiments a test set with data that were not used for the development of the model was used for validation, and the comparisons have been based on the test results. The reported results highlight the flexibility of the RNN in directly treating different classes of structured input data without using input descriptors.

Modeling adaptive kernels from probabilistic phylogenetic trees.

OBJECTIVE: Modeling phylogenetic interactions is an open issue in many computational biology problems. In the context of gene function prediction we introduce a class of kernels for structured data leveraging on a hierarchical probabilistic modeling of phylogeny among species. METHODS AND MATERIALS: We derive three kernels belonging to this setting: a sufficient statistics kernel, a Fisher kernel, and a probability product kernel. The new kernels are used in the context of support vector machine learning. The kernels adaptivity is obtained through the estimation of the parameters of a tree structured model of evolution using as observed data phylogenetic profiles encoding the presence or absence of specific genes in a set of fully sequenced genomes. RESULTS: We report results obtained in the prediction of the functional class of the proteins of the budding yeast Saccharomyces cerevisae which favorably compare to a standard vector based kernel and to a non-adaptive tree kernel function. A further comparative analysis is performed in order to assess the impact of the different components of the proposed approach. CONCLUSIONS: We show that the key features of the proposed kernels are the adaptivity to the input domain and the ability to deal with structured data interpreted through a graphical model representation.

An introduction to recursive neural networks and kernel methods for cheminformatics.

The aim of this paper is to introduce the reader to new developments in Neural Networks and Kernel Machines concerning the treatment of structured domains. Specifically, we discuss the research on these relatively new models to introduce a novel and more general approach to QSPR/QSAR analysis. The focus is on the computational side and not on the experimental one.

Predicting physical-chemical properties of compounds from molecular structures by recursive neural networks.

In this paper, we report on the potential of a recently developed neural network for structures applied to the prediction of physical chemical properties of compounds. The proposed recursive neural network (RecNN) model is able to directly take as input a structured representation of the molecule and to model a direct and adaptive relationship between the molecular structure and target property. Therefore, it combines in a learning system the flexibility and general advantages of a neural network model with the representational power of a structured domain. As a result, a completely new approach to quantitative structure-activity relationship/quantitative structure-property relationship (QSPR/QSAR) analysis is obtained. An original representation of the molecular structures has been developed accounting for both the occurrence of specific atoms/groups and the topological relationships among them. Gibbs free energy of solvation in water, Delta(solv)G degrees , has been chosen as a benchmark for the model. The different approaches proposed in the literature for the prediction of this property have been reconsidered from a general perspective. The advantages of RecNN as a suitable tool for the automatization of fundamental parts of the QSPR/QSAR analysis have been highlighted. The RecNN model has been applied to the analysis of the Delta(solv)G degrees in water of 138 monofunctional acyclic organic compounds and tested on an external data set of 33 compounds. As a result of the statistical analysis, we obtained, for the predictive accuracy estimated on the test set, correlation coefficient R = 0.9985, standard deviation S = 0.68 kJ mol(-1), and mean absolute error MAE = 0.46 kJ mol(-1). The inherent ability of RecNN to abstract chemical knowledge through the adaptive learning process has been investigated by principal components analysis of the internal representations computed by the network. It has been found that the model recognizes the chemical compounds on the basis of a nontrivial combination of their chemical structure and target property.

Can individual repair kinetics of UVC-induced DNA damage in human lymphocytes be assessed through the comet assay?

The suitability of the comet assay for quantifying DNA repair capacity at individual level was studied following the kinetics of nucleotide excision repair (NER) in human lymphocytes from four healthy donors, at various time steps after a single dose of UVC. A significant increase of DNA migration was seen as soon as 20 min after UV exposure, reaching the peak within 60-90 min. Afterwards, a rapid decline was observed, approaching the basal level at 180-240 min. The increase could be ascribed to excision activity, while the reduction to gap filling and rejoining, as demonstrated by the effects of phase-specific inhibitors, novobiocin and aphidicolin. Therefore, the comet assay should allow following the biphasic kinetics of NER. Wide inter-individual differences were observed, although repeated tests on the same donor cells revealed a large experimental variation. To quantitatively compare the individual patterns, a mathematical model was developed that adequately fitted the experimental results and estimated appropriate descriptors for each phase and for each donor. A second approach was also used to directly compare the distributions of damaged cells and to assess the differences between donors and between experiments visualizing them as reciprocal distances on a two-dimensional space computed with a principal component analysis (PCA). The results confirmed the inter-individual differences, but also the strong influence of experimental factors of the comet assay. The two approaches provided the means of accurately comparing DNA repair kinetics at individual level, taking also into account the experimental variability which poses serious doubts on the suitability of the comet assay. Nevertheless, since this methodology allows a detailed analysis of repair kinetics and it is potentially very useful for identifying individual with reduced repair capacity, further efforts have to be addressed to improve the reproducibility of the comet assay.

Recursive self-organizing network models.

Self-organizing models constitute valuable tools for data visualization, clustering, and data mining. Here, we focus on extensions of basic vector-based models by recursive computation in such a way that sequential and tree-structured data can be processed directly. The aim of this article is to give a unified review of important models recently proposed in literature, to investigate fundamental mathematical properties of these models, and to compare the approaches by experiments. We first review several models proposed in literature from a unifying perspective, thereby making use of an underlying general framework which also includes supervised recurrent and recursive models as special cases. We shortly discuss how the models can be related to different neuron lattices. Then, we investigate theoretical properties of the models in detail: we explicitly formalize how structures are internally stored in different context models and which similarity measures are induced by the recursive mapping onto the structures. We assess the representational capabilities of the models, and we shortly discuss the issues of topology preservation and noise tolerance. The models are compared in an experiment with time series data. Finally, we add an experiment for one context model for tree-structured data to demonstrate the capability to process complex structures.

Contextual processing of structured data by recursive cascade correlation.

This paper propose a first approach to deal with contextual information in structured domains by recursive neural networks. The proposed model, i.e., contextual recursive cascade correlation (CRCC), a generalization of the recursive cascade correlation (RCC) model, is able to partially remove the causality assumption by exploiting contextual information stored in frozen units. We formally characterize the properties of CRCC showing that it is able to compute contextual transductions and also some causal supersource transductions that RCC cannot compute. Experimental results on controlled sequences and on a real-world task involving chemical structures confirm the computational limitations of RCC, while assessing the efficiency and efficacy of CRCC in dealing both with pure causal and contextual prediction tasks. Moreover, results obtained for the real-world task show the superiority of the proposed approach versus RCC when exploring a task for which it is not known whether the structural causality assumption holds.