A Two-Layer Self-Organizing Map with Vector Symbolic Architecture for Spatiotemporal Sequence Learning and Prediction.

We propose a new nature- and neuro-science-inspired algorithm for spatiotemporal learning and prediction based on sequential recall and vector symbolic architecture. A key novelty is the learning of spatial and temporal patterns as decoupled concepts where the temporal pattern sequences are constructed using the learned spatial patterns as an alphabet of elements. The decoupling, motivated by cognitive neuroscience research, provides the flexibility for fast and adaptive learning with dynamic changes to data and concept drift and as such is better suited for real-time learning and prediction. The algorithm further addresses several key computational requirements for predicting the next occurrences based on real-life spatiotemporal data, which have been found to be challenging with current state-of-the-art algorithms. Firstly, spatial and temporal patterns are detected using unsupervised learning from unlabeled data streams in changing environments; secondly, vector symbolic architecture (VSA) is used to manage variable-length sequences; and thirdly, hyper dimensional (HD) computing-based associative memory is used to facilitate the continuous prediction of the next occurrences in sequential patterns. The algorithm has been empirically evaluated using two benchmark and three time-series datasets to demonstrate its advantages compared to the state-of-the-art in spatiotemporal unsupervised sequence learning where the proposed ST-SOM algorithm is able to achieve 45% error reduction compared to HTM algorithm.

Hardware Efficient Direct Policy Imitation Learning for Robotic Navigation in Resource-Constrained Settings.

Direct policy learning (DPL) is a widely used approach in imitation learning for time-efficient and effective convergence when training mobile robots. However, using DPL in real-world applications is not sufficiently explored due to the inherent challenges of mobilizing direct human expertise and the difficulty of measuring comparative performance. Furthermore, autonomous systems are often resource-constrained, thereby limiting the potential application and implementation of highly effective deep learning models. In this work, we present a lightweight DPL-based approach to train mobile robots in navigational tasks. We integrated a safety policy alongside the navigational policy to safeguard the robot and the environment. The approach was evaluated in simulations and real-world settings and compared with recent work in this space. The results of these experiments and the efficient transfer from simulations to real-world settings demonstrate that our approach has improved performance compared to its hardware-intensive counterparts. We show that using the proposed methodology, the training agent achieves closer performance to the expert within the first 15 training iterations in simulation and real-world settings.

Hyperseed: Unsupervised Learning With Vector Symbolic Architectures.

Motivated by recent innovations in biologically inspired neuromorphic hardware, this article presents a novel unsupervised machine learning algorithm named Hyperseed that draws on the principles of vector symbolic architectures (VSAs) for fast learning of a topology preserving feature map of unlabeled data. It relies on two major operations of VSA, binding and bundling. The algorithmic part of Hyperseed is expressed within the Fourier holographic reduced representations (FHRR) model, which is specifically suited for implementation on spiking neuromorphic hardware. The two primary contributions of the Hyperseed algorithm are few-shot learning and a learning rule based on single vector operation. These properties are empirically evaluated on synthetic datasets and on illustrative benchmark use cases, IRIS classification, and a language identification task using the n -gram statistics. The results of these experiments confirm the capabilities of Hyperseed and its applications in neuromorphic hardware.

Integer Echo State Networks: Efficient Reservoir Computing for Digital Hardware.

We propose an approximation of echo state networks (ESNs) that can be efficiently implemented on digital hardware based on the mathematics of hyperdimensional computing. The reservoir of the proposed integer ESN (intESN) is a vector containing only n -bits integers (where is normally sufficient for a satisfactory performance). The recurrent matrix multiplication is replaced with an efficient cyclic shift operation. The proposed intESN approach is verified with typical tasks in reservoir computing: memorizing of a sequence of inputs, classifying time series, and learning dynamic processes. Such architecture results in dramatic improvements in memory footprint and computational efficiency, with minimal performance loss. The experiments on a field-programmable gate array confirm that the proposed intESN approach is much more energy efficient than the conventional ESN.

Vector Symbolic Architectures as a Computing Framework for Emerging Hardware.

This article reviews recent progress in the development of the computing framework Vector Symbolic Architectures (also known as Hyperdimensional Computing). This framework is well suited for implementation in stochastic, emerging hardware and it naturally expresses the types of cognitive operations required for Artificial Intelligence (AI). We demonstrate in this article that the field-like algebraic structure of Vector Symbolic Architectures offers simple but powerful operations on high-dimensional vectors that can support all data structures and manipulations relevant to modern computing. In addition, we illustrate the distinguishing feature of Vector Symbolic Architectures, "computing in superposition," which sets it apart from conventional computing. It also opens the door to efficient solutions to the difficult combinatorial search problems inherent in AI applications. We sketch ways of demonstrating that Vector Symbolic Architectures are computationally universal. We see them acting as a framework for computing with distributed representations that can play a role of an abstraction layer for emerging computing hardware. This article serves as a reference for computer architects by illustrating the philosophy behind Vector Symbolic Architectures, techniques of distributed computing with them, and their relevance to emerging computing hardware, such as neuromorphic computing.

Density Encoding Enables Resource-Efficient Randomly Connected Neural Networks.

The deployment of machine learning algorithms on resource-constrained edge devices is an important challenge from both theoretical and applied points of view. In this brief, we focus on resource-efficient randomly connected neural networks known as random vector functional link (RVFL) networks since their simple design and extremely fast training time make them very attractive for solving many applied classification tasks. We propose to represent input features via the density-based encoding known in the area of stochastic computing and use the operations of binding and bundling from the area of hyperdimensional computing for obtaining the activations of the hidden neurons. Using a collection of 121 real-world data sets from the UCI machine learning repository, we empirically show that the proposed approach demonstrates higher average accuracy than the conventional RVFL. We also demonstrate that it is possible to represent the readout matrix using only integers in a limited range with minimal loss in the accuracy. In this case, the proposed approach operates only on small n -bits integers, which results in a computationally efficient architecture. Finally, through hardware field-programmable gate array (FPGA) implementations, we show that such an approach consumes approximately 11 times less energy than that of the conventional RVFL.

Brain structural correlates of insomnia severity in 1053 individuals with major depressive disorder: results from the ENIGMA MDD Working Group.

It has been difficult to find robust brain structural correlates of the overall severity of major depressive disorder (MDD). We hypothesized that specific symptoms may better reveal correlates and investigated this for the severity of insomnia, both a key symptom and a modifiable major risk factor of MDD. Cortical thickness, surface area and subcortical volumes were assessed from T1-weighted brain magnetic resonance imaging (MRI) scans of 1053 MDD patients (age range 13-79 years) from 15 cohorts within the ENIGMA MDD Working Group. Insomnia severity was measured by summing the insomnia items of the Hamilton Depression Rating Scale (HDRS). Symptom specificity was evaluated with correlates of overall depression severity. Disease specificity was evaluated in two independent samples comprising 2108 healthy controls, and in 260 clinical controls with bipolar disorder. Results showed that MDD patients with more severe insomnia had a smaller cortical surface area, mostly driven by the right insula, left inferior frontal gyrus pars triangularis, left frontal pole, right superior parietal cortex, right medial orbitofrontal cortex, and right supramarginal gyrus. Associations were specific for insomnia severity, and were not found for overall depression severity. Associations were also specific to MDD; healthy controls and clinical controls showed differential insomnia severity association profiles. The findings indicate that MDD patients with more severe insomnia show smaller surfaces in several frontoparietal cortical areas. While explained variance remains small, symptom-specific associations could bring us closer to clues on underlying biological phenomena of MDD.

Automatic Recognition of Sucker-Rod Pumping System Working Conditions Using Dynamometer Cards with Transfer Learning and SVM.

Sucker-rod pumping systems are the most widely applied artificial lift equipment in the oil and gas industry. Accurate and intelligent working condition recognition of pumping systems imposes major impacts on oilfield production benefits and efficiency. The shape of dynamometer card reflects the working conditions of sucker-rod pumping systems, and different conditions can be indicated by their typical card characteristics. In traditional identification methods, however, features are manually extracted based on specialist experience and domain knowledge. In this paper, an automatic fault diagnosis method is proposed to recognize the working conditions of sucker-rod pumping systems with massive dynamometer card data collected by sensors. Firstly, AlexNet-based transfer learning is adopted to automatically extract representative features from various dynamometer cards. Secondly, with the extracted features, error-correcting output codes model-based SVM is designed to identify the working conditions and improve the fault diagnosis accuracy and efficiency. The proposed AlexNet-SVM algorithm is validated against a real dataset from an oilfield. The results reveal that the proposed method reduces the need for human labor and improves the recognition accuracy.

A Comprehensive Study of Complexity and Performance of Automatic Detection of Atrial Fibrillation: Classification of Long ECG Recordings Based on the PhysioNet Computing in Cardiology Challenge 2017.

OBJECTIVE: The 2017 PhysioNet/CinC Challenge focused on automatic classification of atrial fibrillation (AF) in short ECGs. This study aimed to evaluate the use of the data and results from the challenge for detection of AF in longer ECGs, taken from three other PhysioNet datasets. APPROACH: The used data-driven models were based on features extracted from ECG recordings, calculated according to three solutions from the challenge. A Random Forest classifier was trained with the data from the challenge. The performance was evaluated on all non-overlapping 30 s segments in all recordings from three MIT-BIH datasets. Fifty-six models were trained using different feature sets, both before and after applying three feature reduction techniques. MAIN RESULTS: Based on rhythm annotations, the AF proportion was 0.00 in the MIT-BIH Normal Sinus Rhythm (N = 46083 segments), 0.10 in the MIT-BIH Arrhythmia (N = 2880), and 0.41 in the MIT-BIH Atrial Fibrillation (N = 28104) dataset. For the best performing model, the corresponding detected proportions of AF were 0.00, 0.11 and 0.36 using all features, and 0.01, 0.10 and 0.38 when using the 15 best performing features. SIGNIFICANCE: The results obtained on the MIT-BIH datasets indicate that the training data and solutions from the 2017 Physionet/Cinc Challenge can be useful tools for developing robust AF detectors also in longer ECG recordings, even when using a low number of carefully selected features. The use of feature selection allows significantly reducing the number of features while preserving the classification performance, which can be important when building low-complexity AF classifiers on ECG devices with constrained computational and energy resources.

White matter disturbances in major depressive disorder: a coordinated analysis across 20 international cohorts in the ENIGMA MDD working group.

Alterations in white matter (WM) microstructure have been implicated in the pathophysiology of major depressive disorder (MDD). However, previous findings have been inconsistent, partially due to low statistical power and the heterogeneity of depression. In the largest multi-site study to date, we examined WM anisotropy and diffusivity in 1305 MDD patients and 1602 healthy controls (age range 12-88 years) from 20 samples worldwide, which included both adults and adolescents, within the MDD Working Group of the Enhancing Neuroimaging Genetics through Meta-Analysis (ENIGMA) consortium. Processing of diffusion tensor imaging (DTI) data and statistical analyses were harmonized across sites and effects were meta-analyzed across studies. We observed subtle, but widespread, lower fractional anisotropy (FA) in adult MDD patients compared with controls in 16 out of 25 WM tracts of interest (Cohen's d between 0.12 and 0.26). The largest differences were observed in the corpus callosum and corona radiata. Widespread higher radial diffusivity (RD) was also observed (all Cohen's d between 0.12 and 0.18). Findings appeared to be driven by patients with recurrent MDD and an adult age of onset of depression. White matter microstructural differences in a smaller sample of adolescent MDD patients and controls did not survive correction for multiple testing. In this coordinated and harmonized multisite DTI study, we showed subtle, but widespread differences in WM microstructure in adult MDD, which may suggest structural disconnectivity in MDD.

No Alterations of Brain Structural Asymmetry in Major Depressive Disorder: An ENIGMA Consortium Analysis.

OBJECTIVE: Asymmetry is a subtle but pervasive aspect of the human brain, and it may be altered in several psychiatric conditions. MRI studies have shown subtle differences of brain anatomy between people with major depressive disorder and healthy control subjects, but few studies have specifically examined brain anatomical asymmetry in relation to this disorder, and results from those studies have remained inconclusive. At the functional level, some electroencephalography studies have indicated left fronto-cortical hypoactivity and right parietal hypoactivity in depressive disorders, so aspects of lateralized anatomy may also be affected. The authors used pooled individual-level data from data sets collected around the world to investigate differences in laterality in measures of cortical thickness, cortical surface area, and subcortical volume between individuals with major depression and healthy control subjects. METHODS: The authors investigated differences in the laterality of thickness and surface area measures of 34 cerebral cortical regions in 2,256 individuals with major depression and 3,504 control subjects from 31 separate data sets, and they investigated volume asymmetries of eight subcortical structures in 2,540 individuals with major depression and 4,230 control subjects from 32 data sets. T1-weighted MRI data were processed with a single protocol using FreeSurfer and the Desikan-Killiany atlas. The large sample size provided 80% power to detect effects of the order of Cohen's d=0.1. RESULTS: The largest effect size (Cohen's d) of major depression diagnosis was 0.085 for the thickness asymmetry of the superior temporal cortex, which was not significant after adjustment for multiple testing. Asymmetry measures were not significantly associated with medication use, acute compared with remitted status, first episode compared with recurrent status, or age at onset. CONCLUSIONS: Altered brain macro-anatomical asymmetry may be of little relevance to major depression etiology in most cases.

Machine learning to support social media empowered patients in cancer care and cancer treatment decisions.

BACKGROUND: A primary variant of social media, online support groups (OSG) extend beyond the standard definition to incorporate a dimension of advice, support and guidance for patients. OSG are complementary, yet significant adjunct to patient journeys. Machine learning and natural language processing techniques can be applied to these large volumes of unstructured text discussions accumulated in OSG for intelligent extraction of patient-reported demographics, behaviours, decisions, treatment, side effects and expressions of emotions. New insights from the fusion and synthesis of such diverse patient-reported information, as expressed throughout the patient journey from diagnosis to treatment and recovery, can contribute towards informed decision-making on personalized healthcare delivery and the development of healthcare policy guidelines. METHODS AND FINDINGS: We have designed and developed an artificial intelligence based analytics framework using machine learning and natural language processing techniques for intelligent analysis and automated aggregation of patient information and interaction trajectories in online support groups. Alongside the social interactions aspect, patient behaviours, decisions, demographics, clinical factors, emotions, as subsequently expressed over time, are extracted and analysed. More specifically, we utilised this platform to investigate the impact of online social influences on the intimate decision scenario of selecting a treatment type, recovery after treatment, side effects and emotions expressed over time, using prostate cancer as a model. Results manifest the three major decision-making behaviours among patients, Paternalistic group, Autonomous group and Shared group. Furthermore, each group demonstrated diverse behaviours in post-decision discussions on clinical outcomes, advice and expressions of emotion during the twelve months following treatment. Over time, the transition of patients from information and emotional support seeking behaviours to providers of information and emotional support to other patients was also observed. CONCLUSIONS: Findings from this study are a rigorous indication of the expectations of social media empowered patients, their potential for individualised decision-making, clinical and emotional needs. The increasing popularity of OSG further confirms that it is timely for clinicians to consider patient voices as expressed in OSG. We have successfully demonstrated that the proposed platform can be utilised to investigate, analyse and derive actionable insights from patient-reported information on prostate cancer, in support of patient focused healthcare delivery. The platform can be extended and applied just as effectively to any other medical condition.

Robust Methods for Automated Selection of Cardiac Signals After Blind Source Separation.

OBJECTIVE: Novel minimum-contact vital signs monitoring techniques like textile or capacitive electrocardiogram (ECG) provide new opportunities for health monitoring. These techniques are sensitive to artifacts and require handling of unstable signal quality. Spatio-temporal blind source separation (BSS) is capable of processing suchlike multichannel signals. However, BSS's permutation indeterminacy requires the selection of the cardiac signal (i.e., the component resembling the electric cardiac activity) after its separation from artifacts. This study evaluates different concepts for solving permutation indeterminacy. METHODS: Novel automated component selection routines based on heartbeat detections are compared with standard concepts, as using higher order moments or frequency-domain features, for solving permutation indeterminacy in spatio-temporal BSS. BSS was applied to a textile and a capacitive ECG dataset of healthy subjects performing a motion protocol, and to the MIT-BIH Arrhythmia Database. The performance of the subsequent component selection was evaluated by means of the heartbeat detection accuracy (ACC) using an automatically selected single component. RESULTS: The proposed heartbeat-detection-based selection routines significantly outperformed the standard selectors based on Skewness, Kurtosis, and frequency-domain features, especially for datasets containing motion artifacts. For arrhythmia data, beat analysis by sparse coding outperformed simple periodicity tests of the detected heartbeats. CONCLUSION: Component selection routines based on heartbeat detections are capable of reliably selecting cardiac signals after spatio-temporal BSS in case of severe motion artifacts and arrhythmia. SIGNIFICANCE: The availability of robust cardiac component selectors for solving permutation indeterminacy facilitates the usage of spatio-temporal BSS to extract cardiac signals in artifact-sensitive minimum-contact vital signs monitoring techniques.

Classification and Recall With Binary Hyperdimensional Computing: Tradeoffs in Choice of Density and Mapping Characteristics.

Hyperdimensional (HD) computing is a promising paradigm for future intelligent electronic appliances operating at low power. This paper discusses tradeoffs of selecting parameters of binary HD representations when applied to pattern recognition tasks. Particular design choices include density of representations and strategies for mapping data from the original representation. It is demonstrated that for the considered pattern recognition tasks (using synthetic and real-world data) both sparse and dense representations behave nearly identically. This paper also discusses implementation peculiarities which may favor one type of representations over the other. Finally, the capacity of representations of various densities is discussed.

Task-positive and task-negative networks in major depressive disorder: A combined fMRI and EEG study.

BACKGROUND: The study of intrinsic connectivity networks, i.e., sets of brain regions that show a high degree of interconnectedness even in the absence of a task, showed that major depressive disorder (MDD) patients demonstrate an increased connectivity within the default mode network (DMN), which is active in a resting state and is implicated in self-referential processing, and a decreased connectivity in task-positive networks (TPNs), which increase their activity in attention tasks. Cortical localization of this 'dominance' of the DMN over the TPN in MDD patients is not fully understood. Besides, this effect has been investigated using fMRI and its electrophysiological underpinning is not known. METHOD: In this study, we tested the dominance hypothesis using seed-based connectivity analysis of resting-state fMRI and EEG data obtained in 41 MDD patients and 23 controls. RESULTS: In MDD patients, as compared to controls, insula, pallidum/putamen, amygdala, and left dorso- and ventrolateral prefrontal cortex are more strongly connected with DMN than with TPN seeds. In EEG, all significant effects were obtained in the delta frequency band. LIMITATIONS: fMRI and EEG data were not obtained simultaneously during the same session. CONCLUSIONS: In MDD patients, major emotion and attention regulation circuits are more strongly connected with DMN than with TPN implying they are more prepared to respond to internally generated self-related thoughts than to environmental challenges.

Evolving stability and pH-dependent activity of the high redox potential Botrytis aclada laccase for enzymatic fuel cells.

Fungal high redox potential laccases are proposed as cathodic biocatalysts in implantable enzymatic fuel cells to generate high cell voltages. Their application is limited mainly through their acidic pH optimum and chloride inhibition. This work investigates evolutionary and engineering strategies to increase the pH optimum of a chloride-tolerant, high redox potential laccase from the ascomycete Botrytis aclada. The laccase was subjected to two rounds of directed evolution and the clones screened for increased stability and activity at pH 6.5. Beneficial mutation sites were investigated by semi-rational and combinatorial mutagenesis. Fourteen variants were characterised in detail to evaluate changes of the kinetic constants. Mutations increasing thermostability were distributed over the entire structure. Among them, T383I showed a 2.6-fold increased half-life by preventing the loss of the T2 copper through unfolding of a loop. Mutations affecting the pH-dependence cluster around the T1 copper and categorise in three types of altered pH profiles: pH-type I changes the monotonic decreasing pH profile into a bell-shaped profile, pH-type II describes increased specific activity below pH 6.5, and pH-type III increased specific activity above pH 6.5. Specific activities of the best variants were up to 5-fold higher (13 U mg-1) than BaL WT at pH 7.5.

Holographic Graph Neuron: A Bioinspired Architecture for Pattern Processing.

In this paper, we propose a new approach to implementing hierarchical graph neuron (HGN), an architecture for memorizing patterns of generic sensor stimuli, through the use of vector symbolic architectures. The adoption of a vector symbolic representation ensures a single-layer design while retaining the existing performance characteristics of HGN. This approach significantly improves the noise resistance of the HGN architecture, and enables a linear (with respect to the number of stored entries) time search for an arbitrary subpattern.

Symphony: a framework for accurate and holistic WSN simulation.

Research on wireless sensor networks has progressed rapidly over the last decade, and these technologies have been widely adopted for both industrial and domestic uses. Several operating systems have been developed, along with a multitude of network protocols for all layers of the communication stack. Industrial Wireless Sensor Network (WSN) systems must satisfy strict criteria and are typically more complex and larger in scale than domestic systems. Together with the non-deterministic behavior of network hardware in real settings, this greatly complicates the debugging and testing of WSN functionality. To facilitate the testing, validation, and debugging of large-scale WSN systems, we have developed a simulation framework that accurately reproduces the processes that occur inside real equipment, including both hardware- and software-induced delays. The core of the framework consists of a virtualized operating system and an emulated hardware platform that is integrated with the general purpose network simulator ns-3. Our framework enables the user to adjust the real code base as would be done in real deployments and also to test the boundary effects of different hardware components on the performance of distributed applications and protocols. Additionally we have developed a clock emulator with several different skew models and a component that handles sensory data feeds. The new framework should substantially shorten WSN application development cycles.

Effect of the L499M mutation of the ascomycetous Botrytis aclada laccase on redox potential and catalytic properties.

Laccases are members of a large family of multicopper oxidases that catalyze the oxidation of a wide range of organic and inorganic substrates accompanied by the reduction of dioxygen to water. These enzymes contain four Cu atoms per molecule organized into three sites: T1, T2 and T3. In all laccases, the T1 copper ion is coordinated by two histidines and one cysteine in the equatorial plane and is covered by the side chains of hydrophobic residues in the axial positions. The redox potential of the T1 copper ion influences the enzymatic reaction and is determined by the nature of the axial ligands and the structure of the second coordination sphere. In this work, the laccase from the ascomycete Botrytis aclada was studied, which contains conserved Ile491 and nonconserved Leu499 residues in the axial positions. The three-dimensional structures of the wild-type enzyme and the L499M mutant were determined by X-ray crystallography at 1.7 Šresolution. Crystals suitable for X-ray analysis could only be grown after deglycosylation. Both structures did not contain the T2 copper ion. The catalytic properties of the enzyme were characterized and the redox potentials of both enzyme forms were determined: E0 = 720 and 580 mV for the wild-type enzyme and the mutant, respectively. Since the structures of the wild-type and mutant forms are very similar, the change in the redox potential can be related to the L499M mutation in the T1 site of the enzyme.

Maestro: an orchestration framework for large-scale WSN simulations.

Contemporary wireless sensor networks (WSNs) have evolved into large and complex systems and are one of the main technologies used in cyber-physical systems and the Internet of Things. Extensive research on WSNs has led to the development of diverse solutions at all levels of software architecture, including protocol stacks for communications. This multitude of solutions is due to the limited computational power and restrictions on energy consumption that must be accounted for when designing typical WSN systems. It is therefore challenging to develop, test and validate even small WSN applications, and this process can easily consume significant resources. Simulations are inexpensive tools for testing, verifying and generally experimenting with new technologies in a repeatable fashion. Consequently, as the size of the systems to be tested increases, so does the need for large-scale simulations. This article describes a tool called Maestro for the automation of large-scale simulation and investigates the feasibility of using cloud computing facilities for such task. Using tools that are built into Maestro, we demonstrate a feasible approach for benchmarking cloud infrastructure in order to identify cloud Virtual Machine (VM)instances that provide an optimal balance of performance and cost for a given simulation.