Unsupervised Anomaly Detection in Stream Data with Online Evolving Spiking Neural Networks.

Unsupervised anomaly discovery in stream data is a research topic with many practical applications. However, in many cases, it is not easy to collect enough training data with labeled anomalies for supervised learning of an anomaly detector in order to deploy it later for identification of real anomalies in streaming data. It is thus important to design anomalies detectors that can correctly detect anomalies without access to labeled training data. Our idea is to adapt the Online evolving Spiking Neural Network (OeSNN) classifier to the anomaly detection task. As a result, we offer an Online evolving Spiking Neural Network for Unsupervised Anomaly Detection algorithm (OeSNN-UAD), which, unlike OeSNN, works in an unsupervised way and does not separate output neurons into disjoint decision classes. OeSNN-UAD uses our proposed new two-step anomaly detection method. Also, we derive new theoretical properties of neuronal model and input layer encoding of OeSNN, which enable more effective and efficient detection of anomalies in our OeSNN-UAD approach. The proposed OeSNN-UAD detector was experimentally compared with state-of-the-art unsupervised and semi-supervised detectors of anomalies in stream data from the Numenta Anomaly Benchmark and Yahoo Anomaly Datasets repositories. Our approach outperforms the other solutions provided in the literature in the case of data streams from the Numenta Anomaly Benchmark repository. Also, in the case of real data files of the Yahoo Anomaly Benchmark repository, OeSNN-UAD outperforms other selected algorithms, whereas in the case of Yahoo Anomaly Benchmark synthetic data files, it provides competitive results to the results recently reported in the literature.

Incremental Unsupervised Domain-Adversarial Training of Neural Networks.

In the context of supervised statistical learning, it is typically assumed that the training set comes from the same distribution that draws the test samples. When this is not the case, the behavior of the learned model is unpredictable and becomes dependent upon the degree of similarity between the distribution of the training set and the distribution of the test set. One of the research topics that investigates this scenario is referred to as domain adaptation (DA). Deep neural networks brought dramatic advances in pattern recognition and that is why there have been many attempts to provide good DA algorithms for these models. Herein we take a different avenue and approach the problem from an incremental point of view, where the model is adapted to the new domain iteratively. We make use of an existing unsupervised domain-adaptation algorithm to identify the target samples on which there is greater confidence about their true label. The output of the model is analyzed in different ways to determine the candidate samples. The selected samples are then added to the source training set by self-labeling, and the process is repeated until all target samples are labeled. This approach implements a form of adversarial training in which, by moving the self-labeled samples from the target to the source set, the DA algorithm is forced to look for new features after each iteration. Our results report a clear improvement with respect to the non-incremental case in several data sets, also outperforming other state-of-the-art DA algorithms.

Graph transform learning.

Transform learning is a new representation learning framework where we learn an operator/transform that analyses the data to generate the coefficient/representation. We propose a variant of it called the graph transform learning; in this we explicitly account for the correlation in the dataset in terms of graph Laplacian. We will give two variants; in the first one the graph is computed from the data and fixed during the operation. In the second, the graph is learnt iteratively from the data during operation. The first technique will be applied for clustering, and the second one for solving inverse problems.

DART: Domain-Adversarial Residual-Transfer networks for unsupervised cross-domain image classification.

The accuracy of deep learning (e.g., convolutional neural networks) for an image classification task critically relies on the amount of labeled training data. Aiming to solve an image classification task on a new domain that lacks labeled data but gains access to cheaply available unlabeled data, unsupervised domain adaptation is a promising technique to boost the performance without incurring extra labeling cost, by assuming images from different domains share some invariant characteristics. In this paper, we propose a new unsupervised domain adaptation method named Domain-Adversarial Residual-Transfer (DART) learning of deep neural networks to tackle cross-domain image classification tasks. In contrast to the existing unsupervised domain adaption approaches, the proposed DART not only learns domain-invariant features via adversarial training, but also achieves robust domain-adaptive classification via a residual-transfer strategy, all in an end-to-end training framework. We evaluate the performance of the proposed method for cross-domain image classification tasks on several well-known benchmark data sets, in which our method clearly outperforms the state-of-the-art approaches.

Generative Adversarial Networks are special cases of Artificial Curiosity (1990) and also closely related to Predictability Minimization (1991).

I review unsupervised or self-supervised neural networks playing minimax games in game-theoretic settings: (i) Artificial Curiosity (AC, 1990) is based on two such networks. One network learns to generate a probability distribution over outputs, the other learns to predict effects of the outputs. Each network minimizes the objective function maximized by the other. (ii) Generative Adversarial Networks (GANs, 2010-2014) are an application of AC where the effect of an output is 1 if the output is in a given set, and 0 otherwise. (iii) Predictability Minimization (PM, 1990s) models data distributions through a neural encoder that maximizes the objective function minimized by a neural predictor of the code components. I correct a previously published claim that PM is not based on a minimax game.

Looking to the future: Learning from experience, averting catastrophe.

As humans go through life sifting vast quantities of complex information, we extract knowledge from settings that are more ambiguous than our early homes and classrooms. Learning from experience in an individual's unique context generally improves expert performance, despite the risks inherent in brain dynamics that can transform previously reliable expectations. Designers of twenty-first century technologies face the challenges and responsibilities posed by fielded systems that continue to learn on their own. The neural model Self-supervised ART, which can acquire significantly new knowledge in unpredictable contexts, is an example of one such system.

ICU staffing feature phenotypes and their relationship with patients' outcomes: an unsupervised machine learning analysis.

PURPOSE: To study whether ICU staffing features are associated with improved hospital mortality, ICU length of stay (LOS) and duration of mechanical ventilation (MV) using cluster analysis directed by machine learning. METHODS: The following variables were included in the analysis: average bed to nurse, physiotherapist and physician ratios, presence of 24/7 board-certified intensivists and dedicated pharmacists in the ICU, and nurse and physiotherapist autonomy scores. Clusters were defined using the partition around medoids method. We assessed the association between clusters and hospital mortality using logistic regression and with ICU LOS and MV duration using competing risk regression. RESULTS: Analysis included data from 129,680 patients admitted to 93 ICUs (2014-2015). Three clusters were identified. The features distinguishing between the clusters were: the presence of board-certified intensivists in the ICU 24/7 (present in Cluster 3), dedicated pharmacists (present in Clusters 2 and 3) and the extent of nurse autonomy (which increased from Clusters 1 to 3). The patients in Cluster 3 exhibited the best outcomes, with lower adjusted hospital mortality [odds ratio 0.92 (95% confidence interval (CI), 0.87-0.98)], shorter ICU LOS [subhazard ratio (SHR) for patients surviving to ICU discharge 1.24 (95% CI 1.22-1.26)] and shorter durations of MV [SHR for undergoing extubation 1.61(95% CI 1.54-1.69)]. Cluster 1 had the worst outcomes. CONCLUSION: Patients treated in ICUs combining 24/7 expert intensivist coverage, a dedicated pharmacist and nurses with greater autonomy had the best outcomes. All of these features represent achievable targets that should be considered by policy makers with an interest in promoting equal and optimal ICU care.

Unsupervised robust discriminative manifold embedding with self-expressiveness.

Dimensionality reduction has obtained increasing attention in the machine learning and computer vision communities due to the curse of dimensionality. Many manifold embedding methods have been proposed for dimensionality reduction. Many of them are supervised and based on graph regularization whose weight affinity is determined by original noiseless data. When data are noisy, their performance may degrade. To address this issue, we present a novel unsupervised robust discriminative manifold embedding approach called URDME, which aims to offer a joint framework of dimensionality reduction, discriminative subspace learning , robust affinity representation and discriminative manifold embedding. The learned robust affinity not only captures the global geometry and intrinsic structure of underlying high-dimensional data, but also satisfies the self-expressiveness property. In addition, the learned projection matrix owns discriminative ability in the low-dimensional subspace. Experimental results on several public benchmark datasets corroborate the effectiveness of our approach and show its competitive performance compared with the related methods.

Deep associative neural network for associative memory based on unsupervised representation learning.

This paper presents a deep associative neural network (DANN) based on unsupervised representation learning for associative memory. In brain, the knowledge is learnt by associating different types of sensory data, such as image and voice. The associative memory models which imitate such a learning process have been studied for decades but with simpler architectures they fail to deal with large scale complex data as compared with deep neural networks. Therefore, we define a deep architecture consisting of a perception layer and hierarchical propagation layers. To learn the network parameters, we define a probabilistic model for the whole network inspired from unsupervised representation learning models. The model is optimized by a modified contrastive divergence algorithm with a novel iterated sampling process. After training, given a new data or corrupted data, the correct label or corrupted part is associated by the network. The DANN is able to achieve many machine learning problems, including not only classification, but also depicting the data given a label and recovering corrupted images. Experiments on MNIST digits and CIFAR-10 datasets demonstrate the learning capability of the proposed DANN.

Artificial intelligence in the diagnosis of cardiovascular disease.

Artificial intelligence (AI) is a field of computer science that aims to mimic human thought processes. AI techniques have been applied in cardiovascular medicine to explore novel genotypes and phenotypes in existing diseases, improve the quality of patient care, enabling cost-effectiveness, and reducing readmission and mortality rates. The potential of AI in cardiovascular medicine is tremendous; however, ignorance of the challenges may overshadow its potential clinical impact. This paper gives a glimpse of AI's application in cardiovascular clinical care and discusses its potential role in facilitating precision cardiovascular medicine.

A density-based competitive data stream clustering network with self-adaptive distance metric.

Data stream clustering is a branch of clustering where patterns are processed as an ordered sequence. In this paper, we propose an unsupervised learning neural network named Density Based Self Organizing Incremental Neural Network(DenSOINN) for data stream clustering tasks. DenSOINN is a self organizing competitive network that grows incrementally to learn suitable nodes to fit the distribution of learning data, combining online unsupervised learning and topology learning by means of competitive Hebbian learning rule. By adopting a density-based clustering mechanism, DenSOINN discovers arbitrarily shaped clusters and diminishes the negative effect of noise. In addition, we adopt a self-adaptive distance framework to obtain good performance for learning unnormalized input data. Experiments show that the DenSOINN can achieve high standard performance comparing to state-of-the-art methods.

Unsupervised Machine Learning to Identify High Likelihood of Dementia in Population-Based Surveys: Development and Validation Study.

BACKGROUND: Dementia is increasing in prevalence worldwide, yet frequently remains undiagnosed, especially in low- and middle-income countries. Population-based surveys represent an underinvestigated source to identify individuals at risk of dementia. OBJECTIVE: The aim is to identify participants with high likelihood of dementia in population-based surveys without the need of the clinical diagnosis of dementia in a subsample. METHODS: Unsupervised machine learning classification (hierarchical clustering on principal components) was developed in the Health and Retirement Study (HRS; 2002-2003, N=18,165 individuals) and validated in the Survey of Health, Ageing and Retirement in Europe (SHARE; 2010-2012, N=58,202 individuals). RESULTS: Unsupervised machine learning classification identified three clusters in HRS: cluster 1 (n=12,231) without any functional or motor limitations, cluster 2 (N=4841) with walking/climbing limitations, and cluster 3 (N=1093) with both functional and walking/climbing limitations. Comparison of cluster 3 with previously published predicted probabilities of dementia in HRS showed that it identified high likelihood of dementia (probability of dementia >0.95; area under the curve [AUC]=0.91). Removing either cognitive or both cognitive and behavioral measures did not impede accurate classification (AUC=0.91 and AUC=0.90, respectively). Three clusters with similar profiles were identified in SHARE (cluster 1: n=40,223; cluster 2: n=15,644; cluster 3: n=2335). Survival rate of participants from cluster 3 reached 39.2% (n=665 deceased) in HRS and 62.2% (n=811 deceased) in SHARE after a 3.9-year follow-up. Surviving participants from cluster 3 in both cohorts worsened their functional and mobility performance over the same period. CONCLUSIONS: Unsupervised machine learning identifies high likelihood of dementia in population-based surveys, even without cognitive and behavioral measures and without the need of clinical diagnosis of dementia in a subsample of the population. This method could be used to tackle the global challenge of dementia.

Unsupervised heart-rate estimation in wearables with Liquid states and a probabilistic readout.

Heart-rate estimation is a fundamental feature of modern wearable devices. In this paper we propose a machine learning technique to estimate heart-rate from electrocardiogram (ECG) data collected using wearable devices. The novelty of our approach lies in (1) encoding spatio-temporal properties of ECG signals directly into spike train and using this to excite recurrently connected spiking neurons in a Liquid State Machine computation model; (2) a novel learning algorithm; and (3) an intelligently designed unsupervised readout based on Fuzzy c-Means clustering of spike responses from a subset of neurons (Liquid states), selected using particle swarm optimization. Our approach differs from existing works by learning directly from ECG signals (allowing personalization), without requiring costly data annotations. Additionally, our approach can be easily implemented on state-of-the-art spiking-based neuromorphic systems, offering high accuracy, yet significantly low energy footprint, leading to an extended battery-life of wearable devices. We validated our approach with CARLsim, a GPU accelerated spiking neural network simulator modeling Izhikevich spiking neurons with Spike Timing Dependent Plasticity (STDP) and homeostatic scaling. A range of subjects is considered from in-house clinical trials and public ECG databases. Results show high accuracy and low energy footprint in heart-rate estimation across subjects with and without cardiac irregularities, signifying the strong potential of this approach to be integrated in future wearable devices.