Multiclass Classification of Visual Electroencephalogram Based on Channel Selection, Minimum Norm Estimation Algorithm, and Deep Network Architectures.

This work addresses the challenge of classifying multiclass visual EEG signals into 40 classes for brain-computer interface applications using deep learning architectures. The visual multiclass classification approach offers BCI applications a significant advantage since it allows the supervision of more than one BCI interaction, considering that each class label supervises a BCI task. However, because of the nonlinearity and nonstationarity of EEG signals, using multiclass classification based on EEG features remains a significant challenge for BCI systems. In the present work, mutual information-based discriminant channel selection and minimum-norm estimate algorithms were implemented to select discriminant channels and enhance the EEG data. Hence, deep EEGNet and convolutional recurrent neural networks were separately implemented to classify the EEG data for image visualization into 40 labels. Using the k-fold cross-validation approach, average classification accuracies of 94.8% and 89.8% were obtained by implementing the aforementioned network architectures. The satisfactory results obtained with this method offer a new implementation opportunity for multitask embedded BCI applications utilizing a reduced number of both channels (<50%) and network parameters (<110 K).

Learning smooth dendrite morphological neurons by stochastic gradient descent for pattern classification.

This article presents a learning algorithm for dendrite morphological neurons (DMN) based on stochastic gradient descent (SGD). In particular, we focus on a DMN topology that comprises spherical dendrites, smooth maximum activation function nodes, and a softmax output layer, whose original learning algorithm is performed in two independent stages: (1) dendrites' centroids are learned by k-means, and (2) softmax layer weights are adjusted by gradient descent. A drawback of this learning method is that both stages are unplugged; once dendrites' centroids are defined, they keep static during weights learning, so no feedback is performed to correct the dendrites' positions to improve classification performance. To overcome this issue, we derive the delta rules for adjusting the dendrites' centroids and the output layer weights by minimizing the cross-entropy loss function under an SGD scheme. This gradient descent-based learning is feasible because the smooth maximum activation function that interfaces the dendrite units with the output layer is differentiable. The proposed DMN is compared against eight morphological neuron models with distinct topologies and learning methods and four well-established classifiers: support vector machine (SVM), multilayer perceptron (MLP), and random forest (RF), and k-nearest neighbors (k-NN). Besides, the classification performance is evaluated on 81 datasets. The experimental results show that the proposed method tends to outperform the DMN methods and is competitive or even better than SVM, MLP, RF, and k-NN. Thus, it is an alternative approach that can effectively be used for pattern classification. Moreover, SGD for DMN learning standardizes this neural model, like current artificial neural networks.

Improving the Classification Performance of Dendrite Morphological Neurons.

Dendrite morphological neurons (DMNs) are neural models for pattern classification, where dendrites are represented by a geometric shape enclosing patterns of the same class. This study evaluates the impact of three dendrite geometries-namely, box, ellipse, and sphere-on pattern classification. In addition, we propose using smooth maximum and minimum functions to reduce the coarseness of decision boundaries generated by typical DMNs, and a softmax layer is attached at the DMN output to provide posterior probabilities from weighted dendrites responses. To adjust the number of dendrites per class automatically, a tuning algorithm based on an incremental-decremental procedure is introduced. The classification performance assessment is conducted on nine synthetic and 49 real-world datasets. Meanwhile, 12 DMN variants are evaluated in terms of accuracy and model complexity. The DMN reaches its highest potential by combining spherical dendrites with smooth activation functions and a learnable softmax layer. It attained the highest accuracy, uses the simplest geometric shape, is insensitive to variables with zero variance, and its structural complexity diminishes by using the smooth maximum function. Furthermore, this DMN configuration performed competitively or even better than other well-established classifiers in terms of accuracy, such as support vector machine, multilayer perceptron, radial basis function network, k -nearest neighbors, and random forest. Thus, the proposed DMN is an attractive alternative for pattern classification in real-world problems.

Implementation of Kalman Filtering with Spiking Neural Networks.

A Kalman filter can be used to fill space-state reconstruction dynamics based on knowledge of a system and partial measurements. However, its performance relies on accurate modeling of the system dynamics and a proper characterization of the uncertainties, which can be hard to obtain in real-life scenarios. In this work, we explore how the values of a Kalman gain matrix can be estimated by using spiking neural networks through a combination of biologically plausible neuron models with spike-time-dependent plasticity learning algorithms. The performance of proposed neural architecture is verified with simulations of some representative nonlinear systems, which show promising results. This approach traces a path for its implementation in neuromorphic analog hardware that can learn and reconstruct partial and changing dynamics of a system without the massive power consumption that is typically needed in a Von Neumann-based computer architecture.

Dynamic balance of a bipedal robot using neural network training with simulated annealing.

This work proposes using an evolutionary optimization method known as simulated annealing to train artificial neural networks. These neural networks are used to control posture stabilization of a humanoid robot in a simulation. A total of eight multilayer perceptron neural networks are used. Although the control is used mainly for posture stabilization and not displacement, we propose a posture set to achieve this, including right leg lift in sagittal plane and right leg lift in frontal plane. At the beginning, tests are carried out only considering gravitational force and reaction force between the floor and the humanoid; then tests are carried out with two disturbances: tilted ground and adding a mass to the humanoid. We found that using simulated annealing the robot maintains its stability at all times, decreasing the number of epochs needed to converge, and also, showing flexibility and adaptability to disturbances. The way neural networks learn is analyzed; videos of the movements made, and the model for further experimentation are provided.

R-STDP Spiking Neural Network Architecture for Motion Control on a Changing Friction Joint Robotic Arm.

Neuromorphic computing is a recent class of brain-inspired high-performance computer platforms and algorithms involving biologically-inspired models adopting hardware implementation in integrated circuits. The neuromorphic computing applications have provoked the rise of highly connected neurons and synapses in analog circuit systems that can be used to solve today's challenging machine learning problems. In conjunction with biologically plausible learning rules, such as the Hebbian learning and memristive devices, biologically-inspired spiking neural networks are considered the next-generation neuromorphic hardware construction blocks that will enable the deployment of new analog in situ learning capable and energetic efficient brain-like devices. These features are envisioned for modern mobile robotic implementations, currently challenging to overcome the pervasive von Neumann computer architecture. This study proposes a new neural architecture using the spike-time-dependent plasticity learning method and step-forward encoding algorithm for a self tuning neural control of motion in a joint robotic arm subjected to dynamic modifications. Simulations were conducted to demonstrate the proposed neural architecture's feasibility as the network successfully compensates for changing dynamics at each simulation run.

Neuromorphic Signal Filter for Robot Sensoring.

Noise management associated with input signals in sensor devices arises as one of the main problems limiting robot control performance. This article introduces a novel neuromorphic filter model based on a leaky integrate and fire (LIF) neural model cell, which encodes the primary information from a noisy input signal and delivers an output signal with a significant noise reduction in practically real-time with energy-efficient consumption. A new approach for neural decoding based on the neuron-cell spiking frequency is introduced to recover the primary signal information. The simulations conducted on the neuromorphic filter demonstrate an outstanding performance of white noise rejecting while preserving the original noiseless signal with a low information loss. The proposed filter model is compatible with the CMOS technology design methodologies for implementing low consumption smart sensors with applications in various fields such as robotics and the automotive industry demanded by Industry 4.0.

Multi-subject classification of Motor Imagery EEG signals using transfer learning in neural networks.

Brain-Computer Interfaces are new technologies with a fast development due to their possible usages, which still require overcoming some challenges to be readily usable. The paradigm of motor imagery is among the ones in these types of systems where the pipeline is tuned to work with only one person as it fails to classify the signals of a different person. Deep Learning methods have been gaining attention for tasks involving high-dimensional unstructured data, like EEG signals, but fail to generalize when trained on small datasets. In this work, to acquire a benchmark, we evaluate the performance of several classifiers while decoding signals from a new subject using a leave-one-out approach. Then we test the classifiers on the previous experiment and a method based on transfer learning in neural networks to classify the signals of multiple persons at a time. The resulting neural network classifier achieves a classification accuracy of 73% on the evaluation sessions of four subjects at a time and 74% on three at a time on the BCI competition IV 2a dataset.

Pixel-Wise Classification in Hippocampus Histological Images.

This paper presents a method for pixel-wise classification applied for the first time on hippocampus histological images. The goal is achieved by representing pixels in a 14-D vector, composed of grey-level information and moment invariants. Then, several popular machine learning models are used to categorize them, and multiple metrics are computed to evaluate the performance of the different models. The multilayer perceptron, random forest, support vector machine, and radial basis function networks were compared, achieving the multilayer perceptron model the highest result on accuracy metric, AUC, and F 1 score with highly satisfactory results for substituting a manual classification task, due to an expert opinion in the hippocampus histological images.

Smooth dendrite morphological neurons.

A typical feature of hyperbox-based dendrite morphological neurons (DMN) is the generation of sharp and rough decision boundaries that inaccurately track the distribution shape of classes of patterns. This feature is because the minimum and maximum activation functions force the decision boundaries to match the faces of the hyperboxes. To improve the DMN response, we introduce a dendritic model that uses smooth maximum and minimum functions to soften the decision boundaries. The classification performance assessment is conducted on nine synthetic and 28 real-world datasets. Based on the experimental results, we demonstrate that the smooth activation functions improve the generalization capacity of DMN. The proposed approach is competitive with four machine learning techniques, namely, Multilayer Perceptron, Radial Basis Function Network, Support Vector Machine, and Nearest Neighbor algorithm. Besides, the computational complexity of DMN training is lower than MLP and SVM classifiers.

CNN Based Detectors on Planetary Environments: A Performance Evaluation.

An essential characteristic that an exploration robot must possess is to be autonomous. This is necessary because it will usually do its task in remote or hard-to-reach places. One of the primary elements of a navigation system is the information that can be acquired by the sensors of the environment in which it will operate. For this reason, an algorithm based on convolutional neural networks is proposed for the detection of rocks in environments similar to Mars. The methodology proposed here is based on the use of a Single-Shot-Detector (SSD) network architecture, which has been modified to evaluate the performance. The main contribution of this study is to provide an alternative methodology to detect rocks in planetary images because most of the previous works only focus on classification problems and used handmade feature vectors.

Expert knowledge for the recognition of leukemic cells.

This work shows the advantage of expert knowledge for leukemic cell recognition. In the medical area, visual analysis of microscopic images has regularly used biological samples to recognize hematological disorders. Nowadays, techniques of image recognition are needed to achieve an adequate identification of blood tissues. This paper presents a procedure to acquire expert knowledge from blood cell images. We apply Gaussian mixtures, evolutionary computing, and standard techniques of image processing to extract knowledge. This information feeds a support vector machine or multilayer perceptron to classify healthy or leukemic cells. Additionally, convolutional neural networks are used as a benchmark to compare our proposed method with the state of the art. We use a public database of 260 healthy and leukemic cell images. Results show that our traditional pattern recognition methodology matches deep learning accuracy since the recognition of blood cells achieves 99.63%, whereas the convolutional neural networks reach 97.74% on average. Moreover, the computational effort of our approach is minimal, while meeting the requirement of being explainable.

Novel Mathematical Model of Breast Cancer Diagnostics Using an Associative Pattern Classification.

Breast cancer is a disease that has emerged as the second leading cause of cancer deaths in women worldwide. The annual mortality rate is estimated to continue growing. Cancer detection at an early stage could significantly reduce breast cancer death rates long-term. Many investigators have studied different breast diagnostic approaches, such as mammography, magnetic resonance imaging, ultrasound, computerized tomography, positron emission tomography and biopsy. However, these techniques have limitations, such as being expensive, time consuming and not suitable for women of all ages. Proposing techniques that support the effective medical diagnosis of this disease has undoubtedly become a priority for the government, for health institutions and for civil society in general. In this paper, an associative pattern classifier (APC) was used for the diagnosis of breast cancer. The rate of efficiency obtained on the Wisconsin breast cancer database was 97.31%. The APC's performance was compared with the performance of a support vector machine (SVM) model, back-propagation neural networks, C4.5, naive Bayes, k-nearest neighbor (k-NN) and minimum distance classifiers. According to our results, the APC performed best. The algorithm of the APC was written and executed in a JAVA platform, as well as the experimental and comparativeness between algorithms.

On the accuracy and computational cost of spiking neuron implementation.

Since more than a decade ago, three statements about spiking neuron (SN) implementations have been widely accepted: 1) Hodgkin and Huxley (HH) model is computationally prohibitive, 2) Izhikevich (IZH) artificial neuron is as efficient as Leaky Integrate-and-Fire (LIF) model, and 3) IZH model is more efficient than HH model (Izhikevich, 2004). As suggested by Hodgkin and Huxley (1952), their model operates in two modes: by using the α's and ß's rate functions directly (HH model) and by storing them into tables (HHT model) for computational cost reduction. Recently, it has been stated that: 1) HHT model (HH using tables) is not prohibitive, 2) IZH model is not efficient, and 3) both HHT and IZH models are comparable in computational cost (Skocik & Long, 2014). That controversy shows that there is no consensus concerning SN simulation capacities. Hence, in this work, we introduce a refined approach, based on the multiobjective optimization theory, describing the SN simulation capacities and ultimately choosing optimal simulation parameters. We have used normalized metrics to define the capacity levels of accuracy, computational cost, and efficiency. Normalized metrics allowed comparisons between SNs at the same level or scale. We conducted tests for balanced, lower, and upper boundary conditions under a regular spiking mode with constant and random current stimuli. We found optimal simulation parameters leading to a balance between computational cost and accuracy. Importantly, and, in general, we found that 1) HH model (without using tables) is the most accurate, computationally inexpensive, and efficient, 2) IZH model is the most expensive and inefficient, 3) both LIF and HHT models are the most inaccurate, 4) HHT model is more expensive and inaccurate than HH model due to α's and ß's table discretization, and 5) HHT model is not comparable in computational cost to IZH model. These results refute the theory formulated over a decade ago (Izhikevich, 2004) and go more in-depth in the statements formulated by Skocik and Long (2014). Our statements imply that the number of dimensions or FLOPS in the SNs are theoretical but not practical indicators of the true computational cost. The metric we propose for the computational cost is more precise than FLOPS and was found to be invariant to computer architecture. Moreover, we found that the firing frequency used in previous works is a necessary but an insufficient metric to evaluate the simulation accuracy. We also show that our results are consistent with the theory of numerical methods and the theory of SN discontinuity. Discontinuous SNs, such LIF and IZH models, introduce a considerable error every time a spike is generated. In addition, compared to the constant input current, the random input current increases the computational cost and inaccuracy. Besides, we found that the search for optimal simulation parameters is problem-specific. That is important because most of the previous works have intended to find a general and unique optimal simulation. Here, we show that this solution could not exist because it is a multiobjective optimization problem that depends on several factors. This work sets up a renewed thesis concerning the SN simulation that is useful to several related research areas, including the emergent Deep Spiking Neural Networks.

Quantitative evaluation of binary digital region asymmetry with application to skin lesion detection.

BACKGROUND: The performance of Computer Aided Diagnosis Systems for early melanoma detection relies mainly on quantitative evaluation of the geometric features corresponding to skin lesions. In these systems, diagnosis is carried out by analyzing four geometric characteristics: asymmetry (A), border (B), color (C) and dimension (D). The main objective of this study is to establish an algorithm for the measurement of asymmetry in biological entities. METHODS: Binary digital images corresponding to lesions are divided into 8 segments from their centroid. For each segment, the discrete compactness value is calculated using Normalized E-Factor (NEF). The asymmetry value is obtained from the sum of the square difference of each NEF value and corresponding value of its opposite by the vertex. Two public skin cancer databases were used. 1) Lee's database with 40 digital regions evaluated by fourteen dermatologists. 2) The PH2 database which consists of 200 images in an 8-bit RGB format. This database provides a pre-classification of asymmetry carried out by experts, and it also indicates if the lesion is a melanoma. RESULTS: The measure was applied using two skin lesion image databases. 1) In Lee's database, Spearman test provided a value of 0.82 between diagnosis of dermatologists and asymmetry values. For the 12 binary images most likely to be melanoma, the correlation between the measurement and dermatologists was 0.98. 2) In the PH2 database a label is provided for each binary image where the type of asymmetry is indicated. Class 0-1 corresponds to symmetry and one axis of symmetry shapes, the completely asymmetrical were assigned to Class 2, the values of sensitivity and specificity were 59.62 and 85.8% respectively between the asymmetry measured by a group of dermatologists and the proposed algorithm. CONCLUSIONS: Simple image digital features such as compactness can be used to quantify the asymmetry of a skin lesion using its digital binary image representation. This measure is stable taking into account translations, rotations, scale changes and can be applied to non-convex regions, including areas with holes.

Retinal vessel extraction using Lattice Neural Networks with Dendritic Processing.

Retinal images can be used to detect and follow up several important chronic diseases. The classification of retinal images requires an experienced ophthalmologist. This has been a bottleneck to implement routine screenings performed by general physicians. It has been proposed to create automated systems that can perform such task with little intervention from humans, with partial success. In this work, we report advances in such endeavor, by using a Lattice Neural Network with Dendritic Processing (LNNDP). We report results using several metrics, and compare against well known methods such as Support Vector Machines (SVM) and Multilayer Perceptrons (MLP). Our proposal shows better performance than other approaches reported in the literature. An additional advantage is that unlike those other tools, LNNDP requires no parameters, and it automatically constructs its structure to solve a particular problem. The proposed methodology requires four steps: (1) Pre-processing, (2) Feature computation, (3) Classification and (4) Post-processing. The Hotelling T(2) control chart was used to reduce the dimensionality of the feature vector, from 7 that were used before to 5 in this work. The experiments were run on images of DRIVE and STARE databases. The results show that on average, F1-Score is better in LNNDP, compared with SVM and MLP implementations. Same improvement is observed for MCC and the accuracy.