Optimizing echo state networks for continuous gesture recognition in mobile devices: A comparative study.

Continuous gesture recognition can be used to enhance human-computer interaction. This can be accomplished by capturing human movement with the use of the Inertial Measurement Units in smartphones and using machine learning algorithms to predict the intended gestures. Echo State Networks (ESNs) consist of a fixed internal reservoir that is able to generate rich and diverse nonlinear dynamics in response to input signals that capture temporal dependencies within the signal. This makes ESNs well-suited for time series prediction tasks, such as continuous gesture recognition. However, their application has not been rigorously explored, with regard to gesture recognition. In this study, we sought to enhance the efficacy of ESN models in continuous gesture recognition by exploring diverse model structures, fine-tuning hyperparameters, and experimenting with various training approaches. We used three different training schemes that used the Leave-one-out Cross-validation (LOOCV) protocol to investigate the performance in real-world scenarios with different levels of data availability: Leaving out data from one user to use for testing (F1-score: 0.89), leaving out a fraction of data from all users to use in testing (F1-score: 0.96), and training and testing using LOOCV on a single user (F1-score: 0.99). The obtained results outperformed the Long Short-Term Memory (LSTM) performance from past research (F1-score: 0.87) while maintaining a low training time of approximately 13 seconds compared to 63 seconds for the LSTM model. Additionally, we further explored the performance of the ESN models through behaviour space analysis using memory capacity, Kernel Rank, and Generalization Rank. Our results demonstrate that ESNs can be optimized to achieve high performance on gesture recognition in mobile devices on multiple levels of data availability. These findings highlight the practical ability of ESNs to enhance human-computer interaction.

A novel online multi-task learning for COVID-19 multi-output spatio-temporal prediction.

In light of the ongoing COVID-19 pandemic, predicting its trend would significantly impact decision-making. However, this is not a straightforward task due to three main difficulties: temporal autocorrelation, spatial dependency, and concept drift caused by virus mutations and lockdown policies. Although machine learning has been extensively used in related work, no previous research has successfully addressed all three challenges simultaneously. To overcome this challenge, we developed a novel online multi-task regression algorithm that incorporates a chain structure to capture spatial dependency, the ADWIN drift detector to adapt to concept drift, and the lag time series feature to capture temporal autocorrelation. We conducted several comparative experiments based on the number of daily confirmed cases in 20 areas in California and affiliated cities. The results from our experiments demonstrate that our proposed model is superior in adapting to concept drift in COVID-19 data and capturing spatial dependencies across various regions. This leads to a significant improvement in prediction accuracy when compared to existing state-of-the-art batch machine learning methods, such as N-Beats, DeepAR, TCN, and LSTM.

The accuracy of international and national fetal growth charts in detecting small-for-gestational-age infants using the Lambda-Mu-Sigma method.

Objective: To construct a national fetal growth chart using retrospective data and compared its diagnostic accuracy in predicting SGA at birth with existing international growth charts. Method: This is a retrospective study where datasets from May 2011 to Apr 2020 were extracted to construct the fetal growth chart using the Lambda-Mu-Sigma method. SGA is defined as birth weight <10th centile. The local growth chart's diagnostic accuracy in detecting SGA at birth was evaluated using datasets from May 2020 to Apr 2021 and was compared with the WHO, Hadlock, and INTERGROWTH-21st charts. Balanced accuracy, sensitivity, and specificity were reported. Results: A total of 68,897 scans were collected and five biometric growth charts were constructed. Our national growth chart achieved an accuracy of 69% and a sensitivity of 42% in identifying SGA at birth. The WHO chart showed similar diagnostic performance as our national growth chart, followed by the Hadlock (67% accuracy and 38% sensitivity) and INTERGROWTH-21st (57% accuracy and 19% sensitivity). The specificities for all charts were 95-96%. All growth charts showed higher accuracy in the third trimester, with an improvement of 8-16%, as compared to that in the second trimester. Conclusion: Using the Hadlock and INTERGROWTH-21st chart in the Malaysian population may results in misdiagnose of SGA. Our population local chart has slightly higher accuracy in predicting preterm SGA in the second trimester which can enable earlier intervention for babies who are detected as SGA. All growth charts' diagnostic accuracies were poor in the second trimester, suggesting the need of improvising alternative techniques for early detection of SGA to improve fetus outcomes.

Multi-Label Classification via Adaptive Resonance Theory-Based Clustering.

This article proposes a multi-label classification algorithm capable of continual learning by applying an Adaptive Resonance Theory (ART)-based clustering algorithm and the Bayesian approach for label probability computation. The ART-based clustering algorithm adaptively and continually generates prototype nodes corresponding to given data, and the generated nodes are used as classifiers. The label probability computation independently counts the number of label appearances for each class and calculates the Bayesian probabilities. Thus, the label probability computation can cope with an increase in the number of labels. Experimental results with synthetic and real-world multi-label datasets show that the proposed algorithm has competitive classification performance to other well-known algorithms while realizing continual learning.

Bidirectional parallel echo state network for speech emotion recognition.

Speech is an effective way for communicating and exchanging complex information between humans. Speech signal has involved a great attention in human-computer interaction. Therefore, emotion recognition from speech has become a hot research topic in the field of interacting machines with humans. In this paper, we proposed a novel speech emotion recognition system by adopting multivariate time series handcrafted feature representation from speech signals. Bidirectional echo state network with two parallel reservoir layers has been applied to capture additional independent information. The parallel reservoirs produce multiple representations for each direction from the bidirectional data with two stages of concatenation. The sparse random projection approach has been adopted to reduce the high-dimensional sparse output for each direction separately from both reservoirs. Random over-sampling and random under-sampling methods are used to overcome the imbalanced nature of the used speech emotion datasets. The performance of the proposed parallel ESN model is evaluated from the speaker-independent experiments on EMO-DB, SAVEE, RAVDESS, and FAU Aibo datasets. The results show that the proposed SER model is superior to the single reservoir and the state-of-the-art studies.

On the Post Hoc Explainability of Optimized Self-Organizing Reservoir Network for Action Recognition.

This work proposes a novel unsupervised self-organizing network, called the Self-Organizing Convolutional Echo State Network (SO-ConvESN), for learning node centroids and interconnectivity maps compatible with the deterministic initialization of Echo State Network (ESN) input and reservoir weights, in the context of human action recognition (HAR). To ensure stability and echo state property in the reservoir, Recurrent Plots (RPs) and Recurrence Quantification Analysis (RQA) techniques are exploited for explainability and characterization of the reservoir dynamics and hence tuning ESN hyperparameters. The optimized self-organizing reservoirs are cascaded with a Convolutional Neural Network (CNN) to ensure that the activation of internal echo state representations (ESRs) echoes similar topological qualities and temporal features of the input time-series, and the CNN efficiently learns the dynamics and multiscale temporal features from the ESRs for action recognition. The hyperparameter optimization (HPO) algorithms are additionally adopted to optimize the CNN stage in SO-ConvESN. Experimental results on the HAR problem using several publicly available 3D-skeleton-based action datasets demonstrate the showcasing of the RPs and RQA technique in examining the explainability of reservoir dynamics for designing stable self-organizing reservoirs and the usefulness of implementing HPOs in SO-ConvESN for the HAR task. The proposed SO-ConvESN exhibits competitive recognition accuracy.

A Comprehensive Survey of Image-Based Food Recognition and Volume Estimation Methods for Dietary Assessment.

Dietary studies showed that dietary problems such as obesity are associated with other chronic diseases, including hypertension, irregular blood sugar levels, and increased risk of heart attacks. The primary cause of these problems is poor lifestyle choices and unhealthy dietary habits, which are manageable using interactive mHealth apps. However, traditional dietary monitoring systems using manual food logging suffer from imprecision, underreporting, time consumption, and low adherence. Recent dietary monitoring systems tackle these challenges by automatic assessment of dietary intake through machine learning methods. This survey discusses the best-performing methodologies that have been developed so far for automatic food recognition and volume estimation. Firstly, the paper presented the rationale of visual-based methods for food recognition. Then, the core of the study is the presentation, discussion, and evaluation of these methods based on popular food image databases. In this context, this study discusses the mobile applications that are implementing these methods for automatic food logging. Our findings indicate that around 66.7% of surveyed studies use visual features from deep neural networks for food recognition. Similarly, all surveyed studies employed a variant of convolutional neural networks (CNN) for ingredient recognition due to recent research interest. Finally, this survey ends with a discussion of potential applications of food image analysis, existing research gaps, and open issues of this research area. Learning from unlabeled image datasets in an unsupervised manner, catastrophic forgetting during continual learning, and improving model transparency using explainable AI are potential areas of interest for future studies.

Explainable deep learning ensemble for food image analysis on edge devices.

Food recognition systems recently garnered much research attention in the relevant field due to their ability to obtain objective measurements for dietary intake. This feature contributes to the management of various chronic conditions. Challenges such as inter and intraclass variations alongside the practical applications of smart glasses, wearable cameras, and mobile devices require resource-efficient food recognition models with high classification performance. Furthermore, explainable AI is also crucial in health-related domains as it characterizes model performance, enhancing its transparency and objectivity. Our proposed architecture attempts to address these challenges by drawing on the strengths of the transfer learning technique upon initializing MobiletNetV3 with weights from a pre-trained model of ImageNet. The MobileNetV3 achieves superior performance using the squeeze and excitation strategy, providing unequal weight to different input channels and contrasting equal weights in other variants. Despite being fast and efficient, there is a high possibility for it to be stuck in the local optima like other deep neural networks, reducing the desired classification performance of the model. Thus, we overcome this issue by applying the snapshot ensemble approach as it enables the M model in a single training process without any increase in the required training time. As a result, each snapshot in the ensemble visits different local minima before converging to the final solution which enhances recognition performance. On overcoming the challenge of explainability, we argue that explanations cannot be monolithic, since each stakeholder perceive the results', explanations based on different objectives and aims. Thus, we proposed a user-centered explainable artificial intelligence (AI) framework to increase the trust of the involved parties by inferencing and rationalizing the results according to needs and user profile. Our framework is comprehensive in terms of a dietary assessment app as it detects Food/Non-Food, food categories, and ingredients. Experimental results on the standard food benchmarks and newly contributed Malaysian food dataset for ingredient detection demonstrated superior performance on an integrated set of measures over other methodologies.

A Kernel Bayesian Adaptive Resonance Theory with A Topological Structure.

This paper attempts to solve the typical problems of self-organizing growing network models, i.e. (a) an influence of the order of input data on the self-organizing ability, (b) an instability to high-dimensional data and an excessive sensitivity to noise, and (c) an expensive computational cost by integrating Kernel Bayes Rule (KBR) and Correntropy-Induced Metric (CIM) into Adaptive Resonance Theory (ART) framework. KBR performs a covariance-free Bayesian computation which is able to maintain a fast and stable computation. CIM is a generalized similarity measurement which can maintain a high-noise reduction ability even in a high-dimensional space. In addition, a Growing Neural Gas (GNG)-based topology construction process is integrated into the ART framework to enhance its self-organizing ability. The simulation experiments with synthetic and real-world datasets show that the proposed model has an outstanding stable self-organizing ability for various test environments.

SLAMM: Visual monocular SLAM with continuous mapping using multiple maps.

This paper presents the concept of Simultaneous Localization and Multi-Mapping (SLAMM). It is a system that ensures continuous mapping and information preservation despite failures in tracking due to corrupted frames or sensor's malfunction; making it suitable for real-world applications. It works with single or multiple robots. In a single robot scenario the algorithm generates a new map at the time of tracking failure, and later it merges maps at the event of loop closure. Similarly, maps generated from multiple robots are merged without prior knowledge of their relative poses; which makes this algorithm flexible. The system works in real time at frame-rate speed. The proposed approach was tested on the KITTI and TUM RGB-D public datasets and it showed superior results compared to the state-of-the-arts in calibrated visual monocular keyframe-based SLAM. The mean tracking time is around 22 milliseconds. The initialization is twice as fast as it is in ORB-SLAM, and the retrieved map can reach up to 90 percent more in terms of information preservation depending on tracking loss and loop closure events. For the benefit of the community, the source code along with a framework to be run with Bebop drone are made available at https://github.com/hdaoud/ORBSLAMM.

Quantum-Inspired Multidirectional Associative Memory With a Self-Convergent Iterative Learning.

Quantum-inspired computing is an emerging research area, which has significantly improved the capabilities of conventional algorithms. In general, quantum-inspired hopfield associative memory (QHAM) has demonstrated quantum information processing in neural structures. This has resulted in an exponential increase in storage capacity while explaining the extensive memory, and it has the potential to illustrate the dynamics of neurons in the human brain when viewed from quantum mechanics perspective although the application of QHAM is limited as an autoassociation. We introduce a quantum-inspired multidirectional associative memory (QMAM) with a one-shot learning model, and QMAM with a self-convergent iterative learning model (IQMAM) based on QHAM in this paper. The self-convergent iterative learning enables the network to progressively develop a resonance state, from inputs to outputs. The simulation experiments demonstrate the advantages of QMAM and IQMAM, especially the stability to recall reliability.

Kernel Bayesian ART and ARTMAP.

Adaptive Resonance Theory (ART) is one of the successful approaches to resolving "the plasticity-stability dilemma" in neural networks, and its supervised learning model called ARTMAP is a powerful tool for classification. Among several improvements, such as Fuzzy or Gaussian based models, the state of art model is Bayesian based one, while solving the drawbacks of others. However, it is known that the Bayesian approach for the high dimensional and a large number of data requires high computational cost, and the covariance matrix in likelihood becomes unstable. This paper introduces Kernel Bayesian ART (KBA) and ARTMAP (KBAM) by integrating Kernel Bayes' Rule (KBR) and Correntropy Induced Metric (CIM) to Bayesian ART (BA) and ARTMAP (BAM), respectively, while maintaining the properties of BA and BAM. The kernel frameworks in KBA and KBAM are able to avoid the curse of dimensionality. In addition, the covariance-free Bayesian computation by KBR provides the efficient and stable computational capability to KBA and KBAM. Furthermore, Correntropy-based similarity measurement allows improving the noise reduction ability even in the high dimensional space. The simulation experiments show that KBA performs an outstanding self-organizing capability than BA, and KBAM provides the superior classification ability than BAM, respectively.

Developmental Approach for Behavior Learning Using Primitive Motion Skills.

Imitation learning through self-exploration is essential in developing sensorimotor skills. Most developmental theories emphasize that social interactions, especially understanding of observed actions, could be first achieved through imitation, yet the discussion on the origin of primitive imitative abilities is often neglected, referring instead to the possibility of its innateness. This paper presents a developmental model of imitation learning based on the hypothesis that humanoid robot acquires imitative abilities as induced by sensorimotor associative learning through self-exploration. In designing such learning system, several key issues will be addressed: automatic segmentation of the observed actions into motion primitives using raw images acquired from the camera without requiring any kinematic model; incremental learning of spatio-temporal motion sequences to dynamically generates a topological structure in a self-stabilizing manner; organization of the learned data for easy and efficient retrieval using a dynamic associative memory; and utilizing segmented motion primitives to generate complex behavior by the combining these motion primitives. In our experiment, the self-posture is acquired through observing the image of its own body posture while performing the action in front of a mirror through body babbling. The complete architecture was evaluated by simulation and real robot experiments performed on DARwIn-OP humanoid robot.

Correction: View-Invariant Visuomotor Processing in Computational Mirror Neuron System for Humanoid.

[This corrects the article DOI: 10.1371/journal.pone.0152003.].

View-Invariant Visuomotor Processing in Computational Mirror Neuron System for Humanoid.

Mirror neurons are visuo-motor neurons found in primates and thought to be significant for imitation learning. The proposition that mirror neurons result from associative learning while the neonate observes his own actions has received noteworthy empirical support. Self-exploration is regarded as a procedure by which infants become perceptually observant to their own body and engage in a perceptual communication with themselves. We assume that crude sense of self is the prerequisite for social interaction. However, the contribution of mirror neurons in encoding the perspective from which the motor acts of others are seen have not been addressed in relation to humanoid robots. In this paper we present a computational model for development of mirror neuron system for humanoid based on the hypothesis that infants acquire MNS by sensorimotor associative learning through self-exploration capable of sustaining early imitation skills. The purpose of our proposed model is to take into account the view-dependency of neurons as a probable outcome of the associative connectivity between motor and visual information. In our experiment, a humanoid robot stands in front of a mirror (represented through self-image using camera) in order to obtain the associative relationship between his own motor generated actions and his own visual body-image. In the learning process the network first forms mapping from each motor representation onto visual representation from the self-exploratory perspective. Afterwards, the representation of the motor commands is learned to be associated with all possible visual perspectives. The complete architecture was evaluated by simulation experiments performed on DARwIn-OP humanoid robot.

What influences patients' acceptance of a blood pressure telemonitoring service in primary care? A qualitative study.

BACKGROUND: Telemonitoring of home blood pressure (BP) is found to have a positive effect on BP control. Delivering a BP telemonitoring service in primary care offers primary care physicians an innovative approach toward management of their patients with hypertension. However, little is known about patients' acceptance of such service in routine clinical care. OBJECTIVE: This study aimed to explore patients' acceptance of a BP telemonitoring service delivered in primary care based on the technology acceptance model (TAM). METHODS: A qualitative study design was used. Primary care patients with uncontrolled office BP who fulfilled the inclusion criteria were enrolled into a BP telemonitoring service offered between the period August 2012 and September 2012. This service was delivered at an urban primary care clinic in Kuala Lumpur, Malaysia. Twenty patients used the BP telemonitoring service. Of these, 17 patients consented to share their views and experiences through five in-depth interviews and two focus group discussions. An interview guide was developed based on the TAM. The interviews were audio-recorded and transcribed verbatim. Thematic analysis was used for analysis. RESULTS: Patients found the BP telemonitoring service easy to use but struggled with the perceived usefulness of doing so. They expressed confusion in making sense of the monitored home BP readings. They often thought about the implications of these readings to their hypertension management and overall health. Patients wanted more feedback from their doctors and suggested improvement to the BP telemonitoring functionalities to improve interactions. Patients cited being involved in research as the main reason for their intention to use the service. They felt that patients with limited experience with the internet and information technology, who worked out of town, or who had an outdoor hobby would not be able to benefit from such a service. CONCLUSION: Patients found BP telemonitoring service in primary care easy to use but needed help to interpret the meanings of monitored BP readings. Implementations of BP telemonitoring service must tackle these issues to maximize the patients' acceptance of a BP telemonitoring service.

Classifying Stress From Heart Rate Variability Using Salivary Biomarkers as Reference.

An accurate and noninvasive stress assessment from human physiology is a strenuous task. In this paper, a pattern recognition system to learn complex correlates between heart rate variability (HRV) features and salivary stress biomarkers is proposed. Using the Trier social stress test, heart rate and salivary measurements were obtained from volunteers under varying levels of stress induction. Measurements of salivary alpha-amylase and cortisol were used as objective measures of stress, and were correlated with the HRV features using fuzzy ARTMAP (FAM). In improving the predictive ability of the ARTMAPs, techniques, such as genetic algorithms for parameter optimization and voting ensembles, were employed. The ensemble of FAMs can be used for predicting stress responses of salivary alpha-amylase or cortisol using heart rate measurements as the input. Using alpha-amylase as the stress indicator, the ensemble was able to classify stress from heart rate features with 75% accuracy, and 80% accuracy when cortisol was used.

Power Quality Analysis Using a Hybrid Model of the Fuzzy Min-Max Neural Network and Clustering Tree.

A hybrid intelligent model comprising a modified fuzzy min-max (FMM) clustering neural network and a modified clustering tree (CT) is developed. A review of clustering models with rule extraction capabilities is presented. The hybrid FMM-CT model is explained. We first use several benchmark problems to illustrate the cluster evolution patterns from the proposed modifications in FMM. Then, we employ a case study with real data related to power quality monitoring to assess the usefulness of FMM-CT. The results are compared with those from other clustering models. More importantly, we extract explanatory rules from FMM-CT to justify its predictions. The empirical findings indicate the usefulness of the proposed model in tackling data clustering and power quality monitoring problems under different environments.

Classification of Implantable Rotary Blood Pump States With Class Noise.

A medical case study related to implantable rotary blood pumps is examined. Five classifiers and two ensemble classifiers are applied to process the signals collected from the pumps for the identification of the aortic valve nonopening pump state. In addition to the noise-free datasets, up to 40% class noise has been added to the signals to evaluate the classification performance when mislabeling is present in the classifier training set. In order to ensure a reliable diagnostic model for the identification of the pump states, classifications performed with and without class noise are evaluated. The multilayer perceptron emerged as the best performing classifier for pump state detection due to its high accuracy as well as robustness against class noise.

Comparative study on interaction of form and motion processing streams by applying two different classifiers in mechanism for recognition of biological movement.

Research on psychophysics, neurophysiology, and functional imaging shows particular representation of biological movements which contains two pathways. The visual perception of biological movements formed through the visual system called dorsal and ventral processing streams. Ventral processing stream is associated with the form information extraction; on the other hand, dorsal processing stream provides motion information. Active basic model (ABM) as hierarchical representation of the human object had revealed novelty in form pathway due to applying Gabor based supervised object recognition method. It creates more biological plausibility along with similarity with original model. Fuzzy inference system is used for motion pattern information in motion pathway creating more robustness in recognition process. Besides, interaction of these paths is intriguing and many studies in various fields considered it. Here, the interaction of the pathways to get more appropriated results has been investigated. Extreme learning machine (ELM) has been implied for classification unit of this model, due to having the main properties of artificial neural networks, but crosses from the difficulty of training time substantially diminished in it. Here, there will be a comparison between two different configurations, interactions using synergetic neural network and ELM, in terms of accuracy and compatibility.