Unraveling Brain Synchronisation Dynamics by Explainable Neural Networks using EEG Signals: Application to Dyslexia Diagnosis.

The electrical activity of the neural processes involved in cognitive functions is captured in EEG signals, allowing the exploration of the integration and coordination of neuronal oscillations across multiple spatiotemporal scales. We have proposed a novel approach that combines the transformation of EEG signal into image sequences, considering cross-frequency phase synchronisation (CFS) dynamics involved in low-level auditory processing, with the development of a two-stage deep learning model for the detection of developmental dyslexia (DD). This deep learning model exploits spatial and temporal information preserved in the image sequences to find discriminative patterns of phase synchronisation over time achieving a balanced accuracy of up to 83%. This result supports the existence of differential brain synchronisation dynamics between typical and dyslexic seven-year-old readers. Furthermore, we have obtained interpretable representations using a novel feature mask to link the most relevant regions during classification with the cognitive processes attributed to normal reading and those corresponding to compensatory mechanisms found in dyslexia.

Artificial Intelligence for Emerging Technology in Surgery: Systematic Review and Validation.

Surgery is a high-risk procedure of therapy and is associated to post trauma complications of longer hospital stay, estimated blood loss and long duration of surgeries. Reports have suggested that over 2.5% patients die during and post operation. This paper is aimed at systematic review of previous research on artificial intelligence (AI) in surgery, analyzing their results with suitable software to validate their research by obtaining same or contrary results. Six published research articles have been reviewed across three continents. These articles have been re-validated using software including SPSS and MedCalc to obtain the statistical features such as the mean, standard deviation, significant level, and standard error. From the significant values, the experiments are then classified according to the null (p < 0.05) or alternative (p>0.05) hypotheses. The results obtained from the analysis have suggested significant difference in operating time, docking time, staging time, and estimated blood loss but show no significant difference in length of hospital stay, recovery time and lymph nodes harvested between robotic assisted surgery using AI and normal conventional surgery. From the evaluations, this research suggests that AI-assisted surgery improves over the conventional surgery as safer and more efficient system of surgery with minimal or no complications.

Improving Inertial Sensor-Based Activity Recognition in Neurological Populations.

Inertial sensor-based human activity recognition (HAR) has a range of healthcare applications as it can indicate the overall health status or functional capabilities of people with impaired mobility. Typically, artificial intelligence models achieve high recognition accuracies when trained with rich and diverse inertial datasets. However, obtaining such datasets may not be feasible in neurological populations due to, e.g., impaired patient mobility to perform many daily activities. This study proposes a novel framework to overcome the challenge of creating rich and diverse datasets for HAR in neurological populations. The framework produces images from numerical inertial time-series data (initial state) and then artificially augments the number of produced images (enhanced state) to achieve a larger dataset. Here, we used convolutional neural network (CNN) architectures by utilizing image input. In addition, CNN enables transfer learning which enables limited datasets to benefit from models that are trained with big data. Initially, two benchmarked public datasets were used to verify the framework. Afterward, the approach was tested in limited local datasets of healthy subjects (HS), Parkinson's disease (PD) population, and stroke survivors (SS) to further investigate validity. The experimental results show that when data augmentation is applied, recognition accuracies have been increased in HS, SS, and PD by 25.6%, 21.4%, and 5.8%, respectively, compared to the no data augmentation state. In addition, data augmentation contributes to better detection of stair ascent and stair descent by 39.1% and 18.0%, respectively, in limited local datasets. Findings also suggest that CNN architectures that have a small number of deep layers can achieve high accuracy. The implication of this study has the potential to reduce the burden on participants and researchers where limited datasets are accrued.

Exploring human activity recognition using feature level fusion of inertial and electromyography data.

Wearables are objective tools for human activity recognition (HAR). Advances in wearables enable synchronized multi-sensing within a single device. This has resulted in studies investigating the use of single or multiple wearable sensor modalities for HAR. Some studies use inertial data, others use surface electromyography (sEMG) from multiple muscles and different post-processing approaches. Yet, questions remain about accuracies relating to e.g., multi-modal approaches, and sEMG post-processing. Here, we explored how inertial and sEMG could be efficiently combined with machine learning and used with post-processing methods for better HAR. This study aims recognition of four basic daily life activities; walking, standing, stair ascent and descent. Firstly, we created a new feature vector based on the domain knowledge gained from previous mobility studies. Then, a feature level data fusion approach was used to combine inertial and sEMG data. Finally, two supervised learning classifiers (Support Vector Machine, SVM, and the k-Nearest Neighbors, kNN) were tested with 5-fold cross-validation. Results show the use of inertial data with sEMG increased overall accuracy by 3.5% (SVM) and 6.3% (kNN). Extracting features from linear envelopes instead of bandpass filtered sEMG improves overall HAR accuracy in both classifiers. Clinical Relevance- Post-processing on sEMG signals can improve the performance of multimodal HAR.

Is the use of deep learning an appropriate means to locate debris in the ocean without harming aquatic wildlife?

With the global issue of marine debris ever expanding, it is imperative that the technology industry steps in. The aim is to find if deep learning can successfully distinguish between marine life and synthetic debris underwater. This study assesses whether we could safely clean up our oceans with Artificial Intelligence without disrupting the delicate balance of aquatic ecosystems. Our research compares a simple convolutional neural network with a VGG-16 model using an original database of 1644 underwater images and a binary classification to sort synthetic material from aquatic life. Our results show first insights to safely distinguishing between debris and life.

Non-Intrusive Fish Weight Estimation in Turbid Water Using Deep Learning and Regression Models.

Underwater fish monitoring is the one of the most challenging problems for efficiently feeding and harvesting fish, while still being environmentally friendly. The proposed 2D computer vision method is aimed at non-intrusively estimating the weight of Tilapia fish in turbid water environments. Additionally, the proposed method avoids the issue of using high-cost stereo cameras and instead uses only a low-cost video camera to observe the underwater life through a single channel recording. An in-house curated Tilapia-image dataset and Tilapia-file dataset with various ages of Tilapia are used. The proposed method consists of a Tilapia detection step and Tilapia weight-estimation step. A Mask Recurrent-Convolutional Neural Network model is first trained for detecting and extracting the image dimensions (i.e., in terms of image pixels) of the fish. Secondly, is the Tilapia weight-estimation step, wherein the proposed method estimates the depth of the fish in the tanks and then converts the Tilapia's extracted image dimensions from pixels to centimeters. Subsequently, the Tilapia's weight is estimated by a trained model based on regression learning. Linear regression, random forest regression, and support vector regression have been developed to determine the best models for weight estimation. The achieved experimental results have demonstrated that the proposed method yields a Mean Absolute Error of 42.54 g, R2 of 0.70, and an average weight error of 30.30 (±23.09) grams in a turbid water environment, respectively, which show the practicality of the proposed framework.

O-Net: A Fast and Precise Deep-Learning Architecture for Computational Super-Resolved Phase-Modulated Optical Microscopy.

We present a fast and precise deep-learning architecture, which we term O-Net, for obtaining super-resolved images from conventional phase-modulated optical microscopical techniques, such as phase-contrast microscopy and differential interference contrast microscopy. O-Net represents a novel deep convolutional neural network that can be trained on both simulated and experimental data, the latter of which is being demonstrated in the present context. The present study demonstrates the ability of the proposed method to achieve super-resolved images even under poor signal-to-noise ratios and does not require prior information on the point spread function or optical character of the system. Moreover, unlike previous state-of-the-art deep neural networks (such as U-Nets), the O-Net architecture seemingly demonstrates an immunity to network hallucination, a commonly cited issue caused by network overfitting when U-Nets are employed. Models derived from the proposed O-Net architecture are validated through empirical comparison with a similar sample imaged via scanning electron microscopy (SEM) and are found to generate ultra-resolved images which came close to that of the actual SEM micrograph.

A Pose-Based Feature Fusion and Classification Framework for the Early Prediction of Cerebral Palsy in Infants.

The early diagnosis of cerebral palsy is an area which has recently seen significant multi-disciplinary research. Diagnostic tools such as the General Movements Assessment (GMA), have produced some very promising results. However, the prospect of automating these processes may improve accessibility of the assessment and also enhance the understanding of movement development of infants. Previous works have established the viability of using pose-based features extracted from RGB video sequences to undertake classification of infant body movements based upon the GMA. In this paper, we propose a series of new and improved features, and a feature fusion pipeline for this classification task. We also introduce the RVI-38 dataset, a series of videos captured as part of routine clinical care. By utilising this challenging dataset we establish the robustness of several motion features for classification, subsequently informing the design of our proposed feature fusion framework based upon the GMA. We evaluate our proposed framework's classification performance using both the RVI-38 dataset and the publicly available MINI-RGBD dataset. We also implement several other methods from the literature for direct comparison using these two independent datasets. Our experimental results and feature analysis show that our proposed pose-based method performs well across both datasets. The proposed features afford us the opportunity to include finer detail than previous methods, and further model GMA specific body movements. These new features also allow us to take advantage of additional body-part specific information as a means of improving the overall classification performance, whilst retaining GMA relevant, interpretable, and shareable features.

Developing and exploring a methodology for multi-modal indoor and outdoor gait assessment.

Gait assessment is emerging as a prominent way to understand impaired mobility and underlying neurological deficits. Various technologies have been used to assess gait inside and outside of laboratory settings, but wearables are the preferred option due to their cost-effective and practical use in both. There are robust conceptual gait models developed to ease the interpretation of gait parameters during indoor and outdoor environments. However, these models examine uni-modal gait characteristics (e.g., spatio-temporal parameters) only. Previous studies reported that understanding the underlying reason for impaired gait requires multi-modal gait assessment. Therefore, this study aims to develop a multi-modal approach using a synchronized inertial and electromyography (EMG) signals. Firstly, initial contact (IC), final contact (FC) moments and corresponding time stamps were identified from inertial data, producing temporal outcomes e.g., step time. Secondly, IC/FC time stamps were used to segment EMG data and define onset and offset times of muscle activities within the gait cycle and its subphases. For investigation purposes, we observed notable differences in temporal characteristics as well as muscle onset/offset timings and amplitudes between indoor and outdoor walking of three stroke survivors. Our preliminary analysis suggests a multi-modal approach may be important to augment and improve current inertial conceptual gait models by providing additional quantitative EMG data.

Wearable Inertial Gait Algorithms: Impact of Wear Location and Environment in Healthy and Parkinson's Populations.

Wearable inertial measurement units (IMUs) are used in gait analysis due to their discrete wearable attachment and long data recording possibilities within indoor and outdoor environments. Previously, lower back and shin/shank-based IMU algorithms detecting initial and final contact events (ICs-FCs) were developed and validated on a limited number of healthy young adults (YA), reporting that both IMU wear locations are suitable to use during indoor and outdoor gait analysis. However, the impact of age (e.g., older adults, OA), pathology (e.g., Parkinson's Disease, PD) and/or environment (e.g., indoor vs. outdoor) on algorithm accuracy have not been fully investigated. Here, we examined IMU gait data from 128 participants (72-YA, 20-OA, and 36-PD) to thoroughly investigate the suitability of ICs-FCs detection algorithms (1 × lower back and 1 × shin/shank-based) for quantifying temporal gait characteristics depending on IMU wear location and walking environment. The level of agreement between algorithms was investigated for different cohorts and walking environments. Although mean temporal characteristics from both algorithms were significantly correlated for all groups and environments, subtle but characteristically nuanced differences were observed between cohorts and environments. The lowest absolute agreement level was observed in PD (ICC2,1 = 0.979, 0.806, 0.730, 0.980) whereas highest in YA (ICC2,1 = 0.987, 0.936, 0.909, 0.989) for mean stride, stance, swing, and step times, respectively. Absolute agreement during treadmill walking (ICC2,1 = 0.975, 0.914, 0.684, 0.945), indoor walking (ICC2,1 = 0.987, 0.936, 0.909, 0.989) and outdoor walking (ICC2,1 = 0.998, 0.940, 0.856, 0.998) was found for mean stride, stance, swing, and step times, respectively. Findings of this study suggest that agreements between algorithms are sensitive to the target cohort and environment. Therefore, researchers/clinicians should be cautious while interpreting temporal parameters that are extracted from inertial sensors-based algorithms especially for those with a neurological condition.

Automated Landslide-Risk Prediction Using Web GIS and Machine Learning Models.

Spatial susceptible landslide prediction is the one of the most challenging research areas which essentially concerns the safety of inhabitants. The novel geographic information web (GIW) application is proposed for dynamically predicting landslide risk in Chiang Rai, Thailand. The automated GIW system is coordinated between machine learning technologies, web technologies, and application programming interfaces (APIs). The new bidirectional long short-term memory (Bi-LSTM) algorithm is presented to forecast landslides. The proposed algorithm consists of 3 major steps, the first of which is the construction of a landslide dataset by using Quantum GIS (QGIS). The second step is to generate the landslide-risk model based on machine learning approaches. Finally, the automated landslide-risk visualization illustrates the likelihood of landslide via Google Maps on the website. Four static factors are considered for landslide-risk prediction, namely, land cover, soil properties, elevation and slope, and a single dynamic factor i.e., precipitation. Data are collected to construct a geospatial landslide database which comprises three historical landslide locations-Phu Chifa at Thoeng District, Ban Pha Duea at Mae Salong Nai, and Mai Salong Nok in Mae Fa Luang District, Chiang Rai, Thailand. Data collection is achieved using QGIS software to interpolate contour, elevation, slope degree and land cover from the Google satellite images, aerial and site survey photographs while the physiographic and rock type are on-site surveyed by experts. The state-of-the-art machine learning models have been trained i.e., linear regression (LR), artificial neural network (ANN), LSTM, and Bi-LSTM. Ablation studies have been conducted to determine the optimal parameters setting for each model. An enhancement method based on two-stage classifications has been presented to improve the landslide prediction of LSTM and Bi-LSTM models. The landslide-risk prediction performances of these models are subsequently evaluated using real-time dataset and it is shown that Bi-LSTM with Random Forest (Bi-LSTM-RF) yields the best prediction performance. Bi-LSTM-RF model has improved the landslide-risk predicting performance over LR, ANNs, LSTM, and Bi-LSTM in terms of the area under the receiver characteristic operator (AUC) scores by 0.42, 0.27, 0.46, and 0.47, respectively. Finally, an automated web GIS has been developed and it consists of software components including the trained models, rainfall API, Google API, and geodatabase. All components have been interfaced together via JavaScript and Node.js tool.

Deep Temporal Convolution Network for Time Series Classification.

A neural network that matches with a complex data function is likely to boost the classification performance as it is able to learn the useful aspect of the highly varying data. In this work, the temporal context of the time series data is chosen as the useful aspect of the data that is passed through the network for learning. By exploiting the compositional locality of the time series data at each level of the network, shift-invariant features can be extracted layer by layer at different time scales. The temporal context is made available to the deeper layers of the network by a set of data processing operations based on the concatenation operation. A matching learning algorithm for the revised network is described in this paper. It uses gradient routing in the backpropagation path. The framework as proposed in this work attains better generalization without overfitting the network to the data, as the weights can be pretrained appropriately. It can be used end-to-end with multivariate time series data in their raw form, without the need for manual feature crafting or data transformation. Data experiments with electroencephalogram signals and human activity signals show that with the right amount of concatenation in the deeper layers of the proposed network, it can improve the performance in signal classification.

Optimization of Fuzzy Energy-Management System for Grid-Connected Microgrid Using NSGA-II.

This article proposes a fuzzy logic-based energy-management system (FEMS) for a grid-connected microgrid with renewable energy sources (RESs) and energy storage system (ESS). The objectives of the FEMS are reducing the average peak load (APL) and operating cost through arbitrage operation of the ESS. These objectives are achieved by controlling the charge and discharge rate of the ESS based on the state of charge of ESS, the power difference between load and RES, and electricity market price. The effectiveness of the fuzzy logic greatly depends on the membership functions (MFs). The fuzzy MFs of the FEMS are optimized offline using a Pareto-based multiobjective evolutionary algorithm, nondominated sorting genetic algorithm (NSGA-II). The best compromise solution is selected as the final solution and implemented in the fuzzy-logic controller. A comparison with other control strategies with similar objectives is carried out at a simulation level. The proposed FEMS is experimentally validated on a real microgrid in the energy storage test bed at Newcastle University, U.K.

A Lightweight Spatial and Temporal Multi-Feature Fusion Network for Defect Detection.

This article proposes a hybrid multi-dimensional features fusion structure of spatial and temporal segmentation model for automated thermography defects detection. In addition, the newly designed attention block encourages local interaction among the neighboring pixels to recalibrate the feature maps adaptively. A Sequence-PCA layer is embedded in the network to provide enhanced semantic information. The final model results in a lightweight structure with smaller number of parameters and yet yields uncompromising performance after model compression. The proposed model allows better capture of the semantic information to improve the detection rate in an end-to-end procedure. Compared with current state-of-the-art deep semantic segmentation algorithms, the proposed model presents more accurate and robust results. In addition, the proposed attention module has led to improved performance on two classification tasks compared with other prevalent attention blocks. In order to verify the effectiveness and robustness of the proposed model, experimental studies have been carried out for defects detection on four different datasets. The demo code of the proposed method can be linked soon: http://faculty.uestc.edu.cn/gaobin/zh_CN/lwcg/153392/list/index.htm.

A feasibility study towards instrumentation of the Sport Concussion Assessment Tool (iSCAT).

The Sports Concussion Assessment Tool (SCAT) is a pen and paper-based evaluation tool for use by healthcare professionals in the acute evaluation of suspected concussion. Here we present a feasibility study towards instrumented SCAT (iSCAT). Traditionally, a healthcare professional subjectively counts errors according to SCAT marking criteria matrix. It is hypothesized that an instrumented version of the test will be more accurate while providing additional digital-based parameters to better inform player management. The feasibility study focuses on the SCAT physical functioning tasks only: double leg stance, single-leg stance, tandem stance and tandem gait. Amateur university rugby players underwent iSCAT testing and data were recorded with 8 inertial units attached at different anatomical locations. Video data were gathered simultaneously as reference. An iSCAT algorithm was used to detect errors and quantify additional concussion-based time and frequency domain parameters to assess participant stability during balance and gait tasks. Future work aims to instrument other SCAT features such as hand-eye coordination while deploying methods within a large concussion project.

Hierarchical low-rank and sparse tensor micro defects decomposition by electromagnetic thermography imaging system.

With the advancement of electromagnetic induction thermography and imaging technology in non-destructive testing field, this system has significantly benefitted modern industries in fast and contactless defects detection. However, due to the limitations of front-end hardware experimental equipment and the complicated test pieces, these have brought forth new challenges to the detection process. Making use of the spatio-temporal video data captured by the thermal imaging device and linking it with advanced video processing algorithm to defects detection has become a necessary alternative way to solve these detection challenges. The extremely weak and sparse defect signal is buried in complex background with the presence of strong noise in the real experimental scene has prevented progress to be made in defects detection. In this paper, we propose a novel hierarchical low-rank and sparse tensor decomposition method to mine anomalous patterns in the induction thermography stream for defects detection. The proposed algorithm offers advantages not only in suppressing the interference of strong background and sharpens the visual features of defects, but also overcoming the problems of over- and under-sparseness suffered by similar state-of-the-art algorithms. Real-time natural defect detection experiments have been conducted to verify that the proposed algorithm is more efficient and accurate than existing algorithms in terms of visual presentations and evaluation criteria. This article is part of the theme issue 'Advanced electromagnetic non-destructive evaluation and smart monitoring'.

Efficient Noisy Sound-Event Mixture Classification Using Adaptive-Sparse Complex-Valued Matrix Factorization and OvsO SVM.

This paper proposes a solution for events classification from a sole noisy mixture that consist of two major steps: a sound-event separation and a sound-event classification. The traditional complex nonnegative matrix factorization (CMF) is extended by cooperation with the optimal adaptive L1 sparsity to decompose a noisy single-channel mixture. The proposed adaptive L1 sparsity CMF algorithm encodes the spectra pattern and estimates the phase of the original signals in time-frequency representation. Their features enhance the temporal decomposition process efficiently. The support vector machine (SVM) based one versus one (OvsO) strategy was applied with a mean supervector to categorize the demixed sound into the matching sound-event class. The first step of the multi-class MSVM method is to segment the separated signal into blocks by sliding demixed signals, then encoding the three features of each block. Mel frequency cepstral coefficients, short-time energy, and short-time zero-crossing rate are learned with multi sound-event classes by the SVM based OvsO method. The mean supervector is encoded from the obtained features. The proposed method has been evaluated with both separation and classification scenarios using real-world single recorded signals and compared with the state-of-the-art separation method. Experimental results confirmed that the proposed method outperformed the state-of-the-art methods.

Sensor Signal and Information Processing II.

This Special Issue compiles a set of innovative developments on the use of sensor signals and information processing. In particular, these contributions report original studies on a wide variety of sensor signals including wireless communication, machinery, ultrasound, imaging, and internet data, and information processing methodologies such as deep learning, machine learning, compressive sensing, and variational Bayesian. All these devices have one point in common: These algorithms have incorporated some form of computational intelligence as part of their core framework in problem solving. They have the capacity to generalize and discover knowledge for themselves, learning to learn new information whenever unseen data are captured.

A Deep-Learning-Driven Light-Weight Phishing Detection Sensor.

This paper designs an accurate and low-cost phishing detection sensor by exploring deep learning techniques. Phishing is a very common social engineering technique. The attackers try to deceive online users by mimicking a uniform resource locator (URL) and a webpage. Traditionally, phishing detection is largely based on manual reports from users. Machine learning techniques have recently been introduced for phishing detection. With the recent rapid development of deep learning techniques, many deep-learning-based recognition methods have also been explored to improve classification performance. This paper proposes a light-weight deep learning algorithm to detect the malicious URLs and enable a real-time and energy-saving phishing detection sensor. Experimental tests and comparisons have been conducted to verify the efficacy of the proposed method. According to the experiments, the true detection rate has been improved. This paper has also verified that the proposed method can run in an energy-saving embedded single board computer in real-time.