Application of Meta-Heuristic Algorithms for Training Neural Networks and Deep Learning Architectures: A Comprehensive Review.

The learning process and hyper-parameter optimization of artificial neural networks (ANNs) and deep learning (DL) architectures is considered one of the most challenging machine learning problems. Several past studies have used gradient-based back propagation methods to train DL architectures. However, gradient-based methods have major drawbacks such as stucking at local minimums in multi-objective cost functions, expensive execution time due to calculating gradient information with thousands of iterations and needing the cost functions to be continuous. Since training the ANNs and DLs is an NP-hard optimization problem, their structure and parameters optimization using the meta-heuristic (MH) algorithms has been considerably raised. MH algorithms can accurately formulate the optimal estimation of DL components (such as hyper-parameter, weights, number of layers, number of neurons, learning rate, etc.). This paper provides a comprehensive review of the optimization of ANNs and DLs using MH algorithms. In this paper, we have reviewed the latest developments in the use of MH algorithms in the DL and ANN methods, presented their disadvantages and advantages, and pointed out some research directions to fill the gaps between MHs and DL methods. Moreover, it has been explained that the evolutionary hybrid architecture still has limited applicability in the literature. Also, this paper classifies the latest MH algorithms in the literature to demonstrate their effectiveness in DL and ANN training for various applications. Most researchers tend to extend novel hybrid algorithms by combining MHs to optimize the hyper-parameters of DLs and ANNs. The development of hybrid MHs helps improving algorithms performance and capable of solving complex optimization problems. In general, the optimal performance of the MHs should be able to achieve a suitable trade-off between exploration and exploitation features. Hence, this paper tries to summarize various MH algorithms in terms of the convergence trend, exploration, exploitation, and the ability to avoid local minima. The integration of MH with DLs is expected to accelerate the training process in the coming few years. However, relevant publications in this way are still rare.

A Novel Deep Supervised Learning-Based Approach for Intrusion Detection in IoT Systems.

The Internet of Things (IoT) has become one of the most important concepts in various aspects of our modern life in recent years. However, the most critical challenge for the world-wide use of the IoT is to address its security issues. One of the most important tasks to address the security challenges in the IoT is to detect intrusion in the network. Although the machine/deep learning-based solutions have been repeatedly used to detect network intrusion through recent years, there is still considerable potential to improve the accuracy and performance of the classifier (intrusion detector). In this paper, we develop a novel training algorithm to better tune the parameters of the used deep architecture. To specifically do so, we first introduce a novel neighborhood search-based particle swarm optimization (NSBPSO) algorithm to improve the exploitation/exploration of the PSO algorithm. Next, we use the advantage of NSBPSO to optimally train the deep architecture as our network intrusion detector in order to obtain better accuracy and performance. For evaluating the performance of the proposed classifier, we use two network intrusion detection datasets named UNSW-NB15 and Bot-IoT to rate the accuracy and performance of the proposed classifier.

Spatio-temporal modeling of COVID-19 prevalence and mortality using artificial neural network algorithms.

The outbreak of coronavirus disease (COVID-19) has become one of the most challenging global concerns in recent years. Due to inadequate worldwide studies on spatio-temporal modeling of COVID-19, this research aims to examine the relative significance of potential explanatory variables (n = 75) concerning COVID-19 prevalence and mortality using multilayer perceptron artificial neural network topology. We utilized ten variable importance analysis methods to identify the relative importance of the explanatory variables. The main findings indicated that several variables were persistently among the most influential variables in all periods. Regarding COVID-19 prevalence, unemployment and population density were among the most influential variables with the highest importance scores. While for COVID-19 mortality, health-related variables such as diabetes prevalence and number of hospital beds were among the most significant variables. The obtained findings from this study might provide general insights for public health policymakers to monitor the spread of disease and support decision-making.