Knowledge-Data Driven Optimal Control for Nonlinear Systems and Its Application to Wastewater Treatment Process.

Optimal control is developed to guarantee nonlinear systems run in an optimum operating state. However, since the operation demands of systems are dynamically changeable, it is difficult for optimal control to obtain reliable optimal solutions to achieve satisfying operation performance. To overcome this problem, a knowledge-data driven optimal control (KDDOC) for nonlinear systems is designed in this article. First, an adaptive initialization strategy, using the knowledge from historical operation information of nonlinear systems, is employed to dynamically preset parameters of KDDOC. Then, the initial performance of KDDOC can be enhanced for nonlinear systems. Second, a knowledge guide-based global best selection mechanism is used to assist KDDOC in searching for the optimal solutions under different operation demands. Then, dynamic optimal solutions of KDDOC can be obtained to adapt to flexible changes in nonlinear systems. Third, a knowledge direct-based exploitation mechanism is presented to accelerate the solving process of KDDOC. Then, the demand response speed of KDDOC can be improved to ensure nonlinear systems with optimal operation performance in different states. Finally, the performance of KDDOC is validated on a simulation and a practical process. Several experimental results illustrate the effectiveness of the proposed optimal control for nonlinear systems.

Advanced optimal tracking integrating a neural critic technique for asymmetric constrained zero-sum games.

This paper investigates the optimal tracking issue for continuous-time (CT) nonlinear asymmetric constrained zero-sum games (ZSGs) by exploiting the neural critic technique. Initially, an improved algorithm is constructed to tackle the tracking control problem of nonlinear CT multiplayer ZSGs. Also, we give a novel nonquadratic function to settle the asymmetric constraints. One thing worth noting is that the method used in this paper to solve asymmetric constraints eliminates the strict restriction on the control matrix compared to the previous ones. Further, the optimal controls, the worst disturbances, and the tracking Hamilton-Jacobi-Isaacs equation are derived. Next, a single critic neural network is built to estimate the optimal cost function, thus obtaining the approximations of the optimal controls and the worst disturbances. The critic network weight is updated by the normalized steepest descent algorithm. Additionally, based on the Lyapunov method, the stability of the tracking error and the weight estimation error of the critic network is analyzed. In the end, two examples are offered to validate the theoretical results.

Neural critic learning with accelerated value iteration for nonlinear model predictive control.

In practical industrial processes, the receding optimization solution of nonlinear model predictive control (NMPC) is always a very knotty problem. Based on adaptive dynamic programming, the accelerated value iteration predictive control (AVI-PC) algorithm is developed in this paper. Integrating iteration learning with the receding horizon mechanism of NMPC, a novel receding optimization solution pattern is exploited to resolve the optimal control law in each prediction horizon. Besides, the basic architecture and the specific form of the AVI-PC algorithm are demonstrated, including the relationship among the iterative learning process, the prediction process, and the control process. On this basis, the convergence and admissibility conditions are established, and the relevant properties are comprehensively analyzed when the accelerated factor satisfies the established conditions. Furthermore, the accelerated value iterative function is approximated through the single critic network constructed by utilizing the multiple linear regression method. Finally, the plentiful simulation experiments are conducted from various perspectives to verify the effectiveness and progressiveness of the AVI-PC algorithm.

Sampled-Data H∞ Dynamic Output-Feedback Control for NCSs With Successive Packet Losses and Stochastic Sampling.

The problem of sampled-data H∞ dynamic output-feedback control for networked control systems with successive packet losses (SPLs) and stochastic sampling is investigated in this article. The aim of using sampled-data control techniques is to alleviate network congestion. SPLs that occur in the sensor-to-controller (S-C) and controller-to-actuator (C-A) channels are modeled using a packet loss model. Additionally, it is assumed that stochastic sampling follows a Bernoulli distribution. A model is established to capture the stochastic characteristics of both the SPL model and stochastic sampling. This model is crucial as it allows us to determine the probability distribution of the sampling interval between successive update instants, which is essential for stability analysis. An exponential mean-square stability condition for the constructed equivalent discrete-time stochastic system, which also guarantees the prescribed H∞ performance, is established by incorporating probability theory. The desired controller is designed using a step-by-step synthesis approach, which may offer lower design conservatism compared to some existing methods. Finally, our designed approach using a networked F-404 engine system model is validated and its merits relative to existing results are discussed. The proposed method is finally validated by employing a networked model of the F-404 engine system. Furthermore, the advantages of our method are presented in comparison to previous results.

A Fast Feedforward Small-World Neural Network for Nonlinear System Modeling.

It is well-documented that cross-layer connections in feedforward small-world neural networks (FSWNNs) enhance the efficient transmission for gradients, thus improving its generalization ability with a fast learning. However, the merits of long-distance cross-layer connections are not fully utilized due to the random rewiring. In this study, aiming to further improve the learning efficiency, a fast FSWNN (FFSWNN) is proposed by taking into account the positive effects of long-distance cross-layer connections, and applied to nonlinear system modeling. First, a novel rewiring rule by giving priority to long-distance cross-layer connections is proposed to increase the gradient transmission efficiency when constructing FFSWNN. Second, an improved ridge regression method is put forward to determine the initial weights with high activation for the sigmoidal neurons in FFSWNN. Finally, to further improve the learning efficiency, an asynchronous learning algorithm is designed to train FFSWNN, with the weights connected to the output layer updated by the ridge regression method and other weights by the gradient descent method. Several experiments are conducted on four benchmark datasets from the University of California Irvine (UCI) machine learning repository and two datasets from real-life problems to evaluate the performance of FFSWNN on nonlinear system modeling. The results show that FFSWNN has significantly faster convergence speed and higher modeling accuracy than the comparative models, and the positive effects of the novel rewiring rule, the improved weight initialization, and the asynchronous learning algorithm on learning efficiency are demonstrated.

Neural Q-learning for discrete-time nonlinear zero-sum games with adjustable convergence rate.

In this paper, an adjustable Q-learning scheme is developed to solve the discrete-time nonlinear zero-sum game problem, which can accelerate the convergence rate of the iterative Q-function sequence. First, the monotonicity and convergence of the iterative Q-function sequence are analyzed under some conditions. Moreover, by employing neural networks, the model-free tracking control problem can be overcome for zero-sum games. Second, two practical algorithms are designed to guarantee the convergence with accelerated learning. In one algorithm, an adjustable acceleration phase is added to the iteration process of Q-learning, which can be adaptively terminated with convergence guarantee. In another algorithm, a novel acceleration function is developed, which can adjust the relaxation factor to ensure the convergence. Finally, through a simulation example with the practical physical background, the fantastic performance of the developed algorithm is demonstrated with neural networks.

Diversified Knowledge Transfer Strategy for Multitasking Particle Swarm Optimization.

Evolutionary multitasking optimization (EMTO) has capability of performing a population of individuals together by sharing their intrinsic knowledge. However, the existed methods of EMTO mainly focus on improving its convergence using parallelism knowledge belonging to different tasks. This fact may lead to the problem of local optimization in EMTO due to unexploited knowledge on behalf of the diversity. To address this problem, in this article, a diversified knowledge transfer strategy is proposed for multitasking particle swarm optimization algorithm (DKT-MTPSO). First, according to the state of population evolution, an adaptive task selection mechanism is introduced to manage the source tasks that contribute to the target tasks. Second, a diversified knowledge reasoning strategy is designed to capture the knowledge of convergence, as well as the knowledge associated with diversity. Third, a diversified knowledge transfer method is developed to expand the region of generated solutions guided by acquired knowledge with different transfer patterns so that the search space of tasks can be explored comprehensively, which is favor of EMTO alleviating local optimization. Finally, the performance of the proposed algorithm is evaluated in comparison with some other state-of-the-art EMTO algorithms on multiobjective multitasking benchmark test suits, and the practicality of the algorithm is verified in a real-world application study. The results of experiments demonstrate the superiority of DKT-MTPSO compared to other algorithms.

Double-Closed-Loop Robust Optimal Control for Uncertain Nonlinear Systems.

Optimal control methods have gained significant attention due to their promising performance in nonlinear systems. In general, an optimal control method is regarded as an optimization process for solving the optimal control laws. However, for uncertain nonlinear systems with complex optimization objectives, the solving of optimal reference trajectories is difficult and significant that might be ignored to obtain robust performance. For this problem, a double-closed-loop robust optimal control (DCL-ROC) is proposed to maintain the optimal control reliability of uncertain nonlinear systems. First, a double-closed-loop scheme is established to divide the optimal control process into a closed-loop optimization process that solves optimal reference trajectories and a closed-loop control process that solves optimal control laws. Then, the ability of the optimal control method can be improved to solve complex uncertain optimization problems. Second, a closed-loop robust optimization (CL-RO) algorithm is developed to express uncertain optimization objectives as data-driven forms and adjust optimal reference trajectories in a close loop. Then, the optimality of reference trajectories can be improved under uncertainties. Third, the optimal reference trajectories are tracked by an adaptive controller to derive the optimal control laws without certain system dynamics. Then, the adaptivity and reliability of optimal control laws can be improved. The experimental results demonstrate that the proposed method can achieve better performance than other optimal control methods.

Application of TCN-biGRU neural network in [Formula: see text] concentration prediction.

Fine particulate matter ([Formula: see text]) poses a significant threat to human life and health, and therefore, accurately predicting [Formula: see text] concentration is critical for controlling air pollution. Two improved types of recurrent neural networks (RNNs), the long short-term memory (LSTM) and gated recurrent unit (GRU), have been widely used in time series data prediction due to their ability to capture temporal features. However, both degrade into random guessing as the time length increases. In order to enhance the accuracy of [Formula: see text] concentration prediction and address the issue of random guessing in RNNs neural networks, this study introduces a TCN-biGRU neural network model. This model is a hybrid prediction approach based on combining temporal convolutional networks (TCN) and bidirectional gated recurrent units (bi-GRU). TCN extracts higher-level feature information from longer time series data of [Formula: see text] concentrations, while bi-GRU captures features from past and future data to achieve more accurate predictive outcomes. This case study utilizes data from monitoring stations in Beijing in 2021 for conducting [Formula: see text] prediction experiments. The TCN-biGRU model achieves an average absolute error, root mean square error, and [Formula: see text] of 4.20, 7.71, and 0.961 in its predictive outcomes. When compared to the predictive outcomes of individual LSTM, GRU, and bi-GRU models, it is evident that the TCN-biGRU model exhibits smaller errors and superior predictive performance.

Self-Organizing Robust Fuzzy Neural Network for Nonlinear System Modeling.

Fuzzy neural network (FNN) is a structured learning technique that has been successfully adopted in nonlinear system modeling. However, since there exist uncertain external disturbances arising from mismatched model errors, sensor noises, or unknown environments, FNN generally fails to achieve the desirable performance of modeling results. To overcome this problem, a self-organization robust FNN (SOR-FNN) is developed in this article. First, an information integration mechanism (IIM), consisting of partition information and individual information, is introduced to dynamically adjust the structure of SOR-FNN. The proposed mechanism can make itself adapt to uncertain environments. Second, a dynamic learning algorithm based on the α -divergence loss function ( α -DLA) is designed to update the parameters of SOR-FNN. Then, this learning algorithm is able to reduce the sensibility of disturbances and improve the robustness of Third, the convergence of SOR-FNN is given by the Lyapunov theorem. Then, the theoretical analysis can ensure the successful application of SOR-FNN. Finally, the proposed SOR-FNN is tested on several benchmark datasets and a practical application to validate its merits. The experimental results indicate that the proposed SOR-FNN can obtain superior performance in terms of model accuracy and robustness.

Intelligent-Critic-Based Tracking Control of Discrete-Time Input-Affine Systems and Approximation Error Analysis With Application Verification.

In recent years, the application of function approximators, such as neural networks and polynomials, has ushered in a new stage of development in solving optimal control problems. However, considering the existence of approximation errors, the stability of the controlled system cannot be guaranteed. Therefore, in view of the prevalence of approximation errors, we investigate optimal tracking control problems for discrete-time systems. First, a novel value function is introduced into the intelligent critic framework. Second, an implicit method is utilized to demonstrate the boundedness of the iterative value functions with approximation errors. An explicit method is applied to prove the stability of the system with approximation errors. Furthermore, an evolving policy is designed to iteratively tackle the optimal tracking control problem and demonstrate the stability of the system. Finally, the effectiveness of the developed method is verified through numerical as well as practical examples.

Anammox granule destruction and reconstruction in a partial nitrification/anammox system under hydroxylamine stress.

Nitrite oxidizing bacteria (NOB) outcompeting anammox bacteria (AnAOB) poses a challenge to the practical implementation of the partial nitrification/anammox (PN/A) process for municipal wastewater. A granules-based PN/A bioreactor was operated for 260 d with hydroxylamine (NH2OH) added halfway through. qPCR results detected the different amounts of NOB among granules and flocs and the dynamic succession during operation. CLSM images revealed a unique layered structure of granules that NOB located inside led to the inhibition effect of NH2OH delayed. Besides, the physical and morphological characteristics revealed that anammox granules experienced destruction. AnAOB took the broken granules as an initial biofilm aggregate to reconstruct new granules. RT-qPCR and high throughput sequencing results suggested that functional gene expression and community structure were regulated for the AnAOB metabolism process. Correspondingly, the rapid proliferation (0.52 â†’ 1.99%) of AnAOB was realized, and the nitrogen removal rate achieved a nearly quadruple improvement (0.21 â†’ 0.83 kg-N/m3·d). This study revealed that anammox granules can self-reconstruct in the PN/A system when granules are disintegrated under NH2OH stress, broadening the feasibility of applying PN/A process.

Dichotomy value iteration with parallel learning design towards discrete-time zero-sum games.

In this paper, a novel parallel learning framework is developed to solve zero-sum games for discrete-time nonlinear systems. Briefly, the purpose of this study is to determine a tentative function according to the prior knowledge of the value iteration (VI) algorithm. The learning process of the parallel controllers can be guided by the tentative function. That is to say, the neighborhood of the optimal cost function can be compressed within a small range via two typical exploration policies. Based on the parallel learning framework, a novel dichotomy VI algorithm is established to accelerate the learning speed. It is shown that the parallel controllers will converge to the optimal policy from contrary initial policies. Finally, two typical systems are used to demonstrate the learning performance of the constructed dichotomy VI algorithm.

Fuzzy Boundary Sampled-Data Control for Nonlinear Parabolic DPSs.

For a nonlinear parabolic distributed parameter system (DPS), a fuzzy boundary sampled-data (SD) control method is introduced in this article, where distributed SD measurement and boundary SD measurement are respected. Initially, this nonlinear parabolic DPS is represented precisely by a Takagi-Sugeno (T-S) fuzzy parabolic partial differential equation (PDE) model. Subsequently, under distributed SD measurement and boundary SD measurement, a fuzzy boundary SD control design is obtained via linear matrix inequalities (LMIs) on the basis of the T-S fuzzy parabolic PDE model to guarantee exponential stability for closed-loop parabolic DPS by using inequality techniques and a LF. Furthermore, respecting the property of membership functions, we present some LMI-based fuzzy boundary SD control design conditions. Finally, the effectiveness of the designed fuzzy boundary SD controller is demonstrated via two simulation examples.

Adaptive event-triggered extended dissipative synchronization of delayed reaction-diffusion neural networks under deception attacks.

Under spatially averaged measurements (SAMs) and deception attacks, this article mainly studies the problem of extended dissipativity output synchronization of delayed reaction-diffusion neural networks via an adaptive event-triggered sampled-data (AETSD) control strategy. Compared with the existing ETSD control methods with constant thresholds, our scheme can be adaptively adjusted according to the current sampling and latest transmitted signals and is realized based on limited sensors and actuators. Firstly, an AETSD control scheme is proposed to save the limited transmission channel. Secondly, some synchronization criteria under SAMs and deception attacks are established by utilizing Lyapunov-Krasovskii functional and inequality techniques. Then, by solving linear matrix inequalities (LMIs), we obtain the desired AETSD controller, which can satisfy the specified level of extended dissipativity behaviors. Lastly, one numerical example is given to demonstrate the validity of the proposed method.

LASSO and attention-TCN: a concurrent method for indoor particulate matter prediction.

Long time exposure to indoor air pollution environments can increase the risk of cardiovascular and respiratory system damage. Most previous studies focus on outdoor air quality, while few studies on indoor air quality. Current neural network-based methods for indoor air quality prediction ignore the optimization of input variables, process input features serially, and still suffer from loss of information during model training, which may lead to the problems of memory-intensive, time-consuming and low-precision. We present a novel concurrent indoor PM prediction model based on the fusion model of Least Absolute Shrinkage and Selection Operator (LASSO) and an Attention Temporal Convolutional Network (ATCN), together called LATCN. First, a LASSO regression algorithm is used to select features from PM1, PM2.5, PM10 and PM (>10) datasets and environmental factors to optimize the inputs for indoor PM prediction model. Then an Attention Mechanism (AM) is applied to reduce the redundant temporal information to extract key features in inputs. Finally, a TCN is used to forecast indoor particulate concentration in parallel with inputting the extracted features, and it reduces information loss by residual connections. The results show that the main environmental factors affecting indoor PM concentration are the indoor heat index, indoor wind chill, wet bulb temperature and relative humidity. Comparing with Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) approaches, LATCN systematically reduced the prediction error rate (19.7% ~ 28.1% for the NAE, and 16.4% ~ 21.5% for the RMSE) and improved the model running speed (30.4% ~ 81.2%) over these classical sequence prediction models. Our study can inform the active prevention of indoor air pollution, and provides a theoretical basis for indoor environmental standards, while laying the foundations for developing novel air pollution prevention equipment in the future.

Convergence and Stability of Optimal Regulation via Generalized N -Step Value Gradient Learning.

In this article, the generalized N -step value gradient learning (GNSVGL) algorithm, which takes a long-term prediction parameter λ into account, is developed for infinite horizon discounted near-optimal control of discrete-time nonlinear systems. The proposed GNSVGL algorithm can accelerate the learning process of adaptive dynamic programming (ADP) and has a better performance by learning from more than one future reward. Compared with the traditional N -step value gradient learning (NSVGL) algorithm with zero initial functions, the proposed GNSVGL algorithm is initialized with positive definite functions. Considering different initial cost functions, the convergence analysis of the value-iteration-based algorithm is provided. The stability condition for the iterative control policy is established to determine the value of the iteration index, under which the control law can make the system asymptotically stable. Under such a condition, if the system is asymptotically stable at the current iteration, then the iterative control laws after this step are guaranteed to be stabilizing. Two critic neural networks and one action network are constructed to approximate the one-return costate function, the λ -return costate function, and the control law, respectively. It is emphasized that one-return and λ -return critic networks are combined to train the action neural network. Finally, via conducting simulation studies and comparisons, the superiority of the developed algorithm is confirmed.

Neural Network-Based Adaptive Tracking Control for Denitrification and Aeration Processes With Time Delays.

Wastewater treatment process (WWTP), consisting of a class of physical, chemical, and biological phenomena, is an important means to reduce environmental pollution and improve recycling efficiency of water resources. Considering characteristics of the complexities, uncertainties, nonlinearities, and multitime delays in WWTPs, an adaptive neural controller is presented to achieve the satisfying control performance for WWTPs. With the advantages of radial basis function neural networks (RBF NNs), the unknown dynamics in WWTPs are identified. Based on the mechanistic analysis, the time-varying delayed models of the denitrification and aeration processes are established. Based on the established delayed models, the Lyapunov-Krasovskii functional (LKF) is used to compensate for the time-varying delays caused by the push-flow and recycle flow phenomenon. The barrier Lyapunov function (BLF) is used to ensure that the dissolved oxygen (DO) and nitrate concentrations are always kept within the specified ranges though the time-varying delays and disturbances exist. Using Lyapunov theorem, the stability of the closed-loop system is proven. Finally, the proposed control method is carried out on the benchmark simulation model 1 (BSM1) to verify the effectiveness and practicability.

Deterministic Learning-Based Adaptive Neural Control for Nonlinear Full-State Constrained Systems.

In this article, an adaptive neural learning method is introduced for a category of nonlinear strict-feedback systems with time-varying full-state constraints. The two challenging problems of state constraints and learning capability are investigated and solved in a unified framework. To obtain the learning of unknown functions and satisfy full-state constraints, three main steps are considered. First, an adaptive dynamic surface controller (DSC) based on barrier Lyapunov functions (BLFs) is structured to implement that the closed-loop systems signals are bounded and full-state variables remain within the prescribed time-varying intervals. Moreover, the radial basis function neural networks (RBF NNs) are used to identify unknown functions. The output of the first-order filter, instead of virtual control derivatives, is used to simplify the complexity of the RBF NN input variables. Second, the state transformation is used to obtain a class of linear time-varying subsystems with small perturbations such that the recurrence of the RBF NN input variables and the partial persistent excitation condition are actualized. Therefore, the unknown functions can be accurately approximated, and the learned knowledge is kept as constant NN weights. Third, the obtained constant weights are borrowed into an adaptive learning scheme to achieve the batter control performance. Finally, simulation studies illustrate the advantage of the reported adaptive learning method on higher tracking accuracy, faster convergence rate, and lower computational expense by reusing learned knowledge.

System Stability of Learning-Based Linear Optimal Control With General Discounted Value Iteration.

For discounted optimal regulation design, the stability of the controlled system is affected by the discount factor. If an inappropriate discount factor is employed, the optimal control policy might be unstabilizing. Therefore, in this article, the effect of the discount factor on the stabilization of control strategies is discussed. We develop the system stability criterion and the selection rules of the discount factor with respect to the linear quadratic regulator problem under the general discounted value iteration algorithm. Based on the monotonicity of the value function sequence, the method to judge the stability of the controlled system is established during the iteration process. In addition, once some stability conditions are satisfied at a certain iteration step, all control policies after this iteration step are stabilizing. Furthermore, combined with the undiscounted optimal control problem, the practical rule of how to select an appropriate discount factor is constructed. Finally, several simulation examples with physical backgrounds are conducted to demonstrate the present theoretical results.