Drone's Objective Inference Using Policy Error Inverse Reinforcement Learning.

Drones are set to penetrate society across transport and smart living sectors. While many are amateur drones that pose no malicious intentions, some may carry deadly capability. It is crucial to infer the drone's objective to prevent risk and guarantee safety. In this article, a policy error inverse reinforcement learning (PEIRL) algorithm is proposed to uncover the hidden objective of drones from online data trajectories obtained from cooperative sensors. A set of error-based polynomial features are used to approximate both the value and policy functions. This set of features is consistent with current onboard storage memories in flight controllers. The real objective function is inferred using an objective constraint and an integral inverse reinforcement learning (IRL) batch least-squares (LS) rule. The convergence of the proposed method is assessed using Lyapunov recursions. Simulation studies using a quadcopter model are provided to demonstrate the benefits of the proposed approach.

Hippocampus experience inference for safety critical control of unknown multi-agent linear systems.

Risk mitigation is usually addressed in simulated environments for safety critical control. The migration of the final controller requires further adjustments due to the simulation assumptions and constraints. This paper presents the design of an experience inference algorithm for safety critical control of unknown multi-agent linear systems. The approach is inspired in the close relationship between three main areas of the brain cortex that enables transfer learning and decision making: the hippocampus, the neocortex, and the striatum. The hippocampus is modelled as a stable linear model that communicates to the striatum how the real-world system is expected to behave. The hippocampus model is controlled by an adaptive dynamic programming (ADP) algorithm to achieve an optimal desired performance. The neocortex and the striatum are designed simultaneously by an actor control policy algorithm that ensures experience inference to the real-world system. Experimental and simulations studies are carried out to verify the proposed approach.

A Closed-Loop Output Error Approach for Physics-Informed Trajectory Inference Using Online Data.

While autonomous systems can be used for a variety of beneficial applications, they can also be used for malicious intentions and it is mandatory to disrupt them before they act. So, an accurate trajectory inference algorithm is required for monitoring purposes that allows to take appropriate countermeasures. This article presents a closed-loop output error approach for trajectory inference of a class of linear systems. The approach combines the main advantages of state estimation and parameter identification algorithms in a complementary fashion using online data and an estimated model, which is constructed by the state and parameter estimates, that inform about the physics of the system to infer the followed noise-free trajectory. Exact model matching and estimation error cases are analyzed. A composite update rule based on a least-squares rule is also proposed to improve robustness and parameter and state convergence. The stability and convergence of the proposed approaches are assessed via the Lyapunov stability theory under the fulfilment of a persistent excitation condition. Simulation studies are carried out to validate the proposed approaches.

Neural H2 Control Using Continuous-Time Reinforcement Learning.

In this article, we discuss continuous-time H2 control for the unknown nonlinear system. We use differential neural networks to model the system, then apply the H2 tracking control based on the neural model. Since the neural H2 control is very sensitive to the neural modeling error, we use reinforcement learning to improve the control performance. The stabilities of the neural modeling and the H2 tracking control are proven. The convergence of the approach is also given. The proposed method is validated with two benchmark control problems.

Optimal sliding mode control for cutting tasks of quick-return mechanisms.

A solution of the constant cutting velocity problem of quick-return mechanisms is the main concern of this paper. An optimal sliding mode control in the task space is used to achieve uniform and accurate cuts throughout the workpiece. The switching hyperplane is designed to minimize the position error of the slider-dynamics in an infinite horizon. A Jacobian compensator is used to exploit the mechanical advantage and ensure controllability. The velocity profile is constructed in terms of the mechanism and workpiece geometric properties. Stability of the closed-loop dynamics is verified with the Lyapunov stability theory. Experiments are carried out in a quick-return mechanism prototype to validate the proposal.

A complementary learning approach for expertise transference of human-optimized controllers.

In this paper, a complementary learning scheme for experience transference of unknown continuous-time linear systems is proposed. The algorithm is inspired in the complementary learning properties that exhibit the hippocampus and neocortex learning systems via the striatum. The hippocampus is modelled as pattern-separated data of a human optimized controller. The neocortex is modelled as a Q-reinforcement learning algorithm which improves the hippocampus control policy. The complementary learning (striatum) is designed as an inverse reinforcement learning algorithm which relates the hippocampus and neocortex learning models to seek and transfer the weights of the hidden expert's utility function. Convergence of the proposed approach is analysed using Lyapunov recursions. Simulations are given to verify the proposed approach.

Discrete-Time H2 Neural Control Using Reinforcement Learning.

In this article, we discuss H2 control for unknown nonlinear systems in discrete time. A discrete-time recurrent neural network is used to model the nonlinear system, and then, the H2 tracking control is applied based on the neural model. Since this neural H2 control is very sensitive to the neural modeling error, we use reinforcement learning and another neural approximator to improve tracking accuracy and robustness of the controller. The stabilities of the neural identifier and the H2 tracking control are proven. The convergence of the approach is also given. The proposed method is validated with the control of the pan and tilt robot and the surge tank.