Challenges and Opportunities in Deep Reinforcement Learning With Graph Neural Networks: A Comprehensive Review of Algorithms and Applications.

Deep reinforcement learning (DRL) has empowered a variety of artificial intelligence fields, including pattern recognition, robotics, recommendation systems, and gaming. Similarly, graph neural networks (GNNs) have also demonstrated their superior performance in supervised learning for graph-structured data. In recent times, the fusion of GNN with DRL for graph-structured environments has attracted a lot of attention. This article provides a comprehensive review of these hybrid works. These works can be classified into two categories: 1) algorithmic contributions, where DRL and GNN complement each other with an objective of addressing each other's shortcomings and 2) application-specific contributions that leverage a combined GNN-DRL formulation to address problems specific to different applications. This fusion effectively addresses various complex problems in engineering and life sciences. Based on the review, we further analyze the applicability and benefits of fusing these two domains, especially in terms of increasing generalizability and reducing computational complexity. Finally, the key challenges in integrating DRL and GNN, and potential future research directions are highlighted, which will be of interest to the broader machine learning community.

On developing a framework for detection of oscillations in data.

Oscillation is a phenomenon very commonly observed in systems, ranging from simple ones to complex distributed network. Several techniques have been proposed in the literature for detecting oscillations to study their importance in domains ranging from physiology to climate studies. However, there is a lack of a common framework accommodative of important features of data such as non-stationarity, intermittent oscillations, measurement noise, multimodal oscillations, and the like. In this article, we outline a framework that addresses these challenges, the results of which can then be analyzed along with appropriate knowledge about the underlying system. We present results of an extensive simulation study that establishes the robustness and reliability of the proposed technique and demonstrate its applicability to real datasets in climate and in industrial datasets.

A novel approach for benchmarking and assessing the performance of state estimators.

State estimation is a widely adopted soft sensing technique that incorporates predictions from an accurate model of the process and measurements to provide reliable estimates of unmeasured variables. The reliability of such estimators is threatened by measurement related challenges and model inaccuracies. In this article, a method for benchmarking of state estimation techniques is proposed. This method can be used to quantify the performance and hence reliability of an estimator. The Hurst exponents of a posteriori filtering errors are analyzed to characterize a benchmark (minimum mean squared error) estimator, similar to the minimum variance control benchmark developed for control loops. A distance metric is then used to quantify the extent of deviation of an estimator from the benchmark. The proposed technique is developed for linear systems and extended to non-linear systems with single as well as multiple measurable variables. Simulation studies are carried out with Kalman based as well as Monte Carlo based estimators whose computational details are significantly different. Results reveal that the technique serves as a tool that can quantify the performance and assess the reliability of a state estimator. The strengths and limitations of the proposed technique are discussed with guidelines on applications and deployment of the technique in a real life system.