Improving the interpretability of deep reinforcement learning in urban drainage system operation.

Deep reinforcement learning (DRL) has been increasingly used as an adaptive and efficient solution for real-time control (RTC) of the urban drainage system (UDS). Despite the promising potential of DRL, it is a black-box model whose control logic and control consequences are difficult to be understood and evaluated. This leads to issues of interpretability and poses risks in practical applications. This study develops an evaluation framework to analyze and improve the interpretability of DRL-based UDS operation. The framework includes three analysis methods: Sobol sensitivity analysis, tree-based surrogate modelling, and conditional probability analysis. It is validated using two different DRL approaches, i.e., deep Q-learning network (DQN) and proximal policy optimization (PPO), which are trained to reduce combined sewer overflow (CSO) discharges and flooding in a real-world UDS. According to the results, the two DRLs have been shown to perform better than a rule-based control system that is currently being used. Sobol sensitivity analysis indicates that DQN is particularly sensitive to the flow of links and rainfall, while PPO is sensitive to all the states. Tree-based surrogate models effectively reveal the control logic behind the DRLs and indicate that PPO is more comprehensible but DQN is more forward-looking. Conditional probability analysis demonstrates the potential control consequences of the DRLs and identifies three situations where the DRLs are ineffective: a) the storage of UDS is fully utilized; b) peak flows have already passed through actuators; c) a substantial amount of water enters one location simultaneously. The proposed evaluation framework enhances the interpretability of DRL in UDS operations, fostering trust and confidence from operators, stakeholders, and regulators.

Enhancing interpretability and generalizability of deep learning-based emulator in three-dimensional lake hydrodynamics using Koopman operator and transfer learning: Demonstrated on the example of lake Zurich.

Three-dimensional lake hydrodynamic model is a powerful tool widely used to assess hydrological condition changes of lake. However, its computational cost becomes problematic when forecasting the state of large lakes or using high-resolution simulation in small-to-medium size lakes. One possible solution is to employ a data-driven emulator, such as a deep learning (DL) based emulator, to replace the original model for fast computing. However, existing DL-based emulators are often black-box and data-dependent models, causing poor interpretability and generalizability in practical applications. In this study, a data-driven emulator is established using deep neural network (DNN) to replace the original model for fast computing of three-dimensional lake hydrodynamics. Then, the Koopman operator and transfer learning (TL) are employed to enhance the interpretability and generalizability of the emulator. Finally, the generalizability of DL-based emulators is comprehensively analyzed through linear regression and correlation analysis. These methods are tested against an existing hydrodynamic model of Lake Zurich (Switzerland) whose data was provided by an open-source web-based platform called Meteolakes/Alplakes. According to the results, (1) The DLEDMD offers better interpretability than DNN because its Koopman operator reveals the linear structure behind the hydrodynamics; (2) The generalization of the DL-based emulators in three-dimensional lake hydrodynamics are influenced by the similarity between the training and testing data; (3) TL effectively improves the generalizability of the DL-based emulators.

Towards coordinated and robust real-time control: a decentralized approach for combined sewer overflow and urban flooding reduction based on multi-agent reinforcement learning.

The real-time control (RTC) of urban drainage systems can make full use of the capabilities of existing infrastructures to mitigate combined sewer overflow (CSO) and urban flooding. Despite the benefits of RTC, it may encounter potential risks and failures, which need further consideration to enhance its robustness. Besides failures of hardware components such as sensors and actuators, the RTC performance is also sensitive to communication failures between the devices that are spatially distributed in a catchment-scale system. This paper proposes a decentralized control strategy based on multi-agent reinforcement learning to enhance communication robustness and coordinate the decentralized control agents through centralized training. To investigate different control structures, a centralized and a fully decentralized strategy are also developed based on reinforcement learning (RL) for comparison. A benchmark drainage model and a real-world drainage model are formulated as two cases, and the control agents are trained to control the orifices or pumps for CSO or flooding mitigation in each case. The three RL strategies reduce the CSO volume by 5.62-9.30% compared with a static baseline in historical rainfalls of the benchmark case and reduce the CSO and flooding volume by 14.39-21.36% compared with currently-used rule-based control in synthetic rainfalls of the real-world case. Benefitting from centralized training, the decentralized agents can achieve similar performance to the centralized agent. The decentralized control also enhances the communication robustness with smaller performance loss than the centralized control when observation communication fails, and provides a robust backup at the local level to limit the uncertainties when action commands from the centralized agent are lost. The results and findings indicate that multi-agent RL contributes to a coordinated and robust solution for RTC of urban drainage systems.

Statistical comparison between SARIMA and ANN's performance for surface water quality time series prediction.

The performance comparison studies of the autoregressive integrated moving average model (ARIMA) and the artificial neural network (ANN) were mostly carried out between the selected model structures through trial-and-error, strongly influenced by model structure uncertainty. This research aims to make up for this inadequacy. First, a surface water quality prediction case study including eight monitoring sites in China was introduced. Second, the ARIMA and ANN's performance was compared statistically between 6912 Seasonal ARIMA (SARIMA) and 110,592 feedforward ANN with different model structures, based on the mean square error (MSE) distributions depicted by boxplots. In a statistical view, the ANN models obtained a significantly lower median value and a more concentrated distribution of validation MSEs, which indicated lighter overfitting and better generalization ability. Furthermore, the optimal SARIMA models' performance is inferior to even the median of the ANN models in the case study. In contrast with the previous comparisons among selected models, the statistical comparison in this study shows lower uncertainty.

Urban flood risk assessment and analysis with a 3D visualization method coupling the PP-PSO algorithm and building data.

Due to the influence of buildings on the distribution of flood and their economic and social attributes, 3D spatial information such as the size of buildings and the flooded ratio of buildings relative to their height has an increasing impact on urban flood risk. However, existing flood risk assessment methods mainly use data in 2D and analysis methods are mostly 2D. In this study, flood variation processes were analyzed in the form of 3D dynamic visualization by coupling an urban drainage model and a flood simulation model with 3D visualization methods. By further combining with 3D building models, the 3D spatial information of buildings related to flood was obtained. In order to study the influence of 3D information on flood risk and combine with other multi-source heterogeneous data for integrated analysis, a 3D visualization assessment and analysis method for flood risk, coupled with the projection pursuit-particle swarm optimization algorithm (PP-PSO) was established (3DVAAM-PP-PSO). A case study from Chaohu City, China, was used to demonstrate the method. The results showed that the PP-PSO algorithm can process high-dimensional information and obtain the objective weight of each index. The 3D information from the influenced buildings had an impact on the evaluation results, which needed to be considered. Through the 3D visualization analysis, the overall distribution of flood risk and that around the buildings were obtained in multi-perspectives. The flood risk during different rainfall return periods were analyzed intuitively and comparatively. This study furnishes a novel method for flood risk assessment and analysis by making the most of 3D spatial information.

Transfer learning for neural network model in chlorophyll-a dynamics prediction.

Neural network models have been used to predict chlorophyll-a concentration dynamics. However, as model generalization ability decreases, (i) the performance of the models gradually decreases over time; (ii) the accuracy and performance of the models need to be improved. In this study, Transfer learning (TL) is employed to optimize neural network models (including feedforward neural networks (FNN), recurrent neural networks (RNN) and long short-term memory (LTSM)) and overcome these problems. Models using TL are able to reduce the influence of mutable data distribution and enhance generalization ability. Thus, it can improve the accuracy of prediction and maintain high performance in long-term applications. Also, TL is compared with parameter norm penalties (PNP) and dropout-two other methods used to improve model generalization ability. In general, TL has a better prediction effect than PNP and dropout. All the models, including FNN with different architectures, RNN and LSTM, as well as models optimized by PNP, dropout, and TL, are applied to an estuary reservoir in eastern China to predict chlorophyll-a dynamics at 5-min intervals. According to the results of this study, (i) models with TL produce the best prediction results; (ii) the original models and the models with PNP and dropout lose their ability to predict within 3 months, while TL models retain a high prediction accuracy.

CBR-based integration of a hydrodynamic and water quality model and GIS-a case study of Chaohu City.

Monitoring on urban water environment and analysis of engineering improvement measures are intricate and time-consuming tasks. In previous studies, the integration of hydrodynamic and water quality models and geographical information system (GIS) usually takes three approaches: loose coupling, tight coupling, and full coupling. However, this paper adopted a special loose coupling approach-case-based reasoning (CBR) to develop an integrated decision support system. This was characterized by invoking the case base stored in the GIS platform as the output of the model. The fused capability of model's water quality predication and strong spatial data processing analysis of GIS can be realized at the same time by integration. The functionality of the integrated system was illustrated through a case study of Chaohu, a medium-sized city in China, which includes case retrieval, result interpretation, and the visual display in the GIS platform. Results verified the feasibility and operability of the developed method. As a useful tool, the integrated decision support system makes it simpler and more convenient for decision makers to make decisions efficiently and quickly.