External validity of machine learning-based prognostic scores for cystic fibrosis: A retrospective study using the UK and Canadian registries.

Precise and timely referral for lung transplantation is critical for the survival of cystic fibrosis patients with terminal illness. While machine learning (ML) models have been shown to achieve significant improvement in prognostic accuracy over current referral guidelines, the external validity of these models and their resulting referral policies has not been fully investigated. Here, we studied the external validity of machine learning-based prognostic models using annual follow-up data from the UK and Canadian Cystic Fibrosis Registries. Using a state-of-the-art automated ML framework, we derived a model for predicting poor clinical outcomes in patients enrolled in the UK registry, and conducted external validation of the derived model using the Canadian Cystic Fibrosis Registry. In particular, we studied the effect of (1) natural variations in patient characteristics across populations and (2) differences in clinical practice on the external validity of ML-based prognostic scores. Overall, decrease in prognostic accuracy on the external validation set (AUCROC: 0.88, 95% CI 0.88-0.88) was observed compared to the internal validation accuracy (AUCROC: 0.91, 95% CI 0.90-0.92). Based on our ML model, analysis on feature contributions and risk strata revealed that, while external validation of ML models exhibited high precision on average, both factors (1) and (2) can undermine the external validity of ML models in patient subgroups with moderate risk for poor outcomes. A significant boost in prognostic power (F1 score) from 0.33 (95% CI 0.31-0.35) to 0.45 (95% CI 0.45-0.45) was observed in external validation when variations in these subgroups were accounted in our model. Our study highlighted the significance of external validation of ML models for cystic fibrosis prognostication. The uncovered insights on key risk factors and patient subgroups can be used to guide the cross-population adaptation of ML-based models and inspire new research on applying transfer learning methods for fine-tuning ML models to cope with regional variations in clinical care.

Delay Analysis for End-to-End Synchronous Communication in Monitoring Systems.

With the rapid development of smart grid technologies, communication systems are further integrated in the existing power grids. The real-time capability and reliability of the power applications are receiving increasing concerns. Thus, it is important to measure the end-to-end delay in communication systems. The network calculus theory has been widely applied in the communication delay measuring tasks. However, for better operation performance of power systems, most power applications require synchronous data communication, in which the network calculus theory cannot be directly applied. In this paper, we expand the network calculus theory such that it can be used to analyze the communication delay for power applications in smart grids. The problem of communication delay calculation for the synchronization system is converted into a maximum path problem in graph theory. Finally, our theoretical results are compared with the experimental ones obtained with the network simulation software EstiNet. The simulation results verify the feasibility and effectiveness of the proposed method.