Considerations for Quality Control Monitoring of Machine Learning Models in Clinical Practice.

Integrating machine learning (ML) models into clinical practice presents a challenge of maintaining their efficacy over time. While existing literature offers valuable strategies for detecting declining model performance, there is a need to document the broader challenges and solutions associated with the real-world development and integration of model monitoring solutions. This work details the development and use of a platform for monitoring the performance of a production-level ML model operating in Mayo Clinic. In this paper, we aimed to provide a series of considerations and guidelines necessary for integrating such a platform into a team's technical infrastructure and workflow. We have documented our experiences with this integration process, discussed the broader challenges encountered with real-world implementation and maintenance, and included the source code for the platform. Our monitoring platform was built as an R shiny application, developed and implemented over the course of 6 months. The platform has been used and maintained for 2 years and is still in use as of July 2023. The considerations necessary for the implementation of the monitoring platform center around 4 pillars: feasibility (what resources can be used for platform development?); design (through what statistics or models will the model be monitored, and how will these results be efficiently displayed to the end user?); implementation (how will this platform be built, and where will it exist within the IT ecosystem?); and policy (based on monitoring feedback, when and what actions will be taken to fix problems, and how will these problems be translated to clinical staff?). While much of the literature surrounding ML performance monitoring emphasizes methodological approaches for capturing changes in performance, there remains a battery of other challenges and considerations that must be addressed for successful real-world implementation.

Early Post-Hospitalization Hemoglobin Recovery and Clinical Outcomes in Survivors of Critical Illness: A Population-Based Cohort Study.

Anemia is common during critical illness, is associated with adverse clinical outcomes, and often persists after hospitalization. The goal of this investigation is to assess the relationships between post-hospitalization hemoglobin recovery and clinical outcomes after survival of critical illness. This is a population-based observational study of adults (≥18 years) surviving hospitalization for critical illness between January 1, 2010 and December 31, 2016 in Olmsted County, Minnesota, United States with hemoglobin concentrations and clinical outcomes assessed through one-year post-hospitalization. Multi-state proportional hazards models were utilized to assess the relationships between 1-month post-hospitalization hemoglobin recovery and hospital readmission or death through one-year after discharge. Among 6460 patients that survived hospitalization for critical illness during the study period, 2736 (42%) were alive, not hospitalized, and had available hemoglobin concentrations assessed at 1-month post-index hospitalization. Median (interquartile range) age was 69 (56, 80) years with 54% of male gender. Overall, 86% of patients had anemia at the time of hospital discharge, with median discharge hemoglobin concentrations of 10.2 (9.1, 11.6) g/dL. In adjusted analyses, each 1 g/dL increase in 1-month hemoglobin recovery was associated with decreased instantaneous hazard for hospital readmission (HR 0.87 [95% CI 0.84-0.90]; p < 0.001) and lower mortality (HR 0.82 [95% CI 0.75-0.89]; p < 0.001) through one-year post-hospitalization. The results were consistent in multiple pre-defined sensitivity analyses. Impaired early post-hospitalization hemoglobin recovery is associated with inferior clinical outcomes in the first year of survival after critical illness. Additional investigations are warranted to evaluate these relationships.

Obesity and survival among a cohort of breast cancer patients is partially mediated by tumor characteristics.

Obesity exerts adverse effects on breast cancer survival, but the means have not been fully elucidated. We evaluated obesity as a contributor to breast cancer survival according to tumor molecular subtypes in a population-based case-cohort study using data from the Surveillance Epidemiology and End Results (SEER) program. We determined whether obese women were more likely to be diagnosed with poor prognosis tumor characteristics and quantified the contribution of obesity to survival. Hazard ratios (HRs) and 95% confidence intervals (CI) were calculated via Cox multivariate models. The effect of obesity on survival was evaluated among 859 incident breast cancers (subcohort; 15% random sample; median survival 7.8 years) and 697 deaths from breast cancer (cases; 100% sample). Obese women had a 1.7- and 1.8-fold increased risk of stage III/IV disease and grade 3/4 tumors, respectively. Obese women with Luminal A- and Luminal B-like breast cancer were 1.8 (95% CI 1.3-2.5) and 2.2 (95% CI 0.9-5.0) times more likely to die from their cancer compared to normal weight women. In mediation analyses, the proportion of excess mortality attributable to tumor characteristics was 36.1% overall and 41% and 38% for Luminal A- and Luminal B-like disease, respectively. Obesity was not associated with breast cancer-specific mortality among women who had Her2-overexpressing or triple-negative tumors. Obesity may influence hormone-positive breast cancer-specific mortality in part through fostering poor prognosis tumors. When tumor biology is considered as part of the causal pathway, the public health impact of obesity on breast cancer survival may be greater than previously estimated.

Carbon Capture Simulation Initiative: a case study in multiscale modeling and new challenges.

Advanced multiscale modeling and simulation have the potential to dramatically reduce the time and cost to develop new carbon capture technologies. The Carbon Capture Simulation Initiative is a partnership among national laboratories, industry, and universities that is developing, demonstrating, and deploying a suite of such tools, including basic data submodels, steady-state and dynamic process models, process optimization and uncertainty quantification tools, an advanced dynamic process control framework, high-resolution filtered computational-fluid-dynamics (CFD) submodels, validated high-fidelity device-scale CFD models with quantified uncertainty, and a risk-analysis framework. These tools and models enable basic data submodels, including thermodynamics and kinetics, to be used within detailed process models to synthesize and optimize a process. The resulting process informs the development of process control systems and more detailed simulations of potential equipment to improve the design and reduce scale-up risk. Quantification and propagation of uncertainty across scales is an essential part of these tools and models.