Appropriate use of blood cultures in the emergency department through machine learning (ABC): study protocol for a randomised controlled non-inferiority trial.

INTRODUCTION: The liberal use of blood cultures in emergency departments (EDs) leads to low yields and high numbers of false-positive results. False-positive, contaminated cultures are associated with prolonged hospital stays, increased antibiotic usage and even higher hospital mortality rates. This trial aims to investigate whether a recently developed and validated machine learning model for predicting blood culture outcomes can safely and effectively guide clinicians in withholding unnecessary blood culture analysis. METHODS AND ANALYSIS: A randomised controlled, non-inferiority trial comparing current practice with a machine learning-guided approach. The primary objective is to determine whether the machine learning based approach is non-inferior to standard practice based on 30-day mortality. Secondary outcomes include hospital length-of stay and hospital admission rates. Other outcomes include model performance and antibiotic usage. Participants will be recruited in the EDs of multiple hospitals in the Netherlands. A total of 7584 participants will be included. ETHICS AND DISSEMINATION: Possible participants will receive verbal information and a paper information brochure regarding the trial. They will be given at least 1 hour consideration time before providing informed consent. Research results will be published in peer-reviewed journals. This study has been approved by the Amsterdam University Medical Centers' local medical ethics review committee (No 22.0567). The study will be conducted in concordance with the principles of the Declaration of Helsinki and in accordance with the Medical Research Involving Human Subjects Act, General Data Privacy Regulation and Medical Device Regulation. TRIAL REGISTRATION NUMBER: NCT06163781.

Detecting changes in the performance of a clinical machine learning tool over time.

BACKGROUND: Excessive use of blood cultures (BCs) in Emergency Departments (EDs) results in low yields and high contamination rates, associated with increased antibiotic use and unnecessary diagnostics. Our team previously developed and validated a machine learning model to predict BC outcomes and enhance diagnostic stewardship. While the model showed promising initial results, concerns over performance drift due to evolving patient demographics, clinical practices, and outcome rates warrant continual monitoring and evaluation of such models. METHODS: A real-time evaluation of the model's performance was conducted between October 2021 and September 2022. The model was integrated into Amsterdam UMC's Electronic Health Record system, predicting BC outcomes for all adult patients with BC draws in real time. The model's performance was assessed monthly using metrics including the Area Under the Curve (AUC), Area Under the Precision-Recall Curve (AUPRC), and Brier scores. Statistical Process Control (SPC) charts were used to monitor variation over time. FINDINGS: Across 3.035 unique adult patient visits, the model achieved an average AUC of 0.78, AUPRC of 0.41, and a Brier score of 0.10 for predicting the outcome of BCs drawn in the ED. While specific population characteristics changed over time, no statistical points outside the statistical control range were detected in the AUC, AUPRC, and Brier scores, indicating stable model performance. The average BC positivity rate during the study period was 13.4%. INTERPRETATION: Despite significant changes in clinical practice, our BC stewardship tool exhibited stable performance, suggesting its robustness to changing environments. Using SPC charts for various metrics enables simple and effective monitoring of potential performance drift. The assessment of the variation of outcome rates and population changes may guide the specific interventions, such as intercept correction or recalibration, that may be needed to maintain a stable model performance over time. This study suggested no need to recalibrate or correct our BC stewardship tool. FUNDING: No funding to disclose.

Digital Health: Today's Solutions and Tomorrow's Impact.

Artificial intelligence (AI) is becoming an indispensable tool to augment decision making in different health care settings and by various members of the patient pathway, including the patient. AI provides the ability to optimize data to bring clinical decision support for clinicians and laboratorians and/or empower patients to actively participate in their own health care. Though there are many examples of AI in health care, the exact role of AI and digital health solutions is still taking shape. Although AI will not replace the clinician, those who do not adopt AI may in time, be left behind.

The World Health Organization Collaborating Center for Emergency and Trauma (WHO-CCET) in South East Asia, The World Academic Council of Emergency Medicine (WACEM), and The American College of Academic International Medicine (ACAIM) 2021 Framework for using Telemedicine Technology at Healthcare Institutions.

The coronavirus disease 2019 crisis has forced the world to integrate telemedicine into health delivery systems in an unprecedented way. To deliver essential care, lawmakers, physicians, patients, payers, and health systems have all adopted telemedicine and redesigned delivery processes with accelerated speed and coordination in a fragmented way without a long-term vision or uniformed standards. There is an opportunity to learn from the experiences gained by this pandemic to help shape a better health-care system that standardizes telemedicine to optimize the overall efficiency of remote health-care delivery. This collaboration focuses on four pillars of telemedicine that will serve as a framework to enable a uniformed, standardized process that allows for remote data capture and quality, aiming to improve ongoing management outside the hospital. In this collaboration, we recommend learning from this experience by proposing a telemedicine framework built on the following four pillars-patient safety and confidentiality; metrics, analytics, and reform; recording of audio-visual data as a health record; and reimbursement and accountability.

Towards Understanding the Effective Use of Antibiotics for Sepsis.

BACKGROUND: The benefits of early antibiotics for sepsis have recently been questioned. Evidence for this mainly comes from observational studies. The only randomized trial on this subject, the Prehospital Antibiotics Against Sepsis (PHANTASi) trial, did not find significant mortality benefits from early antibiotics. That subgroups of patients benefit from this practice is still plausible, given the heterogeneous nature of sepsis. RESEARCH QUESTION: Do subgroups of sepsis patients experience 28-day mortality benefits from early administration of antibiotics in a prehospital setting? And what key traits drive these benefits? STUDY DESIGN AND METHODS: We used machine learning to conduct exploratory partitioning cluster analysis to identify possible subgroups of sepsis patients who may benefit from early antibiotics. We further tested the influence of several traits within these subgroups, using a logistic regression model. RESULTS: We found a significant interaction between age and benefits of early antibiotics (P = .03). When we adjusted for this interaction and several other confounders, there was a significant benefit of early antibiotic treatment (OR, 0.07; 95% CI, 0.01-0.79; P = .03). INTERPRETATION: An interaction between age and benefits of early antibiotics for sepsis has not been reported before. When validated, it can have major implications for clinical practice. This new insight into benefits of early antibiotic treatment for younger sepsis patients may enable more effective care.

The Value of Artificial Intelligence in Laboratory Medicine.

OBJECTIVES: As laboratory medicine continues to undergo digitalization and automation, clinical laboratorians will likely be confronted with the challenges associated with artificial intelligence (AI). Understanding what AI is good for, how to evaluate it, what are its limitations, and how it can be implemented are not well understood. With a survey, we aimed to evaluate the thoughts of stakeholders in laboratory medicine on the value of AI in the diagnostics space and identify anticipated challenges and solutions to introducing AI. METHODS: We conducted a web-based survey on the use of AI with participants from Roche's Strategic Advisory Network that included key stakeholders in laboratory medicine. RESULTS: In total, 128 of 302 stakeholders responded to the survey. Most of the participants were medical practitioners (26%) or laboratory managers (22%). AI is currently used in the organizations of 15.6%, while 66.4% felt they might use it in the future. Most had an unsure attitude on what they would need to adopt AI in the diagnostics space. High investment costs, lack of proven clinical benefits, number of decision makers, and privacy concerns were identified as barriers to adoption. Education in the value of AI, streamlined implementation and integration into existing workflows, and research to prove clinical utility were identified as solutions needed to mainstream AI in laboratory medicine. CONCLUSIONS: This survey demonstrates that specific knowledge of AI in the medical community is poor and that AI education is much needed. One strategy could be to implement new AI tools alongside existing tools.

Development of a "meta-model" to address missing data, predict patient-specific cancer survival and provide a foundation for clinical decision support.

OBJECTIVE: Like most real-world data, electronic health record (EHR)-derived data from oncology patients typically exhibits wide interpatient variability in terms of available data elements. This interpatient variability leads to missing data and can present critical challenges in developing and implementing predictive models to underlie clinical decision support for patient-specific oncology care. Here, we sought to develop a novel ensemble approach to addressing missing data that we term the "meta-model" and apply the meta-model to patient-specific cancer prognosis. MATERIALS AND METHODS: Using real-world data, we developed a suite of individual random survival forest models to predict survival in patients with advanced lung cancer, colorectal cancer, and breast cancer. Individual models varied by the predictor data used. We combined models for each cancer type into a meta-model that predicted survival for each patient using a weighted mean of the individual models for which the patient had all requisite predictors. RESULTS: The meta-model significantly outperformed many of the individual models and performed similarly to the best performing individual models. Comparisons of the meta-model to a more traditional imputation-based method of addressing missing data supported the meta-model's utility. CONCLUSIONS: We developed a novel machine learning-based strategy to underlie clinical decision support and predict survival in cancer patients, despite missing data. The meta-model may more generally provide a tool for addressing missing data across a variety of clinical prediction problems. Moreover, the meta-model may address other challenges in clinical predictive modeling including model extensibility and integration of predictive algorithms trained across different institutions and datasets.

Short Keynote Paper: Mainstreaming Personalized Healthcare-Transforming Healthcare Through New Era of Artificial Intelligence.

Medicine has entered the digital era, driven by data from new modalities, especially genomics and imaging, as well as new sources such as wearables and Internet of Things. As we gain a deeper understanding of the disease biology and how diseases affect an individual, we are developing targeted therapies to personalize treatments. There is a need for technologies like Artificial Intelligence (AI) to be able to support predictions for personalized treatments. In order to mainstream AI in healthcare we will need to address issues such as explainability, liability and privacy. Developing explainable algorithms and including AI training in medical education are many of the solutions that can help alleviate these concerns.

Introducing Artificial Intelligence Training in Medical Education.

Health care is evolving and with it the need to reform medical education. As the practice of medicine enters the age of artificial intelligence (AI), the use of data to improve clinical decision making will grow, pushing the need for skillful medicine-machine interaction. As the rate of medical knowledge grows, technologies such as AI are needed to enable health care professionals to effectively use this knowledge to practice medicine. Medical professionals need to be adequately trained in this new technology, its advantages to improve cost, quality, and access to health care, and its shortfalls such as transparency and liability. AI needs to be seamlessly integrated across different aspects of the curriculum. In this paper, we have addressed the state of medical education at present and have recommended a framework on how to evolve the medical education curriculum to include AI.