Design, validation and implementation of an automated e-alert for acute kidney injury: 6-month pilot study shows increased awareness.

BACKGROUND: Acute kidney injury (AKI) is defined as a sudden episode of kidney failure but is known to be under-recognized by healthcare professionals. The Kidney Disease Improving Global Outcome (KDIGO) guidelines have formulated criteria to facilitate AKI diagnosis by comparing changes in plasma creatinine measurements (PCr). To improve AKI awareness, we implemented these criteria as an electronic alert (e-alert), in our electronic health record (EHR) system. METHODS: For every new PCr measurement measured in the University Medical Center Utrecht that triggered the e-alert, we provided the physician with actionable insights in the form of a memo, to improve or stabilize kidney function. Since e-alerts qualify for software as a medical device (SaMD), we designed, implemented and validated the e-alert according to the European Union In Vitro Diagnostic Regulation (IVDR). RESULTS: We evaluated the impact of the e-alert using pilot data six months before and after implementation. 2,053 e-alerts of 866 patients were triggered in the before implementation, and 1,970 e-alerts of 853 patients were triggered after implementation. We found improvements in AKI awareness as measured by (1) 2 days PCr follow up (56.6-65.8%, p-value: 0.003), and (2) stop of nephrotoxic medication within 7 days of the e-alert (59.2-63.2%, p-value: 0.002). CONCLUSION: Here, we describe the design and implementation of the e-alert in line with the IVDR, leveraging a multi-disciplinary team consisting of physicians, clinical chemists, data managers and data scientists, and share our firsts results that indicate an improved awareness among treating physicians.

Added diagnostic value of routinely measured hematology variables in diagnosing immune checkpoint inhibitor mediated toxicity in the emergency department.

BACKGROUND: Immune checkpoint inhibitors (ICI) show remarkable results in cancer treatment, but at the cost of immune-related adverse events (irAE). irAE can be difficult to differentiate from infections or tumor progression, thereby challenging treatment, especially in the emergency department (ED) where time and clinical information are limited. As infections are traceable in blood, we were interested in the added diagnostic value of routinely measured hematological blood cell characteristics in addition to standard diagnostic practice in the ED to aid irAE assessment. METHODS: Hematological variables routinely measured with our hematological analyzer (Abbott CELL-DYN Sapphire) were retrieved from Utrecht Patient Oriented Database (UPOD) for all patients treated with ICI who visited the ED between 2013 and 2020. To assess the added diagnostic value, we developed and compared two models; a base logistic regression model trained on the preliminary diagnosis at the ED, sex, and gender, and an extended model trained with lasso that also assessed the hematology variables. RESULTS: A total of 413 ED visits were used in this analysis. The extended model showed an improvement in performance (area under the receiver operator characteristic curve) over the base model, 0.79 (95% CI 0.75-0.84), and 0.67 (95% CI 0.60-0.73), respectively. Two standard blood count variables (eosinophil granulocyte count and red blood cell count) and two advanced variables (coefficient of variance of neutrophil depolarization and red blood cell distribution width) were associated with irAE. CONCLUSION: Hematological variables are a valuable and inexpensive aid for irAE diagnosis in the ED. Further exploration of the predictive hematological variables could yield new insights into the pathophysiology underlying irAE and in distinguishing irAE from other inflammatory conditions.

A machine learning approach using endpoint adjudication committee labels for the identification of sepsis predictors at the emergency department.

Accurate sepsis diagnosis is paramount for treatment decisions, especially at the emergency department (ED). To improve diagnosis, clinical decision support (CDS) tools are being developed with machine learning (ML) algorithms, using a wide range of variable groups. ML models can find patterns in Electronic Health Record (EHR) data that are unseen by the human eye. A prerequisite for a good model is the use of high-quality labels. Sepsis gold-standard labels are hard to define due to a lack of reliable diagnostic tools for sepsis at the ED. Therefore, standard clinical tools, such as clinical prediction scores (e.g. modified early warning score and quick sequential organ failure assessment), and claims-based methods (e.g. ICD-10) are used to generate suboptimal labels. As a consequence, models trained with these "silver" labels result in ill-trained models. In this study, we trained ML models for sepsis diagnosis at the ED with labels of 375 ED visits assigned by an endpoint adjudication committee (EAC) that consisted of 18 independent experts. Our objective was to evaluate which routinely measured variables show diagnostic value for sepsis. We performed univariate testing and trained multiple ML models with 95 routinely measured variables of three variable groups; demographic and vital, laboratory and advanced haematological variables. Apart from known diagnostic variables, we identified added diagnostic value for less conventional variables such as eosinophil count and platelet distribution width. In this explorative study, we show that the use of an EAC together with ML can identify new targets for future sepsis diagnosis research.

A Wolf in Sheep's Clothing: Reuse of Routinely Obtained Laboratory Data in Research.

Electronic health records (EHRs) contain valuable data for reuse in science, quality evaluations, and clinical decision support. Because routinely obtained laboratory data are abundantly present, often numeric, generated by certified laboratories, and stored in a structured way, one may assume that they are immediately fit for (re)use in research. However, behind each test result lies an extensive context of choices and considerations, made by both humans and machines, that introduces hidden patterns in the data. If they are unaware, researchers reusing routine laboratory data may eventually draw incorrect conclusions. In this paper, after discussing health care system characteristics on both the macro and micro level, we introduce the reader to hidden aspects of generating structured routine laboratory data in 4 steps (ordering, preanalysis, analysis, and postanalysis) and explain how each of these steps may interfere with the reuse of routine laboratory data. As researchers reusing these data, we underline the importance of domain knowledge of the health care professional, laboratory specialist, data manager, and patient to turn routine laboratory data into meaningful data sets to help obtain relevant insights that create value for clinical care.

Ambiguous definitions for baseline serum creatinine affect acute kidney diagnosis at the emergency department.

BACKGROUND: Acute kidney injury (AKI) incidence is increasing, however AKI is often missed at the emergency department (ED). AKI diagnosis depends on changes in kidney function by comparing a serum creatinine (SCr) measurement to a baseline value. However, it remains unclear to what extent different baseline values may affect AKI diagnosis at ED. METHODS: Routine care data from ED visits between 2012 and 2019 were extracted from the Utrecht Patient Oriented Database. We evaluated baseline definitions with criteria from the RIFLE, AKIN and KDIGO guidelines. We evaluated four baseline SCr definitions (lowest, most recent, mean, median), as well as five different time windows (up to 365 days prior to ED visit) to select a baseline and compared this to the first measured SCr at ED. As an outcome, we assessed AKI prevalence at ED. RESULTS: We included 47,373 ED visits with both SCr-ED and SCr-BL available. Of these, 46,100 visits had a SCr-BL from the - 365/- 7 days time window. Apart from the lowest value, AKI prevalence remained similar for the other definitions when varying the time window. The lowest value with the - 365/- 7 time window resulted in the highest prevalence (21.4%). Importantly, applying the guidelines with all criteria resulted in major differences in prevalence ranging from 5.9 to 24.0%. CONCLUSIONS: AKI prevalence varies with the use of different baseline definitions in ED patients. Clinicians, as well as researchers and developers of automatic diagnostic tools should take these considerations into account when aiming to diagnose AKI in clinical and research settings.

Transportability and Implementation Challenges of Early Warning Scores for Septic Shock in the ICU: A Perspective on the TREWScore.

The increased use of electronic health records (EHRs) has improved the availability of routine care data for medical research. Combined with machine learning techniques this has spurred the development of early warning scores (EWSs) in hospitals worldwide. EWSs are commonly used in the hospital where they have been developed, yet few have been transported to external settings and/or internationally. In this perspective, we describe our experiences in implementing the TREWScore, a septic shock EWS, and the transportability challenges regarding domain, predictors, and clinical outcome we faced. We used data of 53,330 ICU stays from Medical Information Mart for Intensive Care-III (MIMIC-III) and 18,013 ICU stays from the University Medical Center (UMC) Utrecht, including 17,023 (31.9%) and 2,557 (14.2%) cases of sepsis, respectively. The MIMIC-III and UMC populations differed significantly regarding the length of stay (6.9 vs. 9.0 days) and hospital mortality (11.6% vs. 13.6%). We mapped all 54 TREWScore predictors to the UMC database: 31 were readily available, seven required unit conversion, 14 had to be engineered, one predictor required text mining, and one predictor could not be mapped. Lastly, we classified sepsis cases for septic shock using the sepsis-2 criteria. Septic shock populations (UMC 31.3% and MIMIC-III 23.3%) and time to shock events showed significant differences between the two cohorts. In conclusion, we identified challenges to transportability and implementation regarding domain, predictors, and clinical outcome when transporting EWS between hospitals across two continents. These challenges need to be systematically addressed to improve model transportability between centers and unlock the potential clinical utility of EWS.