Use of Community SARS-CoV-2 Case Counts and Instantaneous Reproductive Number to Predict Short-Term COVID-19 Hospital Admission Volumes.

The severe acute respiratory syndrome (SARS-CoV-2) pandemic and high hospitalization rates placed a tremendous strain on hospital resources, necessitating the use of models to predict hospital volumes and the associated resource requirements. Complex epidemiologic models have been developed and published, but many require continued adjustment of input parameters. We developed a simplified model for short-term bed need predictions that self-adjusts to changing patterns of disease in the community and admission rates. The model utilizes public health data on community new case counts for SARS-CoV-2 and projects anticipated hospitalization rates. The model was retrospectively evaluated after the second wave of SARS-CoV-2 in New York, New York (October 2020-April 2021) for its accuracy in predicting numbers of coronavirus disease 2019 (COVID-19) admissions 3, 5, 7, and 10 days into the future, comparing predicted admissions with actual admissions for each day at a large integrated health-care delivery network. The mean absolute percent error of the model was found to be low when evaluated across the entire health system, for a single region of the health system or for a single large hospital (6.1%-7.6% for 3-day predictions, 9.2%-10.4% for 5-day predictions, 12.4%-13.2% for 7-day predictions, and 17.1%-17.8% for 10-day predictions).

Application of a Machine Learning Algorithm to Develop and Validate a Prediction Model for Ambulatory Non-Arrivals.

BACKGROUND: Non-arrivals to scheduled ambulatory visits are common and lead to a discontinuity of care, poor health outcomes, and increased subsequent healthcare utilization. Reducing non-arrivals is important given their association with poorer health outcomes and cost to health systems. OBJECTIVE: To develop and validate a prediction model for ambulatory non-arrivals. DESIGN: Retrospective cohort study. PATIENTS OR SUBJECTS: Patients at an integrated health system who had an outpatient visit scheduled from January 1, 2020, to February 28, 2022. MAIN MEASURES: Non-arrivals to scheduled appointments. KEY RESULTS: There were over 4.3 million ambulatory appointments from 1.2 million adult patients. Patients with appointment non-arrivals were more likely to be single, racial/ethnic minorities, and not having an established primary care provider compared to those who arrived at their appointments. A prediction model using the XGBoost machine learning algorithm had the highest AUC value (0.768 [0.767-0.770]). Using SHAP values, the most impactful features in the model include rescheduled appointments, lead time (number of days from scheduled to appointment date), appointment provider, number of days since last appointment with the same department, and a patient's prior appointment status within the same department. Scheduling visits close to an appointment date is predicted to be less likely to result in a non-arrival. Overall, the prediction model calibrated well for each department, especially over the operationally relevant probability range of 0 to 40%. Departments with fewer observations and lower non-arrival rates generally had a worse calibration. CONCLUSIONS: Using a machine learning algorithm, we developed a prediction model for non-arrivals to scheduled ambulatory appointments usable for all medical specialties. The proposed prediction model can be deployed within an electronic health system or integrated into other dashboards to reduce non-arrivals. Future work will focus on the implementation and application of the model to reduce non-arrivals.

Automated Identification of Patients with Advanced Illness.

Early identification of advanced illness patients within an inpatient population is essential in order to establish the patient's goals of care. Having goals of care conversations enables hospital patients to dictate a plan for care in concordance with their values and wishes. These conversations allow a patient to maintain some control, rather than be subjected to a default care process that may not be desired and may not provide benefit. In this study the performance of two approaches which identify advanced illness patients within an inpatient population were evaluated: LACE (a rule-based approach that uses L - Length of stay, A- Acuity of Admission, C- Co-morbidities, E- Emergency room visits), and a novel approach: Hospital Impairment Score (HIS). The Hospital impairment score is derived by leveraging both rule-based insights and a novel machine learning algorithm. It was identified that HIS significantly outperformed the LACE score, the current model being used in production at Northwell Health. Furthermore, we describe how the HIS model was piloted at a single hospital, was launched into production, and is being successfully used by clinicians at that hospital.

Hospital-Acquired Bloodstream Infections in Patients Hospitalized With Severe Acute Respiratory Syndrome Coronavirus 2 Infection (Coronavirus Disease 2019): Association With Immunosuppressive Therapies.

BACKGROUND: Immunosuppressive therapies proposed for Coronavirus disease 2019 (COVID-19) management may predispose to secondary infections. We evaluated the association of immunosuppressive therapies with bloodstream-infections (BSIs) in hospitalized COVID-19 patients. METHODS: This was an institutional review board-approved retrospective, multicenter, cohort study of adults hospitalized with COVID-19 over a 5-month period. We obtained clinical, microbiologic and laboratory data from electronic medical records. Propensity-score-matching helped create balanced exposure groups. Demographic characteristics were compared across outcome groups (BSI/no BSI) using two-sample t-test and Chi-Square test for continuous and categorical variables respectively, while immunosuppressive therapy use was compared using McNemar's test. Conditional logistic regression helped assess the association between immunosuppressive therapies and BSIs. RESULTS: 13,007 patients were originally included, with propensity-score-matching producing a sample of 6,520 patients. 3.74% and 3.97% were diagnosed with clinically significant BSIs in the original and propensity-score-matched populations respectively. COVID-19 patients with BSIs had significantly longer hospitalizations, higher intensive care unit admission and mortality rates compared to those without BSIs. On univariable analysis, combinations of corticosteroids/anakinra [odds-ratio (OR) 2.00, 95% confidence intervals (C.I.) 1.05-3.80, P value.0342] and corticosteroids/tocilizumab [OR 2.13, 95% C.I. 1.16-3.94, P value .0155] were significantly associated with BSIs. On multivariable analysis (adjusting for confounders), combination corticosteroids/tocilizumab were significantly associated with any BSI [OR 1.97, 95% C.I. 1.04-3.73, P value.0386] and with bacterial BSIs [OR 2.13, 95% C.I. 1.12-4.05, p-value 0.0217]. CONCLUSIONS: Combination immunosuppressive therapies were significantly associated with BSI occurrence in COVID-19 patients; their use warrants increased BSI surveillance. Further studies are needed to establish their causative role.

Design, development, and deployment of an indication- and kidney function-based decision support tool to optimize treatment and reduce medication dosing errors.

Delivering clinical decision support (CDS) at the point of care has long been considered a major advantage of computerized physician order entry (CPOE). Despite the widespread implementation of CPOE, medication ordering errors and associated adverse events still occur at an unacceptable level. Previous attempts at indication- and kidney function-based dosing have mostly employed intrusive CDS, including interruptive alerts with poor usability. This descriptive work describes the design, development, and deployment of the Adult Dosing Methodology (ADM) module, a novel CDS tool that provides indication- and kidney-based dosing at the time of order entry. Inclusion of several antimicrobials in the initial set of medications allowed for the additional goal of optimizing therapy duration for appropriate antimicrobial stewardship. The CDS aims to decrease order entry errors and burden on providers by offering automatic dose and frequency recommendations, integration within the native electronic health record, and reasonable knowledge maintenance requirements. Following implementation, early utilization demonstrated high acceptance of automated recommendations, with up to 96% of provided automated recommendations accepted by users.

Examination of Post-discharge Follow-up Appointment Status and 30-Day Readmission.

BACKGROUND: Post-hospital discharge follow-up appointments are intended to evaluate patients' recovery following a hospitalization, but it is unclear how appointment statuses are associated with readmissions. OBJECTIVE: To examine the association between post-discharge ambulatory follow-up status, (1) having a scheduled appointment and (2) arriving to said appointment, and 30-day readmission. DESIGN AND SETTING: A retrospective cohort study of patients hospitalized at 12 hospitals in an Integrated Delivery Network and their ambulatory appointments in that same network. PATIENTS AND MAIN MEASURES: We included 50,772 patients who had an ambulatory appointment within 18 months of an inpatient admission in 2018. Primary outcome was readmission within 30 days post-discharge. KEY RESULTS: There were 32,108 (63.2%) patients with scheduled follow-up appointments and 18,664 (36.8%) patients with no follow-up; 28,313 (88.2%) patients arrived, 3149 (9.8%) missed, and 646 (2.0%) were readmitted prior to their scheduled appointments. Overall 30-day readmission rate was 7.3%; 6.0% [5.75-6.31] for those who arrived, 8.8% [8.44-9.25] for those without follow-up, and 10.3% [9.28-11.40] for those who missed a scheduled appointment (p < 0.001). After adjusting for covariates, patients who arrived at their appointment in the first week following discharge were significantly less likely to be readmitted than those not having any follow-up scheduled (medical adjusted hazard ratio (aHR) 0.57 [0.47-0.69], p < 0.001; surgical aHR 0.58 [0.44-0.75], p < 0.001) There was an increased risk at weeks 3 and 4 for medical patients who arrived at a follow-up compared to those with no follow-up scheduled (week 3 aHR 1.29 [1.10-1.51], p = 0.001; week 4 aHR 1.46 [1.26-1.70], p < 0.001). CONCLUSIONS: The benefit of patients arriving to their post-discharge appointments compared with patients who missed their follow-up visits or had no follow-up scheduled, is only significant during first week post-discharge, suggesting that coordination within 1 week of discharge is critical in reducing 30-day readmissions.

Let Sleeping Patients Lie, avoiding unnecessary overnight vitals monitoring using a clinically based deep-learning model.

Impaired sleep for hospital patients is an all too common reality. Sleep disruptions due to unnecessary overnight vital sign monitoring are associated with delirium, cognitive impairment, weakened immunity, hypertension, increased stress, and mortality. It is also one of the most common complaints of hospital patients while imposing additional burdens on healthcare providers. Previous efforts to forgo overnight vital sign measurements and improve patient sleep used providers' subjective stability assessment or utilized an expanded, thus harder to retrieve, set of vitals and laboratory results to predict overnight clinical risk. Here, we present a model that incorporates past values of a small set of vital signs and predicts overnight stability for any given patient-night. Using data obtained from a multi-hospital health system between 2012 and 2019, a recurrent deep neural network was trained and evaluated using ~2.3 million admissions and 26 million vital sign assessments. The algorithm is agnostic to patient location, condition, and demographics, and relies only on sequences of five vital sign measurements, a calculated Modified Early Warning Score, and patient age. We achieved an area under the receiver operating characteristic curve of 0.966 (95% confidence interval [CI] 0.956-0.967) on the retrospective testing set, and 0.971 (95% CI 0.965-0.974) on the prospective set to predict overnight patient stability. The model enables safe avoidance of overnight monitoring for ~50% of patient-nights, while only misclassifying 2 out of 10,000 patient-nights as stable. Our approach is straightforward to deploy, only requires regularly obtained vital signs, and delivers easily actionable clinical predictions for a peaceful sleep in hospitals.

An approach to predicting patient experience through machine learning and social network analysis.

OBJECTIVE: Improving the patient experience has become an essential component of any healthcare system's performance metrics portfolio. In this study, we developed a machine learning model to predict a patient's response to the Hospital Consumer Assessment of Healthcare Providers and Systems survey's "Doctor Communications" domain questions while simultaneously identifying most impactful providers in a network. MATERIALS AND METHODS: This is an observational study of patients admitted to a single tertiary care hospital between 2016 and 2020. Using machine learning algorithms, electronic health record data were used to predict patient responses to Hospital Consumer Assessment of Healthcare Providers and Systems survey questions in the doctor domain, and patients who are at risk for responding negatively were identified. Model performance was assessed by area under receiver-operating characteristic curve. Social network analysis metrics were also used to identify providers most impactful to patient experience. RESULTS: Using a random forest algorithm, patients' responses to the following 3 questions were predicted: "During this hospital stay how often did doctors. 1) treat you with courtesy and respect? 2) explain things in a way that you could understand? 3) listen carefully to you?" with areas under the receiver-operating characteristic curve of 0.876, 0.819, and 0.819, respectively. Social network analysis found that doctors with higher centrality appear to have an outsized influence on patient experience, as measured by rank in the random forest model in the doctor domain. CONCLUSIONS: A machine learning algorithm identified patients at risk of a negative experience. Furthermore, a doctor social network framework provides metrics for identifying those providers that are most influential on the patient experience.

Higher Imaging Yield When Clinical Decision Support Is Used.

OBJECTIVE: Increased utilization of CT pulmonary angiography (CTPA) for the evaluation of pulmonary embolism has been associated with decreasing diagnostic yields and rising concerns about the harms of unnecessary testing. The objective of this study was to determine whether clinical decision support (CDS) use would be associated with increased imaging yields after controlling for selection bias. METHODS: We performed a retrospective cohort study in the emergency departments of two tertiary care hospitals of all CTPAs performed between August 2015 and September 2018. Providers ordering a CTPA are routed to an optional CDS tool, which allows them to use Wells' Criteria for pulmonary embolism. After propensity score matching, CTPA yield was calculated for the CDS-use and CDS-dismissal groups and stratified by provider type. RESULTS: A total of 7,367 CTPAs were ordered during the study period. Of those, providers used the CDS tool in 2,568 (35%) cases and did not use the tool in 4,799 (65%) of cases. After propensity score matching, CTPA yield was 11.99% in the CDS-use group and 8.70% in the CDS-dismissal group (P < .001). Attending physicians, residents, and physician assistant CDS users demonstrated a 56.5% (P = .006), 38.7% (P = .01), and 16.7% (P = .03) increased yield compared with those who dismissed the tool, respectively. DISCUSSION: Diagnostic yield was 38% higher for CTPAs when the provider used the CDS tool, after controlling for selection bias. Yields were higher for every provider type. Further research is needed to discover successful strategies to increase provider use of these important tools.

THE HIGH COST OF LOW VALUE CARE.

The main focus of this study is bridging the "evidence gap" between frontline decision-making in health care and the actual evidence, with the hope of reducing unnecessary diagnostic testing and treatments. From our work in pulmonary embolism (PE) and over ordering of computed tomography pulmonary angiography, we integrated the highly validated Wells' criteria into the electronic health record at two of our major academic tertiary hospitals. The Wells' clinical decision support tool triggered for patients being evaluated for PE and therefore determined a patients' pretest probability for having a PE. There were 12,759 patient visits representing 11,836 patients, 51% had no D-dimer, 41% had a negative D-dimer, and 9% had a positive D-dimer. Our study gave us an opportunity to determine which patients were very low probabilities for PE, with no need for further testing.

Clinical Factors Associated with Readmission among Patients with Lower Limb Cellulitis.

BACKGROUND: There is substantial allocation of resources directed towards evaluation and management of lower limb cellulitis (LLC) in the acute care setting. Readmission for LLC is poorly understood, and there is little evidence with which to identify patients at risk for readmission. OBJECTIVE: To describe demographics, comorbidities, admission vital signs, and laboratory markers of infection among patients with LLC who are readmitted, and to investigate which among these factors is associated with readmission. METHODS: A cross-sectional retrospective cohort study was performed at tertiary and community hospitals within a regional health care system in order to summarize readmission characteristics. Univariate and multivariate models were created to estimate the likelihood of independent variables being associated with LLC readmission. RESULTS: The readmission rate was 11.2% with a median age of 68.6 years for the cohort. Increased age and subsidized insurance were associated with more frequent admissions. For every 10-year age increase, cellulitis subjects had a 14% increase in readmission odds (OR 1.14, CI 1.07-1.20). Patients with subsidized insurance had an almost twofold increased risk (OR 1.88, CI 1.42-2.50). Smoking, obesity, hypertension, diabetes mellitus, renal insufficiency, tachycardia, hypotension, leukocytosis, and neutrophilia were not more frequent in readmitted patients. CONCLUSIONS: Older age and subsidized insurance were associated with readmission whereas severity indicators for infection including abnormal vital signs and laboratory markers were not significantly associated. Factors other than severity of infection, such as socioeconomic factors, may influence clinical decisions related to readmission for LLC.

Clinical characteristics associated with days to discharge among patients admitted with a primary diagnosis of lower limb cellulitis.

BACKGROUND: Clinicians have limited ability to classify risk of prolonged hospitalization among patients with lower limb cellulitis. OBJECTIVE: We sought to identify characteristics associated with days to discharge and prolonged stay. METHODS: We conducted retrospective cohort analysis including patients admitted with a primary diagnosis of lower limb cellulitis at community and tertiary hospitals. RESULTS: There were 4224 admissions for lower limb cellulitis among 3692 patients. Mean age of the cohort was 64.4 years. Frequencies of tobacco smoking, obesity, and diabetes mellitus were 25.1%, 44.9%, and 19.3%, respectively. Patients having decreased likelihood of discharge included those with the following: 10-year age increments 0.90 (95% confidence interval [CI] 0.88-0.92), obesity 0.90 (95% CI 0.83-0.97), diabetes mellitus 0.90 (95% CI 0.82-0.98), tachycardia 0.76 (95% CI 0.67-0.85), hypotension 0.77 (95% CI 0.65-0.90), leukocytosis 0.86 (95% CI 0.79-0.93), neutrophilia 0.80 (95% CI 0.73-0.87), elevated serum creatinine 0.74 (95% CI 0.68-0.81), and low serum bicarbonate 0.84 (95% CI 0.75-0.95). LIMITATIONS: This analysis is retrospective and based on coded data. Unknown confounding variables may also influence prolonged stay. CONCLUSIONS: Patients with lower limb cellulitis and prolonged stay have a number of clinical characteristics which may be used to classify risk for prolonged stay.

ClinRefLink: implementation of infobutton-like functionality in a commercial clinical information system incorporating concepts from textual documents.

Clinical information systems offer an opportunity to provide clinicians with medical reference materials during clinical encounters when the information is most beneficial. Implementation of this "Infobutton" concept has been described by a number of institutions with locally developed clinical information systems and electronic medical records. This article describes the development of an infobutton-like application called ClinRefLink embedded within a commercial clinical information system. ClinRefLink is somewhat unique in that it offers clinicians the option to perform reference searches based on clinical entities identified within narrative documents. In the first 30 days after implementation, 1018 reference searches were performed. The characteristics of the clinicians and the clinical context of the search terms are described. These data support the value of clinical term extraction from narrative documents as a component of an infobutton system.

A framework for analyzing the cognitive complexity of computer-assisted clinical ordering.

Computer-assisted provider order entry is a technology that is designed to expedite medical ordering and to reduce the frequency of preventable errors. This paper presents a multifaceted cognitive methodology for the characterization of cognitive demands of a medical information system. Our investigation was informed by the distributed resources (DR) model, a novel approach designed to describe the dimensions of user interfaces that introduce unnecessary cognitive complexity. This method evaluates the relative distribution of external (system) and internal (user) representations embodied in system interaction. We conducted an expert walkthrough evaluation of a commercial order entry system, followed by a simulated clinical ordering task performed by seven clinicians. The DR model was employed to explain variation in user performance and to characterize the relationship of resource distribution and ordering errors. The analysis revealed that the configuration of resources in this ordering application placed unnecessarily heavy cognitive demands on the user, especially on those who lacked a robust conceptual model of the system. The resources model also provided some insight into clinicians' interactive strategies and patterns of associated errors. Implications for user training and interface design based on the principles of human-computer interaction in the medical domain are discussed.

Impact of a computerized alert during physician order entry on medication dosing in patients with renal impairment.

Computerized assistance to clinicians during physician order entry can provide protection against medical errors. However, computer systems that provide too much assistance may adversely affect training of medical students and residents. Trainees may rely on the computer to automatically perform complex calculations and create appropriate orders and are thereby deprived of an important educational exercise. An alternative strategy is to provide a critique at the completion of an order, requiring the trainee to enter the entire order but displaying an alert if an error is made. While this approach preserves the educational components of order-writing, the potential for errors exists if the computerized critique does not induce clinicians to correct the order. The goal of this study was to determine (a) the frequency with which errors are made by trainees in an environment in which renal dosing adjustment calculation for antimicrobials are done by the system after the user has entered an order, and (b) the frequency with which prompts to clinicians regarding these errors leads to correction of those orders.