Can a 5-to-90-day Mortality Predictor Perform Consistently Across Time and Equitably Across Populations?

Advance care planning (ACP) facilitates end-of-life care, yet many die without it. Timely and accurate mortality prediction may encourage ACP. However, performance of predictors typically differs among sub-populations (e.g., rural vs. urban) and worsens over time ("concept drift"). Therefore, we assessed performance equity and consistency for a novel 5-to-90-day mortality predictor across various demographies, geographies, and timeframes (n = 76,812 total encounters). Predictions were made for the first day of included adult inpatient admissions on a retrospective dataset. AUC-PR remained at 29% both pre-COVID (throughout 2018) and during COVID (8 months in 2021). Pre-COVID-19 recall and precision were 58% and 25% respectively at the 12.5% certainty cutoff, and 12% and 44% at the 37.5% cutoff. During COVID-19, recall and precision were 59% and 26% at the 12.5% cutoff, and 11% and 43% at the 37.5% cutoff. Pre-COVID, compared to the overall population, recall was lower at the 12.5% cutoff in the White, non-Hispanic subgroup and at both cutoffs in the rural subgroup. During COVID-19, precision at the 12.5% cutoff was lower than that of the overall population for the non-White and non-White female subgroups. No other significant differences were seen between subgroups and the corresponding overall population. Overall performance during COVID was unchanged from pre-pandemic performance. Although some comparisons (especially precision at the 37.5% cutoff) were underpowered, precision at the 12.5% cutoff was equitable across most demographies, regardless of the pandemic. Mortality prediction to prioritize ACP conversations can be provided consistently and equitably across many studied timeframes and sub-populations.

Derivation of a Unique, Algorithm-Based Approach to Cancer Patient Navigator Workload Management.

PURPOSE: Cancer patient navigators (CPNs) can decrease the time from diagnosis to treatment, but workloads vary widely, which may lead to burnout and less optimal navigation. Current practice for patient distribution among CPNs at our institution approximates random distribution. A literature search did not uncover previous reports of an automated algorithm to distribute patients to CPNs. We sought to develop an automated algorithm to fairly distribute new patients among CPNs specializing in the same cancer type(s) and assess its performance through simulation on a retrospective data set. METHODS: Using a 3-year data set, a proxy for CPN work was identified and multiple models were developed to predict the upcoming week's workload for each patient. An XGBoost-based predictor was retained on the basis of its superior performance. A distribution model was developed to fairly distribute new patients among CPNs within a specialty on the basis of predicted work needed. The predicted work included the week's predicted workload from a CPN's existing patients plus that of newly distributed patients to the CPN. Resulting workload unfairness was compared between predictor-informed and random distribution. RESULTS: Predictor-informed distribution significantly outperformed random distribution for equalizing weekly workloads across CPNs within a specialty. CONCLUSION: This derivation work demonstrates the feasibility of an automated model to distribute new patients more fairly than random assignment (with unfairness assessed using a workload proxy). Improved workload management may help reduce CPN burnout and improve navigation assistance for patients with cancer.

Novel Techniques to Assess Predictive Systems and Reduce Their Alarm Burden.

Machine prediction algorithms (e.g., binary classifiers) often are adopted on the basis of claimed performance using classic metrics such as precision and recall. However, classifier performance depends heavily upon the context (workflow) in which the classifier operates. Classic metrics do not reflect the realized performance of a predictor unless certain implicit assumptions are met, and these assumptions cannot be met in many common clinical scenarios. This often results in suboptimal implementations and in disappointment when expected outcomes are not achieved. One common failure mode for classic metrics arises when multiple predictions can be made for the same event, particularly when redundant true positive predictions produce little additional value. This describes many clinical alerting systems. We explain why classic metrics cannot correctly represent predictor performance in such contexts, and introduce an improved performance assessment technique using utility functions to score predictions based on their utility in a specific workflow context. The resulting utility metrics (u-metrics) explicitly account for the effects of temporal relationships and other sources of variability in prediction utility. Compared to traditional measures, u-metrics more accurately reflect the real-world costs and benefits of a predictor operating in a realized context. The improvement can be significant. We also describe a formal approach to snoozing, a mitigation strategy in which some predictions are suppressed to improve predictor performance by reducing false positives while retaining event capture. Snoozing is especially useful for predictors that generate interruptive alarms. U-metrics correctly measure and predict the performance benefits of snoozing, whereas traditional metrics do not.

Closing the Gap: A Comparison of Engagement Interventions to Achieve Equitable Breast Cancer Screening in Rural Illinois.

Mammography screening rates are typically lower in those with less economic advantage (EA). This study, conducted at an integrated health care system covering a mixed rurality population, assessed the ability of interventions (text messages linking to a Web microsite, digital health care workers, and a community health fair) to affect mammography screening rates and disparity in those rates among different EA populations. Payor type served as a proxy for greater (commercially insured) versus lower (Medicaid insured) EA. 4,342 subjects were included across the preintervention ("Pre") and postintervention ("Post") periods. Interventions were prospectively applied to all Medicaid subjects and randomly selected commercial subjects. Applying interventions only to lower EA subjects reversed the screening rate disparity (2.6% Pre vs. -3.7% Post, odds ratio [OR] 2.4 P < 0.01). When intervention arms ("Least," "More," "Most") were equally applied, screening rates in both EA groups significantly increased in the More arm (Medicaid OR = 2.04 P = 0.04, Commercial OR = 3.08 P < 0.01) and Most arm (Medicaid OR 2.57 P < 0.01, Commercial OR 2.33 P < 0.01), but not in the Least (text-only) arm (Medicaid OR 1.83 P = 0.11, Commercial OR 1.72 P = 0.09), although this text-only arm was inadequately powered to detect a difference. In summary, targeting interventions to those with lower EA reversed screening rate disparities, text messaging combined with other interventions improved screening rates in both groups, and future research is needed to determine whether interventions can simultaneously improve screening rates for all without worsening the disparity.

Delphi consensus on the feasibility of translating the ACEP clinical policies into computerized clinical decision support.

Clinical practice guidelines are developed to reduce variations in clinical practice, with the goal of improving health care quality and cost. However, evidence-based practice guidelines face barriers to dissemination, implementation, usability, integration into practice, and use. The American College of Emergency Physicians (ACEP) clinical policies have been shown to be safe and effective and are even cited by other specialties. In spite of the benefits of the ACEP clinical policies, implementation of these clinical practice guidelines into physician practice continues to be a challenge. Translation of the ACEP clinical policies into real-time computerized clinical decision support systems could help address these barriers and improve clinician decision making at the point of care. The investigators convened an emergency medicine informatics expert panel and used a Delphi consensus process to assess the feasibility of translating the current ACEP clinical policies into clinical decision support content. This resulting consensus document will serve to identify limitations to implementation of the existing ACEP Clinical Policies so that future clinical practice guideline development will consider implementation into clinical decision support at all stages of guideline development.

Case study: the health SmartLibrary experiences in web personalization and customization at the Galter health sciences library, Northwestern University.

QUESTION: How can the user's access to health information, especially full-text articles, be improved? The solution is building and evaluating the Health SmartLibrary (HSL). SETTING: The setting is the Galter Health Sciences Library, Feinberg School of Medicine, Northwestern University. METHOD: The HSL was built on web-based personalization and customization tools: My E-Resources, Stay Current, Quick Search, and File Cabinet. Personalization and customization data were tracked to show user activity with these value-added, online services. MAIN RESULTS: Registration data indicated that users were receptive to personalized resource selection and that the automated application of specialty-based, personalized HSLs was more frequently adopted than manual customization by users. Those who did customize customized My E-Resources and Stay Current more often than Quick Search and File Cabinet. Most of those who customized did so only once. CONCLUSION: Users did not always take advantage of the services designed to aid their library research experiences. When personalization is available at registration, users readily accepted it. Customization tools were used less frequently; however, more research is needed to determine why this was the case.

Indistinguishable from magic: health and wellness in a future of sufficiently advanced technology.

We describe a future in which health and wellness are transformed by (1) the availability of definitive and unambiguous tests to prove or disprove each diagnosis, (2) new methods based in systems biology to help unravel the web of messages transmitted across cellular and subcellular networks, and (3) universal access to data that has been freed from data silos to produce true data liquidity for a constellation of purposes ranging from personal health management to population health research. We believe the resulting "connected health" environment will have a profound impact on every aspect of modern life.

Searching electronic health records for temporal patterns in patient histories: a case study with microsoft amalga.

As electronic health records (EHR) become more widespread, they enable clinicians and researchers to pose complex queries that can benefit immediate patient care and deepen understanding of medical treatment and outcomes. However, current query tools make complex temporal queries difficult to pose, and physicians have to rely on computer professionals to specify the queries for them. This paper describes our efforts to develop a novel query tool implemented in a large operational system at the Washington Hospital Center (Microsoft Amalga, formerly known as Azyxxi). We describe our design of the interface to specify temporal patterns and the visual presentation of results, and report on a pilot user study looking for adverse reactions following radiology studies using contrast.

Cancer-associated neutropenic fever: clinical outcome and economic costs of emergency department care.

Purpose. Febrile neutropenia (FN) is a common, costly, and potentially fatal complication in oncology. While FN in the inpatient setting has been extensively studied, only one study has evaluated emergency department (ED) care for FN cancer patients. That study found that 96% of patients survived the complication. We evaluated clinical and economic outcomes for cancer patients with chemotherapy-associated FN treated in an ED. Methods. ED records for consecutive oncology patients with FN were reviewed for information on death, intensive care unit (ICU) use, blood cultures, and costs. Results. Forty-eight patients (n = 57 visits) were evaluated. Six patients died from FN (12%) and four received ICU care within 2 weeks and survived (8%). Blood cultures were positive for 37% of the ED visits. The median ED time was 3.3 hours. In 91% of visits, i.v. antibiotics were administered in the ED, ordered at a median of 1.7 hours from triage (interquartile range [IQR], 1.2-2.8 hours). All patients with death or ICU in 2 weeks and all but one patient with positive blood cultures received antibiotics. The median per patient ED costs were $1,455 (IQR, $1,300-$1,579)-42.4% for hospital/nursing, 23.5% for radiology, 20.8% for physician services, 10.9% for diagnostic tests, and 2.4% for antibiotics. Conclusions. Cancer patients with FN in this sample presenting to the ED frequently had no identified source of infection. One third of the patients had positive ED blood cultures and one fifth died or required ICU care within 2 weeks. Costs of ED care were similar to the cost of a single day of inpatient care. Disclosure of potential conflicts of interest is found at the end of this article.

Metrics in the science of surge.

Metrics are the driver to positive change toward better patient care. However, the research into the metrics of the science of surge is incomplete, research funding is inadequate, and we lack a criterion standard metric for identifying and quantifying surge capacity. Therefore, a consensus working group was formed through a "viral invitation" process. With a combination of online discussion through a group e-mail list and in-person discussion at a breakout session of the Academic Emergency Medicine 2006 Consensus Conference, "The Science of Surge," seven consensus statements were generated. These statements emphasize the importance of funded research in the area of surge capacity metrics; the utility of an emergency medicine research registry; the need to make the data available to clinicians, administrators, public health officials, and internal and external systems; the importance of real-time data, data standards, and electronic transmission; seamless integration of data capture into the care process; the value of having data available from a single point of access through which data mining, forecasting, and modeling can be performed; and the basic necessity of a criterion standard metric for quantifying surge capacity. Further consensus work is needed to select a criterion standard metric for quantifying surge capacity. These consensus statements cover the future research needs, the infrastructure needs, and the data that are needed for a state-of-the-art approach to surge and surge capacity.

Wireless handheld computers and voluntary utilization of computerized prescribing systems in the emergency department.

Illegible or invalid hand-written prescriptions can result in avoidable medical errors. Computer-based prescribing can mitigate the problem. An observational study was performed to examine the effect of wireless handheld computers (handhelds) on voluntary utilization of computerized prescribing within an Emergency Department. Handhelds with prescription-writing software were provided to physicians and the numbers of hand-written and computer-generated prescriptions were compared before and after the introduction of the handhelds. The resulting increase in computer-based prescribing was statistically significant and was observed largely among physicians who already used desktop computers for prescribing. The study concluded that handhelds increased voluntary utilization of computerized prescribing, but that the physicians most likely to use handhelds were those who already used desktop-based prescribing.

A real-time gesture interface for hands-free control of electronic medical records.

Whether attempting to review digital radiologic images during a procedure or reviewing labs on a clinical ward, computer keyboards and mice are potential sources for contamination of clinicians during sterile and non-sterile activities related to clinical care. The authors describe and demonstrate a live system prototype for hands-free, gesture-based control of an electronic medical record (EMR) system.

Automated patient facial image capture to reduce medical error.

The authors describe their experiences creating technology to automatically capture facial images from patients during triage and registration for integration into the electronic medical record (EMR) to reduce data retrieval and data entry errors. The prototype system was tested across a variety of ethnicities with facial images captured successfully in 100% of cases with a median time to capture of 0.75 seconds.

Improving a UMLS based allergy list for use in live electronic medical record systems.

The SNOMED allergy subset available through the UMLS has a variety of deficits that are substantial barriers to use in live clinical practice. These authors describe a method of enhancing a UMLS based allergy list by combining concepts from other terminologies found within the UMLS. This method resulted in a three-fold increase in the coverage allergy list compared to the standard SNOMED allergy subset.

Use of medical robotics in biothreat situations.

SARS, Avian Flu and other infectious and potentially highly transmissible diseases are threats to the entire healthcare workforce. Complete bio-isolation or the use of biohazard suits are not practical solutions for routine day-to-day patient-doctor interactions with highly infectious patients. The authors share their initial research experiences with utilizing medical robots for teleconferencing and other clinical activities to overcome these hurdles.

Automated medical image library creation for education.

The authors describe a method to create a medical teaching library that is automatically maintained, contains tens of thousands of radiologic images and is built using existing, internal, hospital dictations, radiologic images, and an off-the-shelf commercial search engine product (Google Inc.).