Multisite Pragmatic Cluster-Randomized Controlled Trial of the CONCERN Early Warning System.

Importance: Late predictions of hospitalized patient deterioration, resulting from early warning systems (EWS) with limited data sources and/or a care team's lack of shared situational awareness, contribute to delays in clinical interventions. The COmmunicating Narrative Concerns Entered by RNs (CONCERN) Early Warning System (EWS) uses real-time nursing surveillance documentation patterns in its machine learning algorithm to identify patients' deterioration risk up to 42 hours earlier than other EWSs. Objective: To test our a priori hypothesis that patients with care teams informed by the CONCERN EWS intervention have a lower mortality rate and shorter length of stay (LOS) than the patients with teams not informed by CONCERN EWS. Design: One-year multisite, pragmatic controlled clinical trial with cluster-randomization of acute and intensive care units to intervention or usual-care groups. Setting: Two large U.S. health systems. Participants: Adult patients admitted to acute and intensive care units, excluding those on hospice/palliative/comfort care, or with Do Not Resuscitate/Do Not Intubate orders. Intervention: The CONCERN EWS intervention calculates patient deterioration risk based on nurses' concern levels measured by surveillance documentation patterns, and it displays the categorical risk score (low, increased, high) in the electronic health record (EHR) for care team members. Main Outcomes and Measures: Primary outcomes: in-hospital mortality, LOS; survival analysis was used. Secondary outcomes: cardiopulmonary arrest, sepsis, unanticipated ICU transfers, 30-day hospital readmission. Results: A total of 60 893 hospital encounters (33 024 intervention and 27 869 usual-care) were included. Both groups had similar patient age, race, ethnicity, and illness severity distributions. Patients in the intervention group had a 35.6% decreased risk of death (adjusted hazard ratio [HR], 0.644; 95% confidence interval [CI], 0.532-0.778; P<.0001), 11.2% decreased LOS (adjusted incidence rate ratio, 0.914; 95% CI, 0.902-0.926; P<.0001), 7.5% decreased risk of sepsis (adjusted HR, 0.925; 95% CI, 0.861-0.993; P=.0317), and 24.9% increased risk of unanticipated ICU transfer (adjusted HR, 1.249; 95% CI, 1.093-1.426; P=.0011) compared with patients in the usual-care group. Conclusions and Relevance: A hospital-wide EWS based on nursing surveillance patterns decreased in-hospital mortality, sepsis, and LOS when integrated into the care team's EHR workflow. Trial Registration: ClinicalTrials.gov Identifier: NCT03911687 https://clinicaltrials.gov/ct2/show/NCT03911687. Key Points: Question: Do patients whose care team receive the CONCERN Early Warning System (EWS) intervention have a lower mortality rate and shorter length of stay than patients in the usual-care group?Findings: In this multisite, pragmatic cluster-randomized controlled clinical trial that included 60 893 hospital patient encounters, patients whose care team received the CONCERN EWS intervention had a 35.6% decreased risk of death and 11.2% shorter length of stay compared with those in the usual-care group.Meaning: A machine learning-based EWS modeled on nursing surveillance patterns significantly decreased the risk of inpatient deterioration events.

A Multilevel Primary Care Intervention to Improve Follow-Up of Overdue Abnormal Cancer Screening Test Results: A Cluster Randomized Clinical Trial.

Importance: Realizing the benefits of cancer screening requires testing of eligible individuals and processes to ensure follow-up of abnormal results. Objective: To test interventions to improve timely follow-up of overdue abnormal breast, cervical, colorectal, and lung cancer screening results. Design, Setting, and Participants: Pragmatic, cluster randomized clinical trial conducted at 44 primary care practices within 3 health networks in the US enrolling patients with at least 1 abnormal cancer screening test result not yet followed up between August 24, 2020, and December 13, 2021. Intervention: Automated algorithms developed using data from electronic health records (EHRs) recommended follow-up actions and times for abnormal screening results. Primary care practices were randomized in a 1:1:1:1 ratio to (1) usual care, (2) EHR reminders, (3) EHR reminders and outreach (a patient letter was sent at week 2 and a phone call at week 4), or (4) EHR reminders, outreach, and navigation (a patient letter was sent at week 2 and a navigator outreach phone call at week 4). Patients, physicians, and practices were unblinded to treatment assignment. Main Outcomes and Measures: The primary outcome was completion of recommended follow-up within 120 days of study enrollment. The secondary outcomes included completion of recommended follow-up within 240 days of enrollment and completion of recommended follow-up within 120 days and 240 days for specific cancer types and levels of risk. Results: Among 11 980 patients (median age, 60 years [IQR, 52-69 years]; 64.8% were women; 83.3% were White; and 15.4% were insured through Medicaid) with an abnormal cancer screening test result for colorectal cancer (8245 patients [69%]), cervical cancer (2596 patients [22%]), breast cancer (1005 patients [8%]), or lung cancer (134 patients [1%]) and abnormal test results categorized as low risk (6082 patients [51%]), medium risk (3712 patients [31%]), or high risk (2186 patients [18%]), the adjusted proportion who completed recommended follow-up within 120 days was 31.4% in the EHR reminders, outreach, and navigation group (n = 3455), 31.0% in the EHR reminders and outreach group (n = 2569), 22.7% in the EHR reminders group (n = 3254), and 22.9% in the usual care group (n = 2702) (adjusted absolute difference for comparison of EHR reminders, outreach, and navigation group vs usual care, 8.5% [95% CI, 4.8%-12.0%], P < .001). The secondary outcomes showed similar results for completion of recommended follow-up within 240 days and by subgroups for cancer type and level of risk for the abnormal screening result. Conclusions and Relevance: A multilevel primary care intervention that included EHR reminders and patient outreach with or without patient navigation improved timely follow-up of overdue abnormal cancer screening test results for breast, cervical, colorectal, and lung cancer. Trial Registration: ClinicalTrials.gov Identifier: NCT03979495.

Nursing Documentation Variation Across Different Medical Facilities Within an Integrated Healthcare System.

The purpose of this study was to demonstrate nursing documentation variation based on electronic health record design and its relationship with different levels of care by reviewing how various flowsheet measures, within the same electronic health record across an integrated healthcare system, are documented in different types of medical facilities. Flowsheet data with information on patients who were admitted to academic medical centers, community hospitals, and rehabilitation centers were used to calculate the frequency of flowsheet entries documented. We then compared the distinct flowsheet measures documented in five flowsheet templates across the different facilities. We observed that each type of healthcare facility appeared to have distinct clinical care foci and flowsheet measures documented differed within the same template based on facility type. Designing flowsheets tailored to study settings can meet the needs of end users and increase documentation efficiency by reducing time spent on unrelated flowsheet measures. Furthermore, this process can save nurses time for direct patient care.

Hospital Readmission and Social Risk Factors Identified from Physician Notes.

OBJECTIVE: To evaluate the prevalence of seven social factors using physician notes as compared to claims and structured electronic health records (EHRs) data and the resulting association with 30-day readmissions. STUDY SETTING: A multihospital academic health system in southeastern Massachusetts. STUDY DESIGN: An observational study of 49,319 patients with cardiovascular disease admitted from January 1, 2011, to December 31, 2013, using multivariable logistic regression to adjust for patient characteristics. DATA COLLECTION/EXTRACTION METHODS: All-payer claims, EHR data, and physician notes extracted from a centralized clinical registry. PRINCIPAL FINDINGS: All seven social characteristics were identified at the highest rates in physician notes. For example, we identified 14,872 patient admissions with poor social support in physician notes, increasing the prevalence from 0.4 percent using ICD-9 codes and structured EHR data to 16.0 percent. Compared to an 18.6 percent baseline readmission rate, risk-adjusted analysis showed higher readmission risk for patients with housing instability (readmission rate 24.5 percent; p < .001), depression (20.6 percent; p < .001), drug abuse (20.2 percent; p = .01), and poor social support (20.0 percent; p = .01). CONCLUSIONS: The seven social risk factors studied are substantially more prevalent than represented in administrative data. Automated methods for analyzing physician notes may enable better identification of patients with social needs.

A value set for documenting adverse reactions in electronic health records.

Objective: To develop a comprehensive value set for documenting and encoding adverse reactions in the allergy module of an electronic health record. Materials and Methods: We analyzed 2 471 004 adverse reactions stored in Partners Healthcare's Enterprise-wide Allergy Repository (PEAR) of 2.7 million patients. Using the Medical Text Extraction, Reasoning, and Mapping System, we processed both structured and free-text reaction entries and mapped them to Systematized Nomenclature of Medicine - Clinical Terms. We calculated the frequencies of reaction concepts, including rare, severe, and hypersensitivity reactions. We compared PEAR concepts to a Federal Health Information Modeling and Standards value set and University of Nebraska Medical Center data, and then created an integrated value set. Results: We identified 787 reaction concepts in PEAR. Frequently reported reactions included: rash (14.0%), hives (8.2%), gastrointestinal irritation (5.5%), itching (3.2%), and anaphylaxis (2.5%). We identified an additional 320 concepts from Federal Health Information Modeling and Standards and the University of Nebraska Medical Center to resolve gaps due to missing and partial matches when comparing these external resources to PEAR. This yielded 1106 concepts in our final integrated value set. The presence of rare, severe, and hypersensitivity reactions was limited in both external datasets. Hypersensitivity reactions represented roughly 20% of the reactions within our data. Discussion: We developed a value set for encoding adverse reactions using a large dataset from one health system, enriched by reactions from 2 large external resources. This integrated value set includes clinically important severe and hypersensitivity reactions. Conclusion: This work contributes a value set, harmonized with existing data, to improve the consistency and accuracy of reaction documentation in electronic health records, providing the necessary building blocks for more intelligent clinical decision support for allergies and adverse reactions.

Food entries in a large allergy data repository.

OBJECTIVE: Accurate food adverse sensitivity documentation in electronic health records (EHRs) is crucial to patient safety. This study examined, encoded, and grouped foods that caused any adverse sensitivity in a large allergy repository using natural language processing and standard terminologies. METHODS: Using the Medical Text Extraction, Reasoning, and Mapping System (MTERMS), we processed both structured and free-text entries stored in an enterprise-wide allergy repository (Partners' Enterprise-wide Allergy Repository), normalized diverse food allergen terms into concepts, and encoded these concepts using the Systematized Nomenclature of Medicine - Clinical Terms (SNOMED-CT) and Unique Ingredient Identifiers (UNII) terminologies. Concept coverage also was assessed for these two terminologies. We further categorized allergen concepts into groups and calculated the frequencies of these concepts by group. Finally, we conducted an external validation of MTERMS's performance when identifying food allergen terms, using a randomized sample from a different institution. RESULTS: We identified 158 552 food allergen records (2140 unique terms) in the Partners repository, corresponding to 672 food allergen concepts. High-frequency groups included shellfish (19.3%), fruits or vegetables (18.4%), dairy (9.0%), peanuts (8.5%), tree nuts (8.5%), eggs (6.0%), grains (5.1%), and additives (4.7%). Ambiguous, generic concepts such as "nuts" and "seafood" accounted for 8.8% of the records. SNOMED-CT covered more concepts than UNII in terms of exact (81.7% vs 68.0%) and partial (14.3% vs 9.7%) matches. DISCUSSION: Adverse sensitivities to food are diverse, and existing standard terminologies have gaps in their coverage of the breadth of allergy concepts. CONCLUSION: New strategies are needed to represent and standardize food adverse sensitivity concepts, to improve documentation in EHRs.

Content and functional specifications for a standards-based multidisciplinary rounding tool to maintain continuity across acute and critical care.

BACKGROUND: Maintaining continuity of care (CoC) in the inpatient setting is dependent on aligning goals and tasks with the plan of care (POC) during multidisciplinary rounds (MDRs). A number of locally developed rounding tools exist, yet there is a lack of standard content and functional specifications for electronic tools to support MDRs within and across settings. OBJECTIVE: To identify content and functional requirements for an MDR tool to support CoC. MATERIALS AND METHODS: We collected discrete clinical data elements (CDEs) discussed during rounds for 128 acute and critical care patients. To capture CDEs, we developed and validated an iPad-based observational tool based on informatics CoC standards. We observed 19 days of rounds and conducted eight group and individual interviews. Descriptive and bivariate statistics and network visualization were conducted to understand associations between CDEs discussed during rounds with a particular focus on the POC. Qualitative data were thematically analyzed. All analyses were triangulated. RESULTS: We identified the need for universal and configurable MDR tool views across settings and users and the provision of messaging capability. Eleven empirically derived universal CDEs were identified, including four POC CDEs: problems, plan, goals, and short-term concerns. Configurable POC CDEs were: rationale, tasks/'to dos', pending results and procedures, discharge planning, patient preferences, need for urgent review, prognosis, and advice/guidance. DISCUSSION: Some requirements differed between settings; yet, there was overlap between POC CDEs. CONCLUSIONS: We recommend an initial list of 11 universal CDEs for continuity in MDRs across settings and 27 CDEs that can be configured to meet setting-specific needs.

An evaluation of a natural language processing tool for identifying and encoding allergy information in emergency department clinical notes.

Emergency department (ED) visits due to allergic reactions are common. Allergy information is often recorded in free-text provider notes; however, this domain has not yet been widely studied by the natural language processing (NLP) community. We developed an allergy module built on the MTERMS NLP system to identify and encode food, drug, and environmental allergies and allergic reactions. The module included updates to our lexicon using standard terminologies, and novel disambiguation algorithms. We developed an annotation schema and annotated 400 ED notes that served as a gold standard for comparison to MTERMS output. MTERMS achieved an F-measure of 87.6% for the detection of allergen names and no known allergies, 90% for identifying true reactions in each allergy statement where true allergens were also identified, and 69% for linking reactions to their allergen. These preliminary results demonstrate the feasibility using NLP to extract and encode allergy information from clinical notes.

How many medication orders are entered through free-text in EHRs?--a study on hypoglycemic agents.

Computerized Provider Order Entry (CPOE) can reduce medication errors; however, its benefits are only achieved when data are entered in a structured format and entries are properly coded. This paper aims to explore the incidence of free-text medication order entries involving hypoglycemic agents in an ambulatory electronic health record (EHR) system with CPOE. Our results showed that free-text order entry continues to be frequent. During 2010, 9.3% of hypoglycemic agents were entered as free-text for 2,091 patients. 17.4% of the entries contained misspellings. The highest proportion of free-text entries were found in urgent care clinics (49.4%) and among registered nurses (31.5%). Additionally, 92 drug-drug interaction alerts were not triggered due to free-text entries. Only 25.9% of the patients had diabetes recorded in their problem list. Solutions will require policy to enforce structured entry, ongoing improvement in user-interface design, improved training for users, and strategies for maintaining a complete medication list.

Mapping Partners Master Drug Dictionary to RxNorm using an NLP-based approach.

OBJECTIVE: To develop an automated method based on natural language processing (NLP) to facilitate the creation and maintenance of a mapping between RxNorm and a local medication terminology for interoperability and meaningful use purposes. METHODS: We mapped 5961 terms from Partners Master Drug Dictionary (MDD) and 99 of the top prescribed medications to RxNorm. The mapping was conducted at both term and concept levels using an NLP tool, called MTERMS, followed by a manual review conducted by domain experts who created a gold standard mapping. The gold standard was used to assess the overall mapping between MDD and RxNorm and evaluate the performance of MTERMS. RESULTS: Overall, 74.7% of MDD terms and 82.8% of the top 99 terms had an exact semantic match to RxNorm. Compared to the gold standard, MTERMS achieved a precision of 99.8% and a recall of 73.9% when mapping all MDD terms, and a precision of 100% and a recall of 72.6% when mapping the top prescribed medications. CONCLUSION: The challenges and gaps in mapping MDD to RxNorm are mainly due to unique user or application requirements for representing drug concepts and the different modeling approaches inherent in the two terminologies. An automated approach based on NLP followed by human expert review is an efficient and feasible way for conducting dynamic mapping.

Tailored prevention of inpatient falls: development and usability testing of the fall TIPS toolkit.

Patient falls and fall-related injuries are serious problems in hospitals. The Fall TIPS application aims to prevent patient falls by translating routine nursing fall risk assessment into a decision support intervention that communicates fall risk status and creates a tailored evidence-based plan of care that is accessible to the care team, patients, and family members. In our design and implementation of the Fall TIPS toolkit, we used the Spiral Software Development Life Cycle model. Three output tools available to be generated from the toolkit are bed poster, plan of care, and patient education handout. A preliminary design of the application was based on initial requirements defined by project leaders and informed by focus groups with end users. Preliminary design partially simulated the paper version of the Morse Fall Scale currently used in hospitals involved in the research study. Strengths and weaknesses of the first prototype were identified by heuristic evaluation. Usability testing was performed at sites where research study is implemented. Suggestions mentioned by end users participating in usability studies were either directly incorporated into the toolkit and output tools, were slightly modified, or will be addressed during training. The next step is implementation of the fall prevention toolkit on the pilot testing units.

Tailored prevention of inpatient falls: development and usability testing of the fall TIPS toolkit.

Patient falls and fall-related injuries are serious problems in hospitals. The Fall TIPS application aims to prevent patient falls by translating routine nursing fall risk assessment into a decision support intervention that communicates fall risk status and creates a tailored evidence-based plan of care that is accessible to the care team, patients, and family members. In our design and implementation of the Fall TIPS toolkit, we used the Spiral Software Development Life Cycle model. Three output tools available to be generated from the toolkit are bed poster, plan of care, and patient education handout. A preliminary design of the application was based on initial requirements defined by project leaders and informed by focus groups with end users. Preliminary design partially simulated the paper version of the Morse Fall Scale currently used in hospitals involved in the research study. Strengths and weaknesses of the first prototype were identified by heuristic evaluation. Usability testing was performed at sites where research study is implemented. Suggestions mentioned by end users participating in usability studies were either directly incorporated into the toolkit and output tools, were slightly modified, or will be addressed during training. The next step is implementation of the fall prevention toolkit on the pilot testing units.

Usability testing of a Falls Prevention Tool Kit for an inpatient acute care setting.

Efforts to prevent falls in the hospital setting involves identifying patients at risk of falling and implementing fall prevention strategies. This poster describes the method and results of Performance Usability Testing on a web-based Fall Prevention Tool Kit (FPTK) developed as part of a research study, (Falls TIPS-Tailoring Interventions for Patient Safety) funded by The Robert Wood Johnson Foundation.

Smart Forms: building condition-specific documentation and decision support tools for ambulatory EHR.

Smart Forms are condition-specific documentation tools that integrate pertinent data review, guideline-based decision support, ambulatory order entry, patient education and coded data capture capabilities. Smart Forms are being developed as Web applications in a service oriented architecture and employ a rules engine for dynamic content generation.