Intravenous Medication Administration Safety with Smart Infusion Pumps in the Neonatal Intensive Care Unit: An Observational Study.

INTRODUCTION: Infants in the neonatal intensive care unit (NICU) are among the most vulnerable patient populations and medication errors are a significant source of risk and harm to neonates. Smart infusion pumps have been implemented to support the safe medication administration process; however, the effect of using smart infusion pumps on medication safety in the NICU is still unclear. METHODS: We conducted an observational study with a prospective point-prevalence approach to investigate intravenous (IV) medication administration errors in the NICU at one academic medical center in the USA. Observations were conducted in 48 days in a 3-month data collection period in 2019. RESULTS: We observed a total of 441 patients with 905 IV medication administrations during the data collection period. The total number of errors was 130 (14.4 per 100 administrations). Of these, the most frequent errors were selecting the wrong drug library entry (5.3 per 100 administrations), unauthorized medication (0.7 per 100 administrations), and wrong dose (0.6 per 100 administrations). Sixty-eight errors (7.5 per 100 administrations) were unlikely to cause harm despite reaching the patient (category C errors), while the rest did not reach the patient. CONCLUSION: We identified the medication errors, which was unique to NICU populations, but no harm to the patients were identified. Most errors occurred due to a lack of compliance of using smart pump technology; therefore, potential exists to maximize safety related to medication administration practices in the NICU through hospital policy change and increasing adherence to appropriate use of smart pump technology.

Developing, pilot testing, and refining requirements for 3 EHR-integrated interventions to improve diagnostic safety in acute care: a user-centered approach.

Objective: To describe a user-centered approach to develop, pilot test, and refine requirements for 3 electronic health record (EHR)-integrated interventions that target key diagnostic process failures in hospitalized patients. Materials and Methods: Three interventions were prioritized for development: a Diagnostic Safety Column (DSC) within an EHR-integrated dashboard to identify at-risk patients; a Diagnostic Time-Out (DTO) for clinicians to reassess the working diagnosis; and a Patient Diagnosis Questionnaire (PDQ) to gather patient concerns about the diagnostic process. Initial requirements were refined from analysis of test cases with elevated risk predicted by DSC logic compared to risk perceived by a clinician working group; DTO testing sessions with clinicians; PDQ responses from patients; and focus groups with clinicians and patient advisors using storyboarding to model the integrated interventions. Mixed methods analysis of participant responses was used to identify final requirements and potential implementation barriers. Results: Final requirements from analysis of 10 test cases predicted by the DSC, 18 clinician DTO participants, and 39 PDQ responses included the following: DSC configurable parameters (variables, weights) to adjust baseline risk estimates in real-time based on new clinical data collected during hospitalization; more concise DTO wording and flexibility for clinicians to conduct the DTO with or without the patient present; and integration of PDQ responses into the DSC to ensure closed-looped communication with clinicians. Analysis of focus groups confirmed that tight integration of the interventions with the EHR would be necessary to prompt clinicians to reconsider the working diagnosis in cases with elevated diagnostic error (DE) risk or uncertainty. Potential implementation barriers included alert fatigue and distrust of the risk algorithm (DSC); time constraints, redundancies, and concerns about disclosing uncertainty to patients (DTO); and patient disagreement with the care team's diagnosis (PDQ). Discussion: A user-centered approach led to evolution of requirements for 3 interventions targeting key diagnostic process failures in hospitalized patients at risk for DE. Conclusions: We identify challenges and offer lessons from our user-centered design process.

Providers' and Patients' Perspectives on Diagnostic Errors in the Acute Care Setting.

BACKGROUND: Diagnostic errors (DEs) have been studied extensively in ambulatory care, but less work has been done in the acute care setting. In this study, the authors examined health care providers' and patients' perspectives about the classification of DEs, the main causes and scope of DEs in acute care, the main gaps in current systems, and the need for innovative solutions. METHODS: A qualitative mixed methods study was conducted, including semistructured interviews with health care providers and focus groups with patient advisors. Using grounded theory approach, thematic categories were derived from the interviews and focus groups. RESULTS: The research team conducted interviews with 17 providers and two focus groups with seven patient advisors. Both providers and patient advisors struggled to define and describe DEs in acute care settings. Although participants agreed that DEs pose a significant risk to patient safety, their perception of the frequency of DEs was mixed. Most participants identified communication failures, lack of comfort with diagnostic uncertainty, incorrect clinical evaluation, and cognitive load as key causes of DEs. Most respondents believed that non-information technology (IT) tools and processes (for example, communication improvement strategies) could significantly reduce DEs. CONCLUSION: The study findings represent an important supplement to our understanding of DEs in acute care settings and the advancement of a culture of patient safety in the context of patient-centered care and patient engagement. Health care organizations should consider the key factors identified in this study when trying to create a culture that engages clinicians and patients in reducing DEs.

Comparison of a Voluntary Safety Reporting System to a Global Trigger Tool for Identifying Adverse Events in an Oncology Population.

OBJECTIVE: There is a lack of research on adverse event (AE) detection in oncology patients, despite the propensity for iatrogenic harm. Two common methods include voluntary safety reporting (VSR) and chart review tools, such as the Institute for Healthcare Improvement's Global Trigger Tool (GTT). Our objective was to compare frequency and type of AEs detected by a modified GTT compared with VSR for identifying AEs in oncology patients in a larger clinical trial. METHODS: Patients across 6 oncology units (from July 1, 2013, through May 29, 2015) were randomly selected. Retrospective chart reviews were conducted by a team of nurses and physicians to identify AEs using the GTT. The VSR system was queried by the department of quality and safety of the hospital. Adverse event frequencies, type, and harm code for both methods were compared. RESULTS: The modified GTT detected 0.90 AEs per patient (79 AEs in 88 patients; 95% [0.71-1.12] AEs per patient) that were predominantly medication AEs (53/79); more than half of the AEs caused harm to the patients (41/79, 52%), but only one quarter were preventable (21/79; 27%). The VSR detected 0.24 AEs per patient (21 AEs in 88 patients; 95% [0.15-0.37] AEs per patient), a large plurality of which were medication/intravenous related (8/21); more than half did not cause harm (70%). Only 2% of the AEs (2/100) were detected by both methods. CONCLUSIONS: Neither the modified GTT nor the VSR system alone is sufficient for detecting AEs in oncology patient populations. Further studies exploring methods such as automated AE detection from electronic health records and leveraging patient-reported AEs are needed.

The Impact of Smart Pump Interoperability on Errors in Intravenous Infusion Administrations: A Multihospital Before and After Study.

OBJECTIVE: The objective of this study was to assess the frequency, type, and severity of errors associated with intravenous medication administration before and after smart pump interoperability. METHODS: We conducted an observational study at a community healthcare system before and after implementing smart pump interoperability. Point prevalence methodology was used to collect data on medication administration and errors in adult inpatient settings. RESULTS: Observations were completed for 350 infusions preintervention (178 patients) and 367 postintervention (200 patients). Total errors significantly decreased from 401 (114.6 per 100 infusions) to 354 (96.5 per 100 infusions, P = 0.02). Administration errors decreased from 144 (41.1 per 100 infusions) to 119 (32.4 per 100 infusions, P = 0.12). Expired medication errors significantly reduced from 11 (3.1 per 100 infusions) to 2 (0.5 per 100 infusions, P = 0.02). Errors involving high-risk medications significantly reduced from 45 (12.8 per 100 infusions) to 25 (6.8 per 100 infusions, P = 0.01). Errors involving continuous medications significantly reduced from 44 (12.6 per 100 infusions) to 22 (6.0 per 100 infusions, P = 0.005). When comparing programming type, manual programming resulted in 115 (77.2%) of administration and user documentation errors compared with 34 errors (22.8%) that occurred when autoprogramming was used. Of these, errors involving high-risk medications reduced from 21 (84.0%) to 4 (16.0%) after using autoprogramming. CONCLUSIONS: Smart pump interoperability resulted in a 16% reduction in medication administration errors. Despite using dose error reduction software and autoprogramming, some types of errors persisted. Further studies are needed to understand how technology use can be optimized.

Patient Perceptions of Hospital Experiences: Implications for Innovations in Patient Safety.

OBJECTIVE: The purpose of this study was to qualitatively examine safety experiences of hospitalized patients and families. METHODS: We conducted 5 focus groups at 2 sites with patients and family members of patients who had been hospitalized at least once within the preceding 2 years. Using a semistructured focus group script, participants were asked to describe hospital experiences, including any safety risks or problems, and to discuss trust in the hospital care team or members of the care team. All focus groups were audiorecorded and transcribed, and transcriptions were qualitatively analyzed using thematic analysis by experienced qualitative analysts and experts in patient safety. RESULTS: We collected rich descriptions of safety problems in the hospital. We identified 4 main themes from our focus group data. (1) Experiences with safety problems were not unusual among participants, (2) patients and families develop a structured "care story" about their hospital experiences, (3) there is a spectrum of trust between patients and the hospital care team members that can be diminished or enhanced by experiences, and (4) patients believed having someone who could advocate for them during their hospitalization was important. CONCLUSIONS: Our results suggest that acknowledgment of safety problems, clear communication, building trust, and a role for advocacy are impactful pathways health care providers and health care systems can improve patient experiences. Information technology such as patient- and clinician-facing displays can support each of these actions.

Unique Patient-Reported Hospital Safety Concerns With Online Tool: MySafeCare.

BACKGROUND: Hospitalized patients and their care partners have valuable and unique perspectives of the medical care they receive. Direct and real-time reporting of patients' safety concerns, though limited in the acute care setting, could provide opportunities to improve patient care. METHODS: We implemented the MySafeCare (MSC) application on six acute care units for 18 months as part of a patient-centered health information technology intervention to promote engagement and safety in the acute care setting. The web-based application allowed hospitalized patients to submit safety concerns anonymously and in real time. We describe characteristics of patient submissions including their categorizations. We evaluated rates of submissions to MSC and compared them with rates of submissions to the Patient Family Relations department at the hospital. In addition, we performed thematic analysis of narrative concerns submitted to the application. RESULTS: We received 46 submissions to MSC and 33% of concerns received were anonymous. The overall rate of submissions was 0.6 submissions per 1000 patient-days and was considerably lower than the rate of submissions to the Patient Family Relations during the same period (4.1 per 1000 patient-days). Identified themes of narrative concerns included unmet care needs and preferences, inadequate communication, and concerns about safety of care. CONCLUSIONS: Although the submission rate to the application was low, MSC captured important content directly from hospitalized patients or their care partners. A web-based patient safety reporting tool for patients should be studied further to understand patient and care partner use and willingness to engage, as well as potential effects on patient safety outcomes.

Engaging Patients in the Use of Real-Time Electronic Clinical Data to Improve the Safety and Reliability of Their Own Care.

OBJECTIVES: There is considerable evidence that providing patients with access to their health information is beneficial, but there is limited evidence regarding the effect of providing real-time patient safety-related information on health outcomes. The aim of this study was to evaluate the association between use of an electronic patient safety dashboard (Safety Advisor) and health outcomes. METHODS: The Safety Advisor was implemented in 6 adult medicine units at one hospital in the United States. Study participants were asked to use the Safety Advisor, which provides real-time patient safety-related information through a Web-based portal. The primary outcome was the association between the application usage and health outcomes (readmission rate and mortality rate) per 3 different usage groups, and the secondary outcome was the association of Patient Activation Measure (PAM) scores with use. RESULTS: One hundred eighty-one participants were included for the data analysis. Approximately 90% of users accessed the application during the first 4 days of enrollment: 51.6% of users only accessed it on 1 day, whereas 5.8% used it more than 3 days. Patients who used the application more had lower 30-day readmission rates (P = 0.01) compared with the lower-usage group. The PAM scores for users of Safety Advisor (71.8) were higher than the nonpatient portal users (60.8, P < 0.0001). CONCLUSIONS: We found an association between the use of Safety Advisor and health outcomes. Differences in PAM scores between groups were statistically significant. A larger-scale randomized control trial is warranted to evaluate the impact on patient outcomes among a high-risk patient population.

The Communicating Narrative Concerns Entered by Registered Nurses (CONCERN) Clinical Decision Support Early Warning System: Protocol for a Cluster Randomized Pragmatic Clinical Trial.

BACKGROUND: Every year, hundreds of thousands of inpatients die from cardiac arrest and sepsis, which could be avoided if those patients' risk for deterioration were detected and timely interventions were initiated. Thus, a system is needed to convert real-time, raw patient data into consumable information that clinicians can utilize to identify patients at risk of deterioration and thus prevent mortality and improve patient health outcomes. The overarching goal of the COmmunicating Narrative Concerns Entered by Registered Nurses (CONCERN) study is to implement and evaluate an early warning score system that provides clinical decision support (CDS) in electronic health record systems. With a combination of machine learning and natural language processing, the CONCERN CDS utilizes nursing documentation patterns as indicators of nurses' increased surveillance to predict when patients are at the risk of clinical deterioration. OBJECTIVE: The objective of this cluster randomized pragmatic clinical trial is to evaluate the effectiveness and usability of the CONCERN CDS system at 2 different study sites. The specific aim is to decrease hospitalized patients' negative health outcomes (in-hospital mortality, length of stay, cardiac arrest, unanticipated intensive care unit transfers, and 30-day hospital readmission rates). METHODS: A multiple time-series intervention consisting of 3 phases will be performed through a 1-year period during the cluster randomized pragmatic clinical trial. Phase 1 evaluates the adoption of our algorithm through pilot and trial testing, phase 2 activates optimized versions of the CONCERN CDS based on experience from phase 1, and phase 3 will be a silent release mode where no CDS is viewable to the end user. The intervention deals with a series of processes from system release to evaluation. The system release includes CONCERN CDS implementation and user training. Then, a mixed methods approach will be used with end users to assess the system and clinician perspectives. RESULTS: Data collection and analysis are expected to conclude by August 2022. Based on our previous work on CONCERN, we expect the system to have a positive impact on the mortality rate and length of stay. CONCLUSIONS: The CONCERN CDS will increase team-based situational awareness and shared understanding of patients predicted to be at risk for clinical deterioration in need of intervention to prevent mortality and associated harm. TRIAL REGISTRATION: ClinicalTrials.gov NCT03911687; https://clinicaltrials.gov/ct2/show/NCT03911687. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/30238.

Analyzing diagnostic errors in the acute setting: a process-driven approach.

OBJECTIVES: We describe an approach for analyzing failures in diagnostic processes in a small, enriched cohort of general medicine patients who expired during hospitalization and experienced medical error. Our objective was to delineate a systematic strategy for identifying frequent and significant failures in the diagnostic process to inform strategies for preventing adverse events due to diagnostic error. METHODS: Two clinicians independently reviewed detailed records of purposively sampled cases identified from established institutional case review forums and assessed the likelihood of diagnostic error using the Safer Dx instrument. Each reviewer used the modified Diagnostic Error Evaluation and Research (DEER) taxonomy, revised for acute care (41 possible failure points across six process dimensions), to characterize the frequency of failure points (FPs) and significant FPs in the diagnostic process. RESULTS: Of 166 cases with medical error, 16 were sampled: 13 (81.3%) had one or more diagnostic error(s), and a total of 113 FPs and 30 significant FPs were identified. A majority of significant FPs (63.3%) occurred in "Diagnostic Information and Patient Follow-up" and "Patient and Provider Encounter and Initial Assessment" process dimensions. Fourteen (87.5%) cases had a significant FP in at least one of these dimensions. CONCLUSIONS: Failures in the diagnostic process occurred across multiple dimensions in our purposively sampled cohort. A systematic analytic approach incorporating the modified DEER taxonomy, revised for acute care, offered critical insights into key failures in the diagnostic process that could serve as potential targets for preventative interventions.

Identifying nursing documentation patterns associated with patient deterioration and recovery from deterioration in critical and acute care settings.

OBJECTIVES: Nursing documentation behavior within electronic health records may reflect a nurse's concern about a patient and can be used to predict patient deterioration. Our study objectives were to quantify variations in nursing documentation patterns, confirm those patterns and variations with clinicians, and identify which patterns indicate patient deterioration and recovery from clinical deterioration events in the critical and acute care settings. METHODS: We collected patient data from electronic health records and conducted a regression analysis to identify different nursing documentation patterns associated with patient outcomes resulting from clinical deterioration events in the intensive care unit (ICU) and acute care unit (ACU). The primary outcome measures were whether patients were discharged alive from the hospital or expired during their hospital encounter. Secondary outcome measures were clinical deterioration events. RESULTS: In the ICU, the increased documentation of heart rate, body temperature, and withheld medication administrations were significantly associated with inpatient mortality. In the ACU, the documentation of blood pressure, respiratory rate with comments, singular vital signs, and withheld medications were significantly related to inpatient mortality. In contrast, the documentation of heart rate and "as needed" medication administrations were significantly associated with patient survival to discharge in the ACU. CONCLUSION: We successfully identified and confirmed the clinical relevancy of the nursing documentation patterns indicative of patient deterioration and recovery from clinical deterioration events in both the ICU and ACU.

Nursing Documentation Variation Across Different Medical Facilities Within an Integrated Healthcare System.

The purpose of this study was to demonstrate nursing documentation variation based on electronic health record design and its relationship with different levels of care by reviewing how various flowsheet measures, within the same electronic health record across an integrated healthcare system, are documented in different types of medical facilities. Flowsheet data with information on patients who were admitted to academic medical centers, community hospitals, and rehabilitation centers were used to calculate the frequency of flowsheet entries documented. We then compared the distinct flowsheet measures documented in five flowsheet templates across the different facilities. We observed that each type of healthcare facility appeared to have distinct clinical care foci and flowsheet measures documented differed within the same template based on facility type. Designing flowsheets tailored to study settings can meet the needs of end users and increase documentation efficiency by reducing time spent on unrelated flowsheet measures. Furthermore, this process can save nurses time for direct patient care.

Healthcare Process Modeling to Phenotype Clinician Behaviors for Exploiting the Signal Gain of Clinical Expertise (HPM-ExpertSignals): Development and evaluation of a conceptual framework.

OBJECTIVE: There are signals of clinicians' expert and knowledge-driven behaviors within clinical information systems (CIS) that can be exploited to support clinical prediction. Describe development of the Healthcare Process Modeling Framework to Phenotype Clinician Behaviors for Exploiting the Signal Gain of Clinical Expertise (HPM-ExpertSignals). MATERIALS AND METHODS: We employed an iterative framework development approach that combined data-driven modeling and simulation testing to define and refine a process for phenotyping clinician behaviors. Our framework was developed and evaluated based on the Communicating Narrative Concerns Entered by Registered Nurses (CONCERN) predictive model to detect and leverage signals of clinician expertise for prediction of patient trajectories. RESULTS: Seven themes-identified during development and simulation testing of the CONCERN model-informed framework development. The HPM-ExpertSignals conceptual framework includes a 3-step modeling technique: (1) identify patterns of clinical behaviors from user interaction with CIS; (2) interpret patterns as proxies of an individual's decisions, knowledge, and expertise; and (3) use patterns in predictive models for associations with outcomes. The CONCERN model differentiated at risk patients earlier than other early warning scores, lending confidence to the HPM-ExpertSignals framework. DISCUSSION: The HPM-ExpertSignals framework moves beyond transactional data analytics to model clinical knowledge, decision making, and CIS interactions, which can support predictive modeling with a focus on the rapid and frequent patient surveillance cycle. CONCLUSIONS: We propose this framework as an approach to embed clinicians' knowledge-driven behaviors in predictions and inferences to facilitate capture of healthcare processes that are activated independently, and sometimes well before, physiological changes are apparent.

Evaluating the Impact of Radio Frequency Identification Retained Surgical Instruments Tracking on Patient Safety: Literature Review.

BACKGROUND: Retained surgical instruments (RSI) are one of the most serious preventable complications in operating room settings, potentially leading to profound adverse effects for patients, as well as costly legal and financial consequences for hospitals. Safety measures to eliminate RSIs have been widely adopted in the United States and abroad, but despite widespread efforts, medical errors with RSI have not been eliminated. OBJECTIVE: Through a systematic review of recent studies, we aimed to identify the impact of radio frequency identification (RFID) technology on reducing RSI errors and improving patient safety. METHODS: A literature search on the effects of RFID technology on RSI error reduction was conducted in PubMed and CINAHL (2000-2016). Relevant articles were selected and reviewed by 4 researchers. RESULTS: After the literature search, 385 articles were identified and the full texts of the 88 articles were assessed for eligibility. Of these, 5 articles were included to evaluate the benefits and drawbacks of using RFID for preventing RSI-related errors. The use of RFID resulted in rapid detection of RSI through body tissue with high accuracy rates, reducing risk of counting errors and improving workflow. CONCLUSIONS: Based on the existing literature, RFID technology seems to have the potential to substantially improve patient safety by reducing RSI errors, although the body of evidence is currently limited. Better designed research studies are needed to get a clear understanding of this domain and to find new opportunities to use this technology and improve patient safety.

Information Needs and the Use of Documentation to Support Collaborative Decision-Making: Implications for the Reduction of Central Line-Associated Blood Stream Infections.

It is clear that interdisciplinary communication and collaboration have the potential to mitigate healthcare-associated harm, yet there is limited research on how communication through documentation in the patient record can support collaborative decision making. Understanding what information is needed to support collaborative decision making is necessary to design electronic health information systems that facilitate effective communication and, ultimately, safe care. To explore this issue, we focused on information needs related to central venous catheter management and the prevention of central line-associated blood stream infections. Semistructured interviews were conducted with nurses working in an intensive care unit. Interview transcripts were analyzed using inductive thematic analysis. Three themes were identified: (1) challenges managing documentation in multiple places in the absence of formal documentation processes for central venous catheter management; (2) lack of standardized decision-making processes for managing central venous catheters; and (3) oral communication holds it together. Our findings provide a foundation for the development of EHR functional requirements that enhance communication regarding the management of central venous catheters and facilitate the prompt removal of unnecessary lines.

Development and Validation of WeCares, a Survey Instrument to Assess Hospitalized Patients' and Family Members' "Willingness to Engage in Your Care and Safety".

BACKGROUND: Patient engagement is recognized as a method to improve care quality and safety. A research team developed WeCares (Willingness to Engage in Your Care and Safety), a survey instrument assessing patients' and families' engagement in the safety of their care during their hospital stay. The objective of this study is to establish the preliminary construct validity and internal consistency of WeCares. METHODS: WeCares was distributed to patients and families. With the survey responses, exploratory factor analysis (EFA) was performed to identify the factorial structure of WeCares. The internal consistency was assessed using Cronbach's alpha. Descriptive and comparative analysis was also performed to summarize patients' and families' responses. RESULTS: A total of 247 patients and families responded to the WeCare survey, of which 224 were used for EFA. EFA resulted in a 13-item, four-factor structure, including (1) comfortable sharing concerns, (2) responsibility for preventing errors, (3) perception of care team members' attitude, and (4) patients'/families' role in preventing errors. The Cronbach alphas were 0.716-0.866, indicating acceptable internal consistency. Overall, patients and families were comfortable sharing concerns with clinicians but preferred to remain anonymous. They believed that the care team members hold most responsibility for error prevention, however, and agreed on their ability to help prevent errors. CONCLUSION: WeCares was developed to assess patients' and families' willingness to engage. WeCares can also be used to facilitate conversation about safety concerns and shared responsibility. The study team believes this would lead to patient activation in guarding their own care and ultimately improve patient outcomes and safety.

Development and validation of early warning score system: A systematic literature review.

OBJECTIVES: This review aims to: 1) evaluate the quality of model reporting, 2) provide an overview of methodology for developing and validating Early Warning Score Systems (EWSs) for adult patients in acute care settings, and 3) highlight the strengths and limitations of the methodologies, as well as identify future directions for EWS derivation and validation studies. METHODOLOGY: A systematic search was conducted in PubMed, Cochrane Library, and CINAHL. Only peer reviewed articles and clinical guidelines regarding developing and validating EWSs for adult patients in acute care settings were included. 615 articles were extracted and reviewed by five of the authors. Selected studies were evaluated based on the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) checklist. The studies were analyzed according to their study design, predictor selection, outcome measurement, methodology of modeling, and validation strategy. RESULTS: A total of 29 articles were included in the final analysis. Twenty-six articles reported on the development and validation of a new EWS, while three reported on validation and model modification. Only eight studies met more than 75% of the items in the TRIPOD checklist. Three major techniques were utilized among the studies to inform their predictive algorithms: 1) clinical-consensus models (n = 6), 2) regression models (n = 15), and 3) tree models (n = 5). The number of predictors included in the EWSs varied from 3 to 72 with a median of seven. Twenty-eight models included vital signs, while 11 included lab data. Pulse oximetry, mental status, and other variables extracted from electronic health records (EHRs) were among other frequently used predictors. In-hospital mortality, unplanned transfer to the intensive care unit (ICU), and cardiac arrest were commonly used clinical outcomes. Twenty-eight studies conducted a form of model validation either within the study or against other widely-used EWSs. Only three studies validated their model using an external database separate from the derived database. CONCLUSION: This literature review demonstrates that the characteristics of the cohort, predictors, and outcome selection, as well as the metrics for model validation, vary greatly across EWS studies. There is no consensus on the optimal strategy for developing such algorithms since data-driven models with acceptable predictive accuracy are often site-specific. A standardized checklist for clinical prediction model reporting exists, but few studies have included reporting aligned with it in their publications. Data-driven models are subjected to biases in the use of EHR data, thus it is particularly important to provide detailed study protocols and acknowledge, leverage, or reduce potential biases of the data used for EWS development to improve transparency and generalizability.

Use, Perceived Usability, and Barriers to Implementation of a Patient Safety Dashboard Integrated within a Vendor EHR.

BACKGROUND: Preventable adverse events continue to be a threat to hospitalized patients. Clinical decision support in the form of dashboards may improve compliance with evidence-based safety practices. However, limited research describes providers' experiences with dashboards integrated into vendor electronic health record (EHR) systems. OBJECTIVE: This study was aimed to describe providers' use and perceived usability of the Patient Safety Dashboard and discuss barriers and facilitators to implementation. METHODS: The Patient Safety Dashboard was implemented in a cluster-randomized stepped wedge trial on 12 units in neurology, oncology, and general medicine services over an 18-month period. Use of the Dashboard was tracked during the implementation period and analyzed in-depth for two 1-week periods to gather a detailed representation of use. Providers' perceptions of tool usability were measured using the Health Information Technology Usability Evaluation Scale (rated 1-5). Research assistants conducted field observations throughout the duration of the study to describe use and provide insight into tool adoption. RESULTS: The Dashboard was used 70% of days the tool was available, with use varying by role, service, and time of day. On general medicine units, nurses logged in throughout the day, with many logins occurring during morning rounds, when not rounding with the care team. Prescribers logged in typically before and after morning rounds. On neurology units, physician assistants accounted for most logins, accessing the Dashboard during daily brief interdisciplinary rounding sessions. Use on oncology units was rare. Satisfaction with the tool was highest for perceived ease of use, with attendings giving the highest rating (4.23). The overall lowest rating was for quality of work life, with nurses rating the tool lowest (2.88). CONCLUSION: This mixed methods analysis provides insight into the use and usability of a dashboard tool integrated within a vendor EHR and can guide future improvements and more successful implementation of these types of tools.

Mining clinical phrases from nursing notes to discover risk factors of patient deterioration.

OBJECTIVE: Early identification and treatment of patient deterioration is crucial to improving clinical outcomes. To act, hospital rapid response (RR) teams often rely on nurses' clinical judgement typically documented narratively in the electronic health record (EHR). We developed a data-driven, unsupervised method to discover potential risk factors of RR events from nursing notes. METHODS: We applied multiple natural language processing methods, including language modelling, word embeddings, and two phrase mining methods (TextRank and NC-Value), to identify quality phrases that represent clinical entities from unannotated nursing notes. TextRank was used to determine the important word-sequences in each note. NC-Value was then used to globally rank the locally-important sequences across the whole corpus. We evaluated our method both on its accuracy compared to human judgement and on the ability of the mined phrases to predict a clinical outcome, RR event hazard. RESULTS: When applied to 61,740 hospital encounters with 1,067 RR events and 778,955 notes, our method achieved an average precision of 0.590 to 0.764 (when excluding numeric tokens). Time-dependent covariates Cox model using the phrases achieved a concordance index of 0.739. Clustering the phrases revealed clinical concepts significantly associated with RR event hazard. DISCUSSION: Our findings demonstrate that our minimal-annotation, unsurprised method can rapidly mine quality phrases from a large amount of nursing notes, and these identified phrases are useful for downstream tasks, such as clinical outcome predication and risk factor identification.

Design of a safety dashboard for patients.

BACKGROUND: Nearly one third of hospitalized patients suffer harm from medical errors in U.S. hospitals each year. OBJECTIVE: Our goal was to design a patient-facing application that is intended to engage patients and their caregivers in reviewing and responding to clinical issues that may have safety implications. PATIENT INVOLVEMENT: We conducted user-centered design sessions with recently hospitalized individuals and /or informal caregivers. METHODS: We conducted five user-centered design sessions with total of 37 individuals. Sessions began with individuals sharing personal stories of recent hospitalizations and any experienced safety events. We then solicited feedback on the current iteration of the patient-facing safety application. The design of the app was updated between sessions. RESULTS: The design of our app centers around three key findings. First, involving patients in safety promotion is novel to most patients and their caregivers: therefore the framing of the tool's purpose and appropriate use is critical to engage potential users and manage expectations, this messaging was carefully crafted with patient input. Second, since most patients do not associate specific safety issues with appropriate remedial or preventative actions, the centerpiece of the application is a table that connects safety issues with related "Questions you should ask" and "Things you can do". Third, patients need understandable explanations of medical terms and concepts as well as explanation of changes in risk; the tool includes plain language "translations" of all medical terms used, links to curated patient education materials, and simplified graphics to visualize changes in risk. DISCUSSION: Our findings may generalize to other efforts to engage patients in their care. PRACTICE VALUE: Designing for patient engagement requires patients' perspective both on their current role and their ideal role; framing for expectations, action-oriented design, and clarity of presented information may address that gap in patient engagement.