Longitudinal clinical decision support for assessing decisions over time: State-of-the-art and future directions.

Objective: Patients and clinicians rarely experience healthcare decisions as snapshots in time, but clinical decision support (CDS) systems often represent decisions as snapshots. This scoping review systematically maps challenges and facilitators to longitudinal CDS that are applied at two or more timepoints for the same decision made by the same patient or clinician. Methods: We searched Embase, PubMed, and Medline databases for articles describing development, validation, or implementation of patient- or clinician-facing longitudinal CDS. Validated quality assessment tools were used for article selection. Challenges and facilitators to longitudinal CDS are reported according to PRISMA-ScR guidelines. Results: Eight articles met inclusion criteria; each article described a unique CDS. None used entirely automated data entry, none used living guidelines for updating the evidence base or knowledge engine as new evidence emerged during the longitudinal study, and one included formal readiness for change assessments. Seven of eight CDS were implemented and evaluated prospectively. Challenges were primarily related to suboptimal study design (with unique challenges for each study) or user interface. Facilitators included use of randomized trial designs for prospective enrollment, increased CDS uptake during longitudinal exposure, and machine-learning applications that are tailored to the CDS use case. Conclusions: Despite the intuitive advantages of representing healthcare decisions longitudinally, peer-reviewed literature on longitudinal CDS is sparse. Existing reports suggest opportunities to incorporate longitudinal CDS frameworks, automated data entry, living guidelines, and user readiness assessments. Generating best practice guidelines for longitudinal CDS would require a greater depth and breadth of published work and expert opinion.

Buy or Build: Challenges Developing Consumer Digital Health Interventions.

BACKGROUND: Digital health interventions offer opportunities to improve collaborative care between clinicians and patients. Designing and implementing digital health interventions requires decisions about buying or building each technology-related component, all of which can lead to unanticipated issues. OBJECTIVES: This study aimed to describe issues encountered from our "buy or build" decisions developing two digital health interventions over different timeframes, designed to use patient-generated health data to: (1) improve hypertension control and (2) measure and improve adherence to HIV-related medications. METHODS: CONDUIT-HID (CONtrolling Disease Using Information Technology-Hypertension In Diabetes) was developed during 2010 to 2015 to allow patients receiving care from a multispecialty group practice to easily upload home blood pressure readings into their electronic health record and trigger clinician action if mean blood pressure values indicated inadequate control. USE-MI (Unobtrusive SEnsing of Medication Intake) was developed from 2016 to 2022 to allow entry of patients' HIV-related medication regimens, send reminders if patients had not taken their medications by the scheduled time(s), attempt to detect medication ingestion through machine learning analysis of smartwatch motion data, and present graphical adherence summaries to patients and clinicians. RESULTS: Both projects required multiple "buy or build" decisions across all system components, including data collection, transfer, analysis, and display. We used commercial, off-the-shelf technology where possible, but virtually all of these components still required substantial custom development. We found that, even though our projects spanned years, issues related to our "buy or build" decisions stemmed from several common themes, including mismatches between existing and new technologies, our use case being new or unanticipated, technology stability, technology longevity, and resource limitations. CONCLUSION: Those designing and implementing digital health interventions need to make numerous "buy or build" decisions as they create the technologies that underpin their intervention. These "buy or build" decisions, and the ensuing issues that will arise because of them, require careful planning, particularly if they represent an "edge case" use of existing commercial systems.

Electronic Progress Note Reading Patterns: An Eye Tracking Analysis.

This study used eye-tracking to understand how the order of note sections influences the way physicians read electronic progress notes. Participants (n = 7) wore an eye-tracking device while reviewing progress notes for four patient cases and then provided a verbal summary. We reviewed and analyzed verbal summaries and eye tracking recordings. Wide variation in reading behaviors existed. There was no relationship between time spent reading a section and section origin of verbal summaries.

Challenges and Opportunities to Improve the Clinician Experience Reviewing Electronic Progress Notes.

BACKGROUND: High-quality clinical notes are essential to effective clinical communication. However, electronic clinical notes are often long, difficult to review, and contain information that is potentially extraneous or out of date. Additionally, many clinicians write electronic clinical notes using customized templates, resulting in notes with significant variability in structure. There is a need to understand better how clinicians review electronic notes and how note structure variability may impact clinicians' note-reviewing experiences. OBJECTIVE: This article aims to understand how physicians review electronic clinical notes and what impact section order has on note-reviewing patterns. MATERIALS AND METHODS: We conducted an experiment utilizing an electronic health record (EHR) system prototype containing four anonymized patient cases, each composed of nine progress notes that were presented with note sections organized in different orders to different subjects (i.e., Subjective, Objective, Assessment, and Plan, Assessment, Plan, Subjective, and Objective, Subjective, Assessment, Objective, and Plan, and Mixed). Participants, who were mid-level residents and fellows, reviewed the cases and provided a brief summary after reviewing each case. Time-related data were collected and analyzed using descriptive statistics. Surveys were administered and interviews regarding experiences reviewing notes were collected and analyzed qualitatively. RESULTS: Qualitatively, participants reported challenges related to reviewing electronic clinical notes. Experimentally, time spent reviewing notes varied based on the note section organization. Consistency in note section organization improved performance (e.g., less scrolling and searching) compared with Mixed section organization when reviewing progress notes. DISCUSSION: Clinicians face significant challenges reviewing electronic clinical notes. Our findings support minimizing extraneous information in notes, removing information that can be found in other parts of the EHR, and standardizing the display and order of note sections to improve clinicians' note review experience. CONCLUSION: Our findings support the need to improve EHR note design and presentation to support optimal note review patterns for clinicians.

Designing a wrist-worn sensor to improve medication adherence: accommodating diverse user behaviors and technology preferences.

OBJECTIVES: High medication adherence is important for HIV suppression (antiretroviral therapy) and pre-exposure prophylaxis efficacy. We are developing sensor-based technologies to detect pill-taking gestures, trigger reminders, and generate adherence reports. MATERIALS AND METHODS: We collected interview, observation, and questionnaire data from individuals with and at-risk for HIV (N = 17). We assessed their medication-taking practices and physical actions, and feedback on our initial design. RESULTS: While participants displayed diverse medication taking practices and physical actions, most (67%) wanted to use the system to receive real-time and summative feedback, and most (69%) wanted to share data with their physicians. Participants preferred reminders via the wrist-worn device or mobile app, and summative feedback via mobile app or email. DISCUSSION: Adoption of these systems is promising if designs accommodate diverse behaviors and preferences. CONCLUSION: Our findings may help improve the accuracy and adoption of the system by accounting for user behaviors, physical actions, and preferences.

The Stay S.A.F.E. Strategy for Managing Interruptions Reduces Distraction Time in the Simulated Clinical Setting.

Interruptions occurring during the delivery of health care are frequent and create a serious threat to patient safety. It is important to test strategies directed at decreasing the negative effects of interruptions. The purpose of this pilot study was to test the Stay S.A.F.E. strategy for managing interruptions. A pretest, posttest quasi-experimental design was used to test the primary hypothesis that the Stay S.A.F.E. interruption management strategy would significantly (P < .05) reduce distraction time away from a primary task following an interruption. Twenty nurses with a median of 12 years of experience (range: 1-45 years) participated in the study. There was a significant decrease in the amount of time that participants were distracted away from the primary task between the pretest (134.47 seconds, SD = 6.87) and posttest (6.08 seconds, SD = 1.27) periods; P = .0004. The results of this study suggest that the Stay S.A.F.E. interruption management strategy was effective in reducing the length of time participants were distracted from the primary task in a simulated clinical setting. In addition, nurses confirmed the reports of others that interruptions are frequent, dangerous, and result in errors.

Usability Evaluation of Electronic Health Record System around Clinical Notes Usage-An Ethnographic Study.

Background A substantial gap exists between current Electronic Health Record (EHR) usability and potential optimal usability. One of the fundamental reasons for this discrepancy is poor incorporation of a User-Centered Design (UCD) approach during the Graphical User Interface (GUI) development process. Objective To evaluate usability strengths and weaknesses of two widely implemented EHR GUIs for critical clinical notes usage tasks. Methods Twelve Internal Medicine resident physicians interacting with one of the two EHR systems (System-1 at Location-A and System-2 at Location-B) were observed by two usability evaluators employing an ethnographic approach. User comments and observer findings were analyzed for two critical tasks: (1) clinical notes entry and (2) related information-seeking tasks. Data were analyzed from two standpoints: (1) usability references categorized by usability evaluators as positive, negative, or equivocal and (2) usability impact of each feature measured through a 7-point severity rating scale. Findings were also validated by user responses to a post observation questionnaire. Results For clinical notes entry, System-1 surpassed System-2 with more positive (26% vs. 12%) than negative (12% vs. 34%) usability references. Greatest impact features on EHR usability (severity score pertaining to each feature) for clinical notes entry were: autopopulation (6), screen options (5.5), communication (5), copy pasting (4.5), error prevention (4.5), edit ability (4), and dictation and transcription (3.5). Both systems performed equally well on information-seeking tasks and features with greatest impacts on EHR usability were navigation for notes (7) and others (e.g., looking for ancillary data; 5.5). Ethnographic observations were supported by follow-up questionnaire responses. Conclusion This study provides usability-specific insights to inform future, improved, EHR interface that is better aligned with UCD approach.

Eye Tracking: A Novel Approach for Evaluating and Improving the Safety of Healthcare Processes in the Simulated Setting.

INTRODUCTION: Eye tracking, used to evaluate a clinician's eye movements, is an example of an existing technology being used in novel ways by patient safety researchers in the simulated setting. The use of eye-tracking technology has the potential to augment current teaching, evaluation, and research methods in simulated settings by using this quantitative, objective data to better understand why an individual performed as he or she did on a simulated or naturalistic task. METHODS: Selected literature was reviewed with the purpose of explicating how eye tracking can be used by researchers and educators to evaluate error-prone processes. The literature reviewed was obtained by querying the databases PubMed, CINHAL, and Google Scholar using the key words eye tracking, patient safety, and medical errors from 2005 through 2015.An introduction to the use of eye tracking, including both theoretical underpinnings and technological considerations, is presented. In addition, examples of how eye tracking has been used in research studies conducted in both simulated and naturalistic settings are provided. CONCLUSIONS: The use of eye-tracking technology to capture the eye movements of novice and expert clinicians has provided new insight into behaviors associated with the identification of medical errors. The study of novices' and experts' eye movements provides data about clinician performance not possible with existing evaluation methods such as direct observation, verbal reports, and thinking out loud. The use of eye tracking to capture the behaviors of experts can lead to the development of training protocols to guide the education of students and novice practitioners. Eye-tracking technology clearly has the potential to transform the way clinical simulation is used to improve patient safety practices.

Assessing the effectiveness of five process elicitation methods: A case study of chemotherapy treatment plan review.

To reduce the probability of failures and to improve outcomes of safety-critical human-intensive processes, such as health care processes, it is important to be able to rigorously analyze such processes. The quality of that analysis often depends on having an accurate, detailed, and sufficiently complete understanding of the process being analyzed, where this understanding is typically represented as a formal process model that could then drive various rigorous analysis approaches. Developing this understanding and the corresponding formal process model may be difficult and, thus, a variety of process elicitation methods are often used. The work presented in this paper evaluates the effectiveness of five common elicitation methods in terms of their ability to elicit detailed process information necessary to support rigorous process analysis. These methods are employed to elicit typical steps and steps for responding to exceptional situations in a safety-critical health care process, the chemotherapy treatment plan review process. The results indicate strengths and weaknesses of each of the elicitation methods and suggest that it is preferable to apply multiple elicitation methods.

Usability Evaluation of an EHR's Clinical Notes Interface from the Perspective of Attending and Resident Physicians: An Exploratory Study.

Usability gaps between current and future improved Electronic Health Record (EHR) system designs exist due to insufficient incorporation of User-Centered Design (UCD) principles during System Development Life Cycle (SDLC). Usability of a commercial, inpatient EHR clinical notes documentation interface was analyzed from standpoints of two provider groups employing two standardized patient cases. Both objective and subjective data were collected from attending (n = 6) and resident physicians (n = 8) through usability testing employing a mixed method approach. The study results suggested that (i) EHR usability and desirability is influenced by user characteristics, (ii) workloads associated with H&P and progress notes writing are perceived differently between two groups, (iii) repeated task performance improves user efficiency and (iv) user performance is correlated to their subjective system assessments. Understanding usability of clinical documentation interface from perspectives of two different user groups, provides interface designers with an opportunity to develop an EHR system centered on UCD principles.

Eye tracking as a debriefing mechanism in the simulated setting improves patient safety practices.

INTRODUCTION: Human patient simulation has been widely adopted in healthcare education despite little research supporting its efficacy. The debriefing process is central to simulation education, yet alternative evaluation methods to support providing optimal feedback to students have not been well explored. Eye tracking technology is an innovative method for providing objective evaluative feedback to students after a simulation experience. The purpose of this study was to compare 3 forms of simulation-based student feedback (verbal debrief only, eye tracking only, and combined verbal debrief and eye tracking) to determine the most effective method for improving student knowledge and performance. METHODS: An experimental study using a pretest-posttest design was used to compare the effectiveness of 3 types of feedback. The subjects were senior baccalaureate nursing students in their final semester enrolled at a large university in the northeast United States. Students were randomly assigned to 1 of the 3 intervention groups. RESULTS: All groups performed better in the posttest evaluation than in the pretest. Certain safety practices improved significantly in the eye tracking-only group. These criteria were those that required an auditory and visual comparison of 2 artifacts such as "Compares patient stated name with name on ID band." CONCLUSIONS: Eye tracking offers a unique opportunity to provide students with objective data about their behaviors during simulation experiences, particularly related to safety practices that involve the comparison of patient stated data to an artifact such as an ID band. Despite the limitations of current eye tracking technology, there is significant potential for the use of this technology as a method for the study and evaluation of patient safety practices.

How do interruptions impact nurses' visual scanning patterns when using barcode medication administration systems?

While barcode medication administration (BCMA) systems have the potential to reduce medication errors, they may introduce errors, side effects, and hazards into the medication administration process. Studies of BCMA systems should therefore consider the interrelated nature of health information technology (IT) use and sociotechnical systems. We aimed to understand how the introduction of interruptions into the BCMA process impacts nurses' visual scanning patterns, a proxy for one component of cognitive processing. We used an eye tracker to record nurses' visual scanning patterns while administering a medication using BCMA. Nurses either performed the BCMA process in a controlled setting with no interruptions (n=25) or in a real clinical setting with interruptions (n=21). By comparing the visual scanning patterns between the two groups, we found that nurses in the interruptive environment identified less task-related information in a given period of time, and engaged in more information searching than information processing.

Overcoming challenges integrating patient-generated data into the clinical EHR: lessons from the CONtrolling Disease Using Inexpensive IT--Hypertension in Diabetes (CONDUIT-HID) Project.

INTRODUCTION: The CONDUIT-HID intervention integrates patients' electronic blood pressure measurements directly into the clinical EHR using Microsoft HealthVault as an intermediary data store. The goal of this paper is to describe generalizable categories of patient and technical challenges encountered in the development and implementation of this inexpensive, commercial off-the-shelf consumer health informatics intervention, examples of challenges within each category, and how the example challenges were resolved prior to conducting an RCT of the intervention. METHODS: The research team logged all challenges and mediation strategies during the technical development of the intervention, conducted home visits to observe patients using the intervention, and conducted telephone calls with patients to understand challenges they encountered. We then used these data to iteratively refine the intervention. RESULTS: The research team identified a variety of generalizable categories of challenges associated with patients uploading data from their homes, patients uploading data from clinics because they did not have or were not comfortable using home computers, and patients establishing the connection between HealthVault and the clinical EHR. Specific challenges within these categories arose because: (1) the research team had little control over the device and application design, (2) multiple vendors needed to coordinate their actions and design changes, (3) the intervention use cases were not anticipated by the device and application designers, (4) PHI accessed on clinic computers needed to be kept secure, (5) the research team wanted the data in the clinical EHR to be valid and reliable, (6) patients needed the ability to share only the data they wanted, and (7) the development of some EHR functionalities were new to the organization. While these challenges were varied and complex, the research team was able to successfully resolve each one prior to the start of the RCT. CONCLUSIONS: By identifying these generalizable categories of challenges, we aim to help others proactively search for and remedy potential challenges associated with their interventions, rather than reactively responding to problems as they arise. We posit that this approach will significantly increase the likelihood that these types of interventions will be successful.

Association of comorbidities with home care service utilization of patients with heart failure while receiving telehealth.

BACKGROUND: Comorbidities adversely impact heart failure (HF) outcomes. Telehealth can assist healthcare providers, especially nurses, in guiding their patients to follow the HF regimen. However, factors, including comorbidity patterns, that act in combination with telehealth to reduce home care nursing utilization are still unclear. PURPOSE: The purpose of this article was to examine the association of the comorbidity characteristics of HF patients with nursing utilization along with withdrawal from telehealth service during an episode of tele-home care. METHODOLOGY: A descriptive, correlational study design using retrospective chart review was used. The sample comprised Medicare patients admitted to a New England home care agency who had HF as a diagnosis and had used telehealth from 2008 to 2010. The electronic documentation at the home care agency served as the data source, which included Outcome and Assessment Information Set data of patients with HF. Logistic and multiple regression analyses were used to analyze data. RESULTS: The sample consisted of 403 participants, of whom 70% were older than 75 years, 55% were female, and 94% were white. Comorbidities averaged 5.19 (SD, 1.92), ranging from 1 to 11, and nearly 40% of the participants had 5 or more comorbidities. The mean (SD) nursing contacts in the sample was 9.9 (4.7), ranging from 1 to 26, and 52 (12.7%) patients withdrew from telehealth service. For patients with HF on telehealth, comorbidity characteristics of anemia, anxiety, musculoskeletal, and depression were significantly associated with nursing utilization patterns, and renal failure, cancer, and depression comorbidities were significantly associated with withdrawal from telehealth service. CLINICAL IMPLICATIONS: Knowledge of the association of comorbidity characteristics with the home care service utilization patterns of patients with HF on telehealth can assist the home health nurse to develop a tailored care plan that attains optimal patient outcomes. Knowledge of such associations would also focus home care resources, avoiding redundancy of resource utilization in this era of strained healthcare resources.

Using process elicitation and validation to understand and improve chemotherapy ordering and delivery.

BACKGROUND: Chemotherapy ordering and administration, in which errors have potentially severe consequences, was quantitatively and qualitatively evaluated by employing process formalism (or formal process definition), a technique derived from software engineering, to elicit and rigorously describe the process, after which validation techniques were applied to confirm the accuracy of the described process. METHODS: The chemotherapy ordering and administration process, including exceptional situations and individuals' recognition of and responses to those situations, was elicited through informal, unstructured interviews with members of an interdisciplinary team. The process description (or process definition), written in a notation developed for software quality assessment purposes, guided process validation (which consisted of direct observations and semistructured interviews to confirm the elicited details for the treatment plan portion of the process). RESULTS: The overall process definition yielded 467 steps; 207 steps (44%) were dedicated to handling 59 exceptional situations. Validation yielded 82 unique process events (35 new expected but not yet described steps, 16 new exceptional situations, and 31 new steps in response to exceptional situations). Process participants actively altered the process as ambiguities and conflicts were discovered by the elicitation and validation components of the study. Chemotherapy error rates declined significantly during and after the project, which was conducted from October 2007 through August 2008. DISCUSSION: Each elicitation method and the subsequent validation discussions contributed uniquely to understanding the chemotherapy treatment plan review process, supporting rapid adoption of changes, improved communication regarding the process, and ensuing error reduction.

Bar-code verification: reducing but not eliminating medication errors.

Using observation, eye tracking, and clinical simulation with embedded errors, we studied the impact of bar-code verification on error identification and recovery during medication administration. Data supported that bar-code verification may reduce but does not eliminate patient identification (ID) and medication errors during clinical simulation of medication administration.

Commercial off-the-shelf consumer health informatics interventions: recommendations for their design, evaluation and redesign.

OBJECTIVE: The goal of this paper is to describe the successful application of a use case-based evaluation approach to guide the effective design, evaluation and redesign of inexpensive, commercial, off-the-shelf consumer health informatics (CHI) interventions. DESIGN: Researchers developed four CHI intervention use cases representing two distinct patient populations (patients with diabetes with high blood pressure, post-bariatric surgery patients), two commercial off-the-shelf CHI applications (Microsoft HealthVault, Google Health), and related devices (blood pressure monitor, pedometer, weight scale). Three patient proxies tested each intervention for 10 days. MEASUREMENTS: The patient proxies recorded their challenges while completing use case tasks, rating the severity of each challenge based on how much it hindered their use of the intervention. Two independent evaluators categorized the challenges by human factors domain (physical, cognitive, macroergonomic). RESULTS: The use case-based approach resulted in the identification of 122 challenges, with 12% physical, 50% cognitive and 38% macroergonomic. Thirty-nine challenges (32%) were at least moderately severe. Nine of 22 use case tasks (41%) accounted for 72% of the challenges. LIMITATIONS: The study used two patient proxies and addressed two specific patient populations and low-cost, off-the-shelf CHI interventions, which may not perfectly generalize to a larger number of proxies, actual patient populations, or other CHI interventions. CONCLUSION: CHI designers can employ the use case-based evaluation approach to assess the fit of a CHI intervention with patients' health work, in the context of their daily activities and environment, which would be difficult or impossible to evaluate by laboratory-based studies.

Nurses' behaviors and visual scanning patterns may reduce patient identification errors.

Patient identification (ID) errors occurring during the medication administration process can be fatal. The aim of this study is to determine whether differences in nurses' behaviors and visual scanning patterns during the medication administration process influence their capacities to identify patient ID errors. Nurse participants (n = 20) administered medications to 3 patients in a simulated clinical setting, with 1 patient having an embedded ID error. Error-identifying nurses tended to complete more process steps in a similar amount of time than non-error-identifying nurses and tended to scan information across artifacts (e.g., ID band, patient chart, medication label) rather than fixating on several pieces of information on a single artifact before fixating on another artifact. Non-error-indentifying nurses tended to increase their durations of off-topic conversations-a type of process interruption-over the course of the trials; the difference between groups was significant in the trial with the embedded ID error. Error-identifying nurses tended to have their most fixations in a row on the patient's chart, whereas non-error-identifying nurses did not tend to have a single artifact on which they consistently fixated. Finally, error-identifying nurses tended to have predictable eye fixation sequences across artifacts, whereas non-error-identifying nurses tended to have seemingly random eye fixation sequences. This finding has implications for nurse training and the design of tools and technologies that support nurses as they complete the medication administration process.