Electronic Health Record Legal Settlements in the US Since the 2009 Health Information Technology for Economic and Clinical Health Act.

This cross-sectional study describes allegations of misconduct by electronic health record vendors.

Development and pilot evaluation of an electronic health record usability and safety self-assessment tool.

Objective: Poor electronic health record (EHR) usability contributes to clinician burnout and poses patent safety risks. Site-specific customization and configuration of EHRs require individual EHR system usability and safety testing which is resource intensive. We developed and pilot-tested a self-administered EHR usability and safety assessment tool, focused on computerized provider order entry (CPOE), which can be used by any facility to identify specific issues. In addition, the tool provides recommendations for improvement. Materials and Methods: An assessment tool consisting of 104 questions was developed and pilot-tested at 2 hospitals, one using a Cerner EHR and the other using Epic. Five physicians at each site participated in and completed the assessment. Participant response accuracy compared to actual EHR interactions, consistency across participants, and usability issues identified through the tool were measured at each site. Results: Across sites, participants answered an average of 46 questions in 23 min with 89.9% of responses either correct or partially correct. The tool identified 8 usability and safety issues at one site and 7 at the other site across medication, laboratory, and radiology CPOE functions. Discussion: The tool shows promise as a method to rapidly evaluate EHR usability and safety and provide guidance on specific areas for improvement. Important improvements to the evaluation tool were identified including the need to clarify certain questions and provide definitions for usability terminology. Conclusion: A self-administered usability and safety assessment tool can serve to identify specific usability and safety issues in the EHR and provide guidance for improvements.

Usability and Accessibility of Publicly Available Patient Safety Databases.

OBJECTIVES: The aims of the study were to identify publicly available patient safety report databases and to determine whether these databases support safety analyst and data scientist use to identify patterns and trends. METHODS: An Internet search was conducted to identify publicly available patient safety databases that contained patient safety reports. Each database was analyzed to identify features that enable patient safety analyst and data scientist use of these databases. RESULTS: Seven databases (6 hosted by federal agencies, 1 hosted by a nonprofit organization) containing more than 28.3 million safety reports were identified. Some, but not all, databases contained features to support patient safety analyst use: 57.1% provided the ability to sort/compare/filter data, 42.9% provided data visualization, and 85.7% enabled free-text search. None of the databases provided regular updates or monitoring and only one database suggested solutions to patient safety reports. Analysis of features to support data scientist use showed that only 42.9% provided an application programing interface, most (85.7%) provided batch downloading, all provided documentation about the database, and 71.4% provided a data dictionary. All databases provided open access. Only 28.6% provided a data diagram. CONCLUSIONS: Patient safety databases should be improved to support patient safety analyst use by, at a minimum, allowing for data to be sorted/compared/filtered, providing data visualization, and enabling free-text search. Databases should also enable data scientist use by, at a minimum, providing an application programing interface, batch downloading, and a data dictionary.

Identifying Health Information Technology Usability Issues Contributing to Medication Errors Across Medication Process Stages.

OBJECTIVE: Different health information technology (health IT) systems are intended to support medication ordering, reviewing, and administration. We sought to identify the types of medication errors associated with health IT use, whether they reached the patient, where in the medication process those errors occurred, and the specific usability issues contributing to those errors. METHODS: Patient safety event reports from more than 595 healthcare facilities entered between January 2013 and September 2018 were analyzed. We computationally identified reports associated with health IT intended to support the medication process, including computerized provider order entry, electronic medication administration record, and barcode medication administration. From these, 2700 reports were manually reviewed to determine the type of medication error, medication process stage, and health IT usability issue. RESULTS: Of the 2700 manually reviewed reports, 1508 (55.9%) described a medication error that was associated with health IT use and 750 (49.7%) reached the patient. Improper dose errors were frequent (1214 of 1508, 80.5%) with most errors during ordering (673 of 1508, 44.6%) and reviewing medications (639 of 1508, 42.4%). Most health IT-associated medication error reports described usability issues (n = 1468 of 1508, 97.3%) including data entry, workflow support, and alerting. Data entry usability issues impacted few medication process stages, whereas workflow support and alerting impacted several stages. CONCLUSIONS: Health IT usability issues are a prevalent contributing factor to medication errors, many of which reach the patient. Data entry, workflow support, and alerting should be prioritized during usability and safety optimization efforts.

Emergency Physician Perceptions of Electronic Health Record Usability and Safety.

OBJECTIVES: Despite requirements for electronic health record (EHR) vendor usability testing, usability challenges persist, contributing to patient safety concerns. We sought to identify emergency physicians' perceived EHR usability and safety strengths and shortcomings across major EHR vendor products. METHODS: Fifty-five emergency physicians from 4 different hospitals were interviewed. The interviews were qualitatively analyzed, and physician comments were aligned with a usability taxonomy to identify emerging themes by vendor and hospital. RESULTS: Of the 194 comments about usability, the 3 most commonly discussed usability topics were Workflow Support (33.5% of comments), Visual Display (20.1%), and Data Entry (14.4%). Electronic health record usability strengths were centered on Visual Display, and the most common shortcoming was the lack of Workflow Support. Fourteen cross-hospital/cross-vendor themes, 6 vendor-specific themes, and 4 hospital-specific themes were identified. CONCLUSIONS: Usability shortcomings that spanned across hospitals and vendors may suggest a need for more applied research and improved design to resolve these issues. Shortcomings that are localized to a specific product or hospital may be due to customization and may be addressable by learning from other organizations.

Workflow Analysis Driven Recommendations for Integration of Electronically-Enhanced Sexually Transmitted Infection Screening Tools in Pediatric Emergency Departments.

Adolescents are disproportionately affected by sexually transmitted infections (STIs). Failure to diagnose and treat STIs in a timely manner may result in serious sequelae. Adolescents frequently access the emergency department (ED) for care. Although ED-based STI screening is acceptable to both patients and clinicians, understanding how best to implement STI screening processes into the ED clinical workflow without compromising patient safety or efficiency is critical. The objective of this study was to conduct direct observations documenting current workflow processes and tasks during patient visits at six Pediatric Emergency Care Applied Research Network (PECARN) EDs for site-specific integration of STI electronically-enhanced screening processes. Workflow observations were captured via TaskTracker, a time and motion electronic data collection application that allows researchers to categorize general work processes and record multitasking by providing a timestamp of when tasks began and ended. Workflow was captured during 118 patient visits across six PECARN EDs. The average time to initial assessment by the most senior provider was 76 min (range 59-106 min, SD = 43 min). Care teams were consistent across sites, and included attending physicians, advanced practice providers, nurses, registration clerks, technicians, and students. A timeline belt comparison was performed. Across most sites, the most promising implementation of a STI screening tool was in the patient examination room following the initial patient assessment by the nurse.

Varying rates of patient identity verification when using computerized provider order entry.

OBJECTIVE: We sought to determine rates of computerized provider order entry (CPOE) patient identity verification and when and where in the ordering process verification occurred. MATERIALS AND METHODS: Fifty-five physicians from 4 healthcare systems completed simulated patient scenarios using their respective CPOE system (Epic or Cerner). Eye movements were recorded and analyzed. RESULTS: Across all participants patient id was verified significantly more often than not (62.4% vs 37.6%). Vendor A had significantly higher verification rates than not; vendor B had no difference. Participants using vendor A verified information significantly more often before signing the order than after (88.4% vs 11.6%); there was no difference in vendor B. The banner bar was the most frequent verification location. DISCUSSION: Factors such as CPOE design, physician training, and the use of a simulated methodology may be impacting verification rates. CONCLUSIONS: Verification rates vary by CPOE product, and this can have patient safety consequences.

Exploration and Initial Development of Text Classification Models to Identify Health Information Technology Usability-Related Patient Safety Event Reports.

BACKGROUND: With the pervasive use of health information technology (HIT) there has been increased concern over the usability and safety of this technology. Identifying HIT usability and safety hazards, mitigating those hazards to prevent patient harm, and using this knowledge to improve future HIT systems are critical to advancing health care. PURPOSE: The purpose of this work is to demonstrate the feasibility of a modeling approach to identify HIT usability-related patient safety events (PSEs) from the free-text of safety reports and the utility of such models for supporting patient safety analysts in their analysis of event data. METHODS: We evaluated three feature representations (bag-of-words [BOWs], topic modeling, and document embeddings) to classify HIT usability-related PSE reports using 5,911 manually annotated reports. Model results were reviewed with patient safety analysts to gather feedback on their usefulness and integration into workflow. RESULTS: The combination of term frequency-inverse document frequency BOWs and document embedding features modeled with support vector machine (SVM) with radial basis function (RBF) had the highest overall precision-recall area under the curve (AUC) and f1-score, 72 and 66%, respectively. Using only document embedding features achieved a similar precision-recall AUC and f1-score performance with the SVM RBF model, 70 and 66%, respectively. Models generally favored specificity and sensitivity over precision. Patient safety analysts found the model results to be useful and offered three suggestions on how it can be integrated into their workflow at the point of report entry, in a visual dashboard layer, and to support data retrievals. CONCLUSION: Text mining and document embeddings can support identification of HIT usability-related PSE reports. The positive feedback received on the HIT usability model shows its potential utility in real-world applications.

A socio-technical systems approach to the use of health IT for patient reported outcomes: Patient and healthcare provider perspectives.

BACKGROUND: Patient-Reported Outcomes (PROs) can be used to inform the clinical management of individuals, including patient self-management, care planning, and goal setting. Despite a rapid proliferation of technology to collect and integrate PROs in clinical care, uptake by patients and healthcare providers remains sub optimal. A consideration of systems factors to understand these challenges is needed. OBJECTIVES: To apply the socio-technical systems (STS) model as a framework for understanding the usability and functional requirements of patients collecting PRO data using applications (apps), and of healthcare providers using these data at the point of care in ambulatory settings. METHODS: With questions guided by the STS model, semi-structured interviews were conducted with eighteen patients and nine healthcare providers to elicit feedback about facilitators and barriers to successful use of PRO apps and PRO data in ambulatory settings. Patient participants were selected to fit into two categories: older, low utilizers of technology with less than a bachelor's degree, and younger higher utilizers of technology with at least a bachelor's degree. Participants were from primary and specialty care practices. Data were analyzed inductively to identify emergent themes. RESULTS: Younger patients were only interested in using a PRO app if they had an active health issue to track. The nine older patients preferred passive means of data collection if they were to track a health issue, and preferred direct contact with their healthcare provider and using office visits to share information. All patients desired optimal usability and emphasized bidirectional communication in an app that is transparent about privacy. All nine healthcare providers agreed that PRO data would be most useful and relevant if key patient populations were targeted based on the specific measure. In this case the healthcare providers noted potentially optimal utility of collecting physical function PRO data for patients 65 and older. Access to the data was highlighted by each healthcare provider stating that these data would be most useful if they were seamlessly integrated into the electronic health record. DISCUSSION: Several emergent themes were identified under the five selected dimensions of the STS model (clinical content, human computer interface, hardware and software computing infrastructure, people, and workflow and communication). Findings highlighted the continued need for innovative methods to obtain more rapid cycle, continuous feedback to identify system factors impacting use of these technologies. CONCLUSION: The STS model provides a comprehensive framework that can be applied to collect patient and healthcare provider feedback to better guide the design and implementation of new health information technology.

Identifying health information technology related safety event reports from patient safety event report databases.

OBJECTIVE: The objective of this paper was to identify health information technology (HIT) related events from patient safety event (PSE) report free-text descriptions. A difference-based scoring approach was used to prioritize and select model features. A feature-constraint model was developed and evaluated to support the analysis of PSE reports. METHODS: 5287 PSE reports manually coded as likely or unlikely related to HIT were used to train unigram, bigram, and combined unigram-bigram logistic regression and support vector machine models using five-fold cross validation. A difference-based scoring approach was used to prioritize and select unigram and bigram features by their relative importance to likely and unlikely HIT reports. A held-out set of 2000 manually coded reports were used for testing. RESULTS: Unigram models tended to perform better than bigram and combined models. A 300-unigram logistic regression had comparable classification performance to a 4030-unigram SVM model but with a faster relative run-time. The 300-unigram logistic regression model evaluated with the testing data had an AUC of 0.931 and a F1-score of 0.765. DISCUSSION: A difference-based scoring, prioritization, and feature selection approach can be used to generate simplified models with high performance. A feature-constraint model may be more easily shared across healthcare organizations seeking to analyze their respective datasets and customized for local variations in PSE reporting practices. CONCLUSION: The feature-constraint model provides a method to identify HIT-related patient safety hazards using a method that is applicable across healthcare systems with variability in their PSE report structures.

The Impact of Interruptions on Chest Radiograph Interpretation: Effects on Reading Time and Accuracy.

RATIONALE AND OBJECTIVES: The objective of this study was to experimentally test the effect of interruptions on image interpretation by comparing reading time and response accuracy of interrupted case reads to uninterrupted case reads in resident and attending radiologists. MATERIALS AND METHODS: Institutional review board approval was obtained before participant recruitment from an urban academic health-care system during January 2016-March 2016. Eleven resident and 12 attending radiologists examined 30 chest radiographs, rating their confidence regarding the presence or the absence of a pneumothorax. Ten cases were normal (ie, no pneumothorax present), 10 cases had an unsubtle pneumothorax (ie, readily perceivable by a nonexpert), and 10 cases had a subtle pneumothorax. During three reads of each case type, the participants were interrupted with 30 seconds of a secondary task. The total reading time and the accuracy of interrupted and uninterrupted cases were compared. A mixed-factors analysis of variance was run on reading time and accuracy with experience (resident vs attending) as a between-subjects factor and case type (normal, unsubtle, or subtle) and interruption (interruption vs no interruption) as within-subjects factors. RESULTS: Interrupted tasks had significantly longer reading times than uninterrupted cases (P = .032). During subtle cases, interruptions reduced accuracy (P = .034), but during normal cases, interruptions increased accuracy (P = .038). CONCLUSIONS: Interruptions increased reading times and increased the tendency for a radiologist to conclude that a case is normal for both resident and attending radiologists, demonstrating that interruptions reduce efficiency and introduce patient safety concerns during reads of abnormal cases.

Using Active Learning to Identify Health Information Technology Related Patient Safety Events.

The widespread adoption of health information technology (HIT) has led to new patient safety hazards that are often difficult to identify. Patient safety event reports, which are self-reported descriptions of safety hazards, provide one view of potential HIT-related safety events. However, identifying HIT-related reports can be challenging as they are often categorized under other more predominate clinical categories. This challenge of identifying HIT-related reports is exacerbated by the increasing number and complexity of reports which pose challenges to human annotators that must manually review reports. In this paper, we apply active learning techniques to support classification of patient safety event reports as HIT-related. We evaluated different strategies and demonstrated a 30% increase in average precision of a confirmatory sampling strategy over a baseline no active learning approach after 10 learning iterations.

An Analysis of Patient Safety Incident Reports Associated with Electronic Health Record Interoperability

Background: With the widespread use of electronic health records (EHRs) for many clinical tasks, interoperability with other health information technology (health IT) is critical for the effective delivery of care. While it is generally recognized that poor interoperability negatively impacts patient care, little is known about the specific patient safety implications. Understanding the patient safety implications will help prioritize interoperability efforts around architectures and standards. Objectives: Our objectives were to (1) identify patient safety incident reports that reflect EHR interoperability challenges with other health IT, and (2) perform a detailed analysis of these reports to understand the health IT systems involved, the clinical care processes impacted, whether the incident occurred within or between provider organizations, and the reported severity of the patient safety events. Methods: From a database of 1.735 million patient safety event (PSE) reports spanning multiple provider organizations, 2625 reports that were indicated as being health IT related by the event reporter were reviewed to identify EHR interoperability related reports. Through a rigorous coding process 209 EHR interoperability related events were identified and coded. Results: The majority of EHR interoperability PSE reports involved interfacing with pharmacy systems (i.e. medication related), followed by laboratory, and radiology. Most of the interoperability challenges in these clinical areas were associated with the EHR receiving information from other health IT systems as opposed to the EHR sending information to other systems. The majority of EHR interoperability challenges were within a provider organization and while many of the safety events reached the patient, only a few resulted in patient harm. Conclusions: Interoperability efforts should prioritize systems in pharmacy, laboratory, and radiology. Providers should recognize the need to improve EHRs interfacing with other health IT systems within their own organization.

The Effects of Load Carriage and Physical Fatigue on Cognitive Performance.

In the current study, ten participants walked for two hours while carrying no load or a 40 kg load. During the second hour, treadmill grade was manipulated between a constant downhill or changing between flat, uphill, and downhill grades. Throughout the prolonged walk, participants performed two cognitive tasks, an auditory go no/go task and a visual target detection task. The main findings were that the number of false alarms increased over time in the loaded condition relative to the unloaded condition on the go no/go auditory task. There were also shifts in response criterion towards responding yes and decreased sensitivity in responding in the loaded condition compared to the unloaded condition. In the visual target detection there were no reliable effects of load carriage in the overall analysis however, there were slower reaction times in the loaded compared to unloaded condition during the second hour.

Seeing the crowd for the bomber: Spontaneous threat perception from static and randomly moving crowd simulations.

Human observers are often relied upon for monitoring suspicious crowd behavior in both civilian and military contexts. However, little research has examined what individual- and crowd-level variables independently and interactively modulate threat perception among human observers. Five experiments gathered threat estimates while participants viewed static or dynamic crowd simulations. Experiments 1 and 2 used static crowd stimuli and manipulated crowd size (number of entities), crowd density (distance between entities), and historical information about adverse events. Experiments 3-5 used moving crowd stimuli and either fixed (Experiment 3) or dynamic (Experiment 4-5) crowd size and density. Experiments 4 and 5 further examined several individual- and crowd-level parameters subjectively reported by observers as critical to generating risk estimates. Overall, results demonstrated that human observers rely heavily on both crowd size and density cues, but also consider several other cues, such as perceived individual isolation and grouping behavior, when estimating risk levels within a crowd. We also show that reliance on such parameters is highly variable across participants in terms of both directionality and magnitude. Results are discussed within the context of continuing sensor system and modeling efforts, and understanding how threat perception emerges from the observation of intentional agents.

Variable transmission lens influences on the dynamics of pupillary light reflexes.

This study examined the influence of liquid crystal variable transmission lenses on pupillary light reflexes in response to sudden bright light onset. Participants were exposed to bright light while pupil size was monitored using an eye tracker; eyewear was configured across four transition conditions: constant low-light filtering, constant high-light filtering, variable-light filtering in response to light detection and a control condition without eyewear. Before light onset, pupil diameter was largest in the high-filter condition, medium in the variable- and low-light filtering conditions and smallest in the control condition. Following light onset, the low-light filtering and control conditions, and the high-light filtering and variable-light filtering conditions converged over time. Critically, automatically transitioning between low- and high-light filtering reduced the magnitude (approximately 0.2 mm) and duration (approximately 360 ms) of the pupillary response relative to constant low-light filtering.