Safeuristics! Do Heuristic Evaluation Violation Severity Ratings Correlate with Patient Safety Severity Ratings for a Native Electronic Health Record Mobile Application?

OBJECTIVE: Usability of electronic health records (EHRs) remains challenging, and poor EHR design has patient safety implications. Heuristic evaluation detects usability issues that can be classified by severity. The National Institute of Standards and Technology provides a safety scale for EHR usability. Our objectives were to investigate the relationship between heuristic severity ratings and safety scale ratings in an effort to analyze EHR safety. MATERIALS AND METHODS: Heuristic evaluation was conducted on seven common mobile EHR tasks, revealing 58 heuristic violations and 28 unique usability issues. Each usability issue was independently scored for severity by trained hospitalists and a Human Factors researcher and for safety severity by two physician informaticists and two clinical safety professionals. RESULTS: Results demonstrated a positive correlation between heuristic severity and safety severity ratings. Regression analysis demonstrated that 49% of safety risk variability by clinical safety professionals (r = 0.70; n = 28) and 42% of safety risk variability by clinical informatics specialists (r = 0.65; n = 28) was explained by usability severity scoring of problems outlined by heuristic evaluation. Higher severity ratings of the usability issues were associated with increased perceptions of patient safety risk. DISCUSSION: This study demonstrated the use of heuristic evaluation as a technique to quickly identify usability problems in an EHR that could lead to safety issues. Detection of higher severity ratings could help prioritize failures in EHR design that more urgently require design changes. This approach is a cost-effective technique for improving usability while impacting patient safety. CONCLUSION: Results from this study demonstrate the efficacy of the heuristic evaluation technique to identify usability problems that impact safety of the EHR. Also, the use of interdisciplinary teams for evaluation should be considered for severity assessment.

Development and implementation of a mobile device-based pediatric electronic decision support tool as part of a national practice standardization project.

Objective: Implementing evidence-based practices requires a multi-faceted approach. Electronic clinical decision support (ECDS) tools may encourage evidence-based practice adoption. However, data regarding the role of mobile ECDS tools in pediatrics is scant. Our objective is to describe the development, distribution, and usage patterns of a smartphone-based ECDS tool within a national practice standardization project. Materials and Methods: We developed a smartphone-based ECDS tool for use in the American Academy of Pediatrics, Value in Inpatient Pediatrics Network project entitled "Reducing Excessive Variation in the Infant Sepsis Evaluation (REVISE)." The mobile application (app), PedsGuide, was developed using evidence-based recommendations created by an interdisciplinary panel. App workflow and content were aligned with clinical benchmarks; app interface was adjusted after usability heuristic review. Usage patterns were measured using Google Analytics. Results: Overall, 3805 users across the United States downloaded PedsGuide from December 1, 2016, to July 31, 2017, leading to 14 256 use sessions (average 3.75 sessions per user). Users engaged in 60 442 screen views, including 37 424 (61.8%) screen views that displayed content related to the REVISE clinical practice benchmarks, including hospital admission appropriateness (26.8%), length of hospitalization (14.6%), and diagnostic testing recommendations (17.0%). Median user touch depth was 5 [IQR 5]. Discussion: We observed rapid dissemination and in-depth engagement with PedsGuide, demonstrating feasibility for using smartphone-based ECDS tools within national practice improvement projects. Conclusions: ECDS tools may prove valuable in future national practice standardization initiatives. Work should next focus on developing robust analytics to determine ECDS tools' impact on medical decision making, clinical practice, and health outcomes.