Usability Testing of Situation Awareness Clinical Decision Support in the Intensive Care Unit.

OBJECTIVE: Our objective was to evaluate the usability of an automated clinical decision support (CDS) tool previously implemented in the pediatric intensive care unit (PICU) to promote shared situation awareness among the medical team to prevent serious safety events within children's hospitals. METHODS: We conducted a mixed-methods usability evaluation of a CDS tool in a PICU at a large, urban, quaternary, free-standing children's hospital in the Midwest. Quantitative assessment was done using the system usability scale (SUS), while qualitative assessment involved think-aloud usability testing. The SUS was scored according to survey guidelines. For think-aloud testing, task times were calculated, and means and standard deviations were determined, stratified by role. Qualitative feedback from participants and moderator observations were summarized. RESULTS: Fifty-one PICU staff members, including physicians, advanced practice providers, nurses, and respiratory therapists, completed the SUS, while ten participants underwent think-aloud usability testing. The overall median usability score was 87.5 (interquartile range: 80-95), with over 96% rating the tool's usability as "good" or "excellent." Task completion times ranged from 2 to 92 seconds, with the quickest completion for reviewing high-risk criteria and the slowest for adding to high-risk criteria. Observations and participant responses from think-aloud testing highlighted positive aspects of learnability and clear display of complex information that is easily accessed, as well as opportunities for improvement in tool integration into clinical workflows. CONCLUSION: The PICU Warning Tool demonstrates good usability in the critical care setting. This study demonstrates the value of postimplementation usability testing in identifying opportunities for continued improvement of CDS tools.

Use of design thinking and human factors approach to improve situation awareness in the pediatric intensive care unit.

BACKGROUND: Optimal design of healthcare spaces can enhance patient care. We applied design thinking and human factors principles to optimize communication and signage on high risk patients to improve situation awareness in a new clinical space for the pediatric ICU. OBJECTIVE: To assess the impact of these tools in mitigating situation awareness concerns within the new clinical space. We hypothesized that implementing these design-informed tools would either maintain or improve situation awareness. DESIGN, SETTINGS, AND PARTICIPANTS: A 15-week design thinking process was employed, involving research, ideation, and refinement to develop and implement new situation awareness tools. The process included engagement with interprofessional clinical teams, scenario planning, workflow mapping, iterative feedback collection, and collaboration with an industry partner for signage development and implementation. INTERVENTION: Improved and updated communication devices and bedside mitigation plans. MAIN OUTCOME AND MEASURES: Process metrics included individual and shared situation awareness of PICU care teams and our patient outcome metric was the rate of cardiopulmonary resuscitation (CPR) events pre- and post-transition. RESULTS: When evaluating all patients, shared situation awareness for accurate high-risk status improved from 81% pre-transition to 92% post-transition (p = .006). When assessing individual care team roles, accuracy of patient high-risk status improved from 88% to 95% (p = .05) for RNs, 85% to 96% (p = .003) for residents, and 88% to 95% (p = .03) for RTs. There was no change in the rate of CPR events following the transition.