Avoiding Unintended Consequences of Pediatric Blood Order Set Updates through In Situ Usability Testing.

Background Blood product ordering is a complex process, and mistakes can lead to patient harm and poor outcomes. Orders and order sets can be designed to help mitigate errors, but major changes in design can unintentionally cause new errors. Objectives (1) Utilize formative in situ usability testing to iteratively improve the design of a redesigned blood product order set prior to go-live, (2) implement changes based on feedback derived from this testing, and (3) Compare the error rate, System Usability Scale (SUS) score, time to task completion, and click counts between the prior order set in use at the time and the revised redesigned order set. Methods A multidisciplinary project team convened to redesign blood product orders and order sets from scratch based on a review of literature and benchmarking against four pediatric academic institutions with the goal of addressing prior ordering errors. The new redesigned blood product order set was iteratively updated via in situ formative usability testing performed with available clinical users using a concurrent think-aloud protocol in real clinical environments. Errors, SUS scores, time to task completion, and click counts were assessed for the revised redesigned order set using summative testing. Results Formative usability testing with 20 participants led to seven design changes in the redesigned order set which reduced the error rate at go-live. Summative usability testing showed that even though the usability scores were only slightly improved for the revised redesigned order set, the error rates in blood orders were significantly decreased. Conclusion Usability testing can identify design errors early in the process which can be rectified prior to implementation, thus avoiding unintended consequences of changes.

Lessons Learned from OpenNotes Learning Mode and Subsequent Implementation across a Pediatric Health System.

BACKGROUND: The 21st Century Cures Act has accelerated adoption of OpenNotes, providing new opportunities for patient and family engagement in their care. However, these regulations present new challenges, particularly for pediatric health systems aiming to improve information sharing while minimizing risks associated with adolescent confidentiality and safety. OBJECTIVE: Describe lessons learned preparing for OpenNotes across a pediatric health system during a 4-month trial period (referred to as "Learning Mode") in which clinical notes were not shared by default but decision support was present describing the upcoming change and physicians could request feedback on complex cases from a multidisciplinary team. METHODS: During Learning Mode (December 3, 2020-March 9, 2021), implementation included (1) educational text at the top of commonly used note types indicating that notes would soon be shared and providing guidance, (2) a new confidential note type, and (3) a mechanism for physicians to elicit feedback from a multidisciplinary OpenNotes working group for complex cases with questions related to OpenNotes. The working group reviewed lessons learned from this period, as well as implementation of OpenNotes from March 10, 2021 to June 30, 2021. RESULTS: During Learning Mode, 779 confidential notes were written across the system. The working group provided feedback on 14 complex cases and also reviewed 7 randomly selected confidential notes. The proportion of physician notes shared with patients increased from 1.3% to 88.4% after default sharing of notes to the patient portal. Key lessons learned included (1) sensitive information was often present in autopopulated elements, differential diagnoses, and supervising physician note attestations; and (2) incorrect reasons were often selected by clinicians for withholding notes but this accuracy improved with new designs. CONCLUSION: While OpenNotes provides an unprecedented opportunity to engage pediatric patients and their families, targeted education and electronic health record designs are needed to mitigate potential harms of inappropriate disclosures.

Formative Usability Testing Reduces Severe Blood Product Ordering Errors.

BACKGROUND: Medical errors in blood product orders and administration are common, especially for pediatric patients. A failure modes and effects analysis in our health care system indicated high risk from the electronic blood ordering process. OBJECTIVES: There are two objectives of this study as follows:(1) To describe differences in the design of the original blood product orders and order sets in the system (original design), new orders and order sets designed by expert committee (DEC), and a third-version developed through user-centered design (UCD).(2) To compare the number and type of ordering errors, task completion rates, time on task, and user preferences between the original design and that developed via UCD. METHODS: A multidisciplinary expert committee proposed adjustments to existing blood product order sets resulting in the DEC order set. When that order set was tested with front-line users, persistent failure modes were detected, so orders and order sets were redesigned again via formative usability testing. Front-line users in their native clinical workspaces were observed ordering blood in realistic simulated scenarios using a think-aloud protocol. Iterative adjustments were made between participants. In summative testing, participants were randomized to use the original design or UCD for five simulated scenarios. We evaluated differences in ordering errors, time on task, and users' design preference with two-sample t-tests. RESULTS: Formative usability testing with 27 providers from seven specialties led to 18 changes made to the DEC to produce the UCD. In summative testing, error-free task completion for the original design was 36%, which increased to 66% in UCD (30%, 95% confidence interval [CI]: 3.9-57%; p = 0.03). Time on task did not vary significantly. CONCLUSION: UCD led to substantially different blood product orders and order sets than DEC. Users made fewer errors when ordering blood products for pediatric patients in simulated scenarios when using the UCD orders and order sets compared with the original design.