A human factors approach to subcutaneous insulin chart design improves user-performance: An experimental study.

Insulin is a high-risk medicine that has been implicated in serious adverse events for hospital inpatients, including medication-error related deaths. Most insulin errors occur during administration, and "wrong dose" is the most common type. A paper-based subcutaneous insulin chart (the "NSIC") was developed for the Australian Commission on Safety and Quality in Health Care, using a range of human factors methods, with the aim of reducing the opportunity for errors. The present lab-based study empirically assessed whether the NSIC's human factors design translates into improved user-performance in the determination of insulin doses, compared with a pre-existing chart. Forty-one experienced nurses and 48 novice chart-users completed 60 experimental trials (30 per chart), in which they determined doses to administer to patients. Both groups determined insulin doses faster, and made fewer dose errors, when using the NSIC. These results support the utility of the usability heuristics employed in developing the chart.

Quantitative systematic review: Sources of inaccuracy in manually measured adult respiratory rate data.

AIMS: To identify the potential sources of inaccuracy in manually measured adult respiratory rate (RR) data and quantify their effects. DESIGN: Quantitative systematic review with meta-analyses where appropriate. DATA SOURCES: Medline, CINAHL, and Cochrane Library (from database inception to 31 July 2019). REVIEW METHODS: Studies presenting data on individual sources of inaccuracy in the manual measurement of adult RR were analysed, assessed for quality, and grouped according to the source of inaccuracy investigated. Quantitative data were extracted and synthesized and meta-analyses performed where appropriate. RESULTS: Included studies (N = 49) identified five sources of inaccuracy. The awareness effect creates an artefactual reduction in actual RR, and observation methods involving shorter counts cause systematic underscoring. Individual RR measurements can differ substantially in either direction between observations due to inter- or intra-observer variability. Value bias, where particular RRs are over-represented (suggesting estimation), is a widespread problem. Recording omission is also widespread, with higher average rates in inpatient versus triage/admission contexts. CONCLUSION: This review demonstrates that manually measured RR data are subject to several potential sources of inaccuracy. IMPACT: RR is an important indicator of clinical deterioration and commonly included in track-and-trigger systems. However, the usefulness of RR data depends on the accuracy of the observations and documentation, which are subject to five potential sources of inaccuracy identified in this review. A single measurement may be affected by several factors. Hence, clinicians should interpret recorded RR data cautiously unless systems are in place to ensure its accuracy. For nurses, this includes counting rather than estimating RRs, employing 60-s counts whenever possible, ensuring patients are unaware that their RR is being measured, and documenting the resulting value. For any given site, interventions to improve measurement should take into account the local organizational and cultural context, available resources, and the specific measurement issues that need to be addressed.

A novel training device for tip control in colonoscopy: preliminary validation and efficacy as a training tool.

BACKGROUND: Effective control of the colonoscope tip is one of the most fundamental components of colonoscopy skill. Mastering fine tip control can be problematic for novice trainees, yet no validated training regimes exist for developing this specific skill component in isolation. We aimed to conduct a preliminary validation of a novel training device for colonoscopic tip control, and to assess its efficacy as a training tool. METHODS: In study 1 (validation), 13 experienced colonoscopists and 16 novices used a colonoscope to accurately track 28 targets on each of four concave "training surfaces" as quickly as possible, and we compared their performance. In study 2 (pre-post-training study), another 16 novices were tested before and after a six-session training program. In both studies, the main outcome measurements were completion time (measured automatically by the device) and variability of individual performance (the SD of each individual's completion times across trials). RESULTS: Compared with novices, experienced colonoscopists were faster (P < 0.0001) and their performances less variable (P < 0.0001). With training, novices became faster (P < 0.0001) and more consistent (P = 0.003), and these improvements also generalized to novel training surfaces (P's < 0.01). After training, the novices' tip control performance was indistinguishable from that of the experienced colonoscopists (P's > 0.05). The composite measures of completion time used in both studies all had acceptable to excellent internal consistency reliability (α's ranged from 0.72 to 0.93). CONCLUSIONS: We found that performance measures derived from using the device to assess skill can discriminate between experienced colonoscopists and novices in terms of their ability to control and guide the colonoscope tip precisely, providing preliminary evidence to support the construct validity of the metrics. The device is also an effective training tool for this fundamental component of colonoscopy skill.

Observation chart design features affect the detection of patient deterioration: a systematic experimental evaluation.

AIM: To systematically evaluate the impact of several design features on chart-users' detection of patient deterioration on observation charts with early-warning scoring-systems. BACKGROUND: Research has shown that observation chart design affects the speed and accuracy with which abnormal observations are detected. However, little is known about the contribution of individual design features to these effects. DESIGN: A 2 × 2 × 2 × 2 mixed factorial design, with data-recording format (drawn dots vs. written numbers), scoring-system integration (integrated colour-based system vs. non-integrated tabular system) and scoring-row placement (grouped vs. separate) varied within-participants and scores (present vs. absent) varied between-participants by random assignment. METHODS: 205 novice chart-users, tested between March 2011-March 2014, completed 64 trials where they saw real patient data presented on an observation chart. Each participant saw eight cases (four containing abnormal observations) on each of eight designs (which represented a factorial combination of the within-participants variables). On each trial, they assessed whether any of the observations were physiologically abnormal, or whether all observations were normal. Response times and error rates were recorded for each design. RESULTS: Participants responded faster (scores present and absent) and made fewer errors (scores absent) using drawn-dot (vs. written-number) observations and an integrated colour-based (vs. non-integrated tabular) scoring-system. Participants responded faster using grouped (vs. separate) scoring-rows when scores were absent, but separate scoring-rows when scores were present. CONCLUSION: Our findings suggest that several individual design features can affect novice chart-users' ability to detect patient deterioration. More broadly, the study further demonstrates the need to evaluate chart designs empirically.

Less is more: the design of early-warning scoring systems affects the speed and accuracy of scoring.

AIM: To evaluate the effect of early-warning scoring system design on the speed and accuracy of scoring. BACKGROUND: Despite the widespread implementation of early-warning scoring systems in hospitals, the speed and accuracy with which chart-users determine patients' early-warning scores has received minimal research attention. DESIGN: Within-subjects, with scoring-system design as the independent variable. METHODS: Forty-seven novice chart-users were presented with realistic vital sign observations recorded on charts with three different scoring-system designs. The rows for recording individual vital sign scores were either: (1) grouped together beneath all of the vital sign rows; (2) separated, with each row presented immediately below the corresponding vital sign row; or (3) excluded altogether. Participants' response times and error rates for determining the overall scores were measured for 54 time-points per design. Data were collected in December 2012-January 2013. RESULTS: Contrary to predictions, participants responded fastest and made the fewest errors when using the chart design without individual vital sign scoring-rows. For the other two designs, participants were faster when the rows for scoring individual vital signs were separated (vs. grouped), but accuracy did not differ. For both of these designs, significantly more time-points were affected by scoring errors compared with adding errors. Finally, data for patients with more serious derangements yielded greater response times and error rates on all three charts. CONCLUSION: Early-warning scoring systems may be more effective without individual vital sign scoring-rows. Even when charts are designed by multi-disciplinary teams of human factors specialists and clinicians, empirical evaluations are essential.

Observation charts with overlapping blood pressure and heart rate graphs do not yield the performance advantage that health professionals assume: an experimental study.

AIM: To investigate whether overlapping blood pressure and heart rate graphs improve chart-users' ability to recognize derangements in these vital signs on hospital observation charts. BACKGROUND: Many health professionals prefer blood pressure and heart rate graphs to overlap. One justification is the use of a visual cue called the 'Seagull Sign' to detect physiological abnormalities. DESIGN: A 3 × 2 × 2 mixed-design experiment, with three independent variables: participant group, graph format (separate vs. overlapping) and alerting system (integrated colour-based track-and-trigger system present vs. absent). METHODS: Over 64 experimental trials, 'Seagull-trained' nurses and novices randomly assigned to receive 'Seagull training' or remain untrained, viewed sequences of blood pressure and heart rate observations recorded on four different chart design extracts. The designs represented a crossing of the graph format and alerting system variables. For each design, eight cases contained normal data and eight contained an abnormal systolic blood pressure or heart rate observation (half of which yielded a Seagull Sign on overlapping plots). Participants (tested between January-May 2011) judged whether observations were physiologically normal or abnormal. RESULTS: Across all cases, participants from all groups responded faster and made fewer errors when blood pressure and heart rate observations were graphed separately, especially when a track-and-trigger system was present. Even for 'Seagull-trained' participants viewing 'Seagull Sign available' cases, no advantage of overlapping graphs was found. CONCLUSIONS: These findings suggest that overlapping graphs do not yield the performance advantage that many health professionals assume, either for novices or experienced nurses, even when the Seagull Sign is used.

A human factors approach to observation chart design can trump health professionals' prior chart experience.

AIM: To determine whether experienced health professionals recognise patient deterioration more accurately and efficiently using (a) novel observation charts, designed from a human factors perspective, or (b) chart designs with which they have long-term experience. METHODS: Participants were 101 health professionals experienced in using either a multiple parameter track-and-trigger chart or a graphical chart with no track-and-trigger system. Participants were presented with realistic abnormal and normal patient observations recorded on six hospital observation charts of varying design quality, including the chart that participants were familiar with (or a very similar design). Across 48 trials, the participant was asked to specify if any of the vital sign observations were abnormal, or if all of the observations were normal. Participants' overall error rates (i.e., proportion of incorrect responses) and response times, the main outcome measures, were calculated for each observation chart. RESULTS: Participants made significantly fewer errors and responded significantly faster when using a novel user-friendly chart compared with all the other designs, including the charts that they were experienced with in a clinical setting. CONCLUSIONS: The findings suggest that, at least in the contexts examined, superior observation chart design appears to trump familiarity. Hence, hospitals motivated to improve the detection of patient deterioration should implement charts designed from a human factors perspective, rather than simply maintaining the status quo of reliance on clinical experience.