Towards a safer culture: implementing multidisciplinary simulation-based team training in New Zealand operating theatres - a framework analysis.

AIM: NetworkZ is a simulation-based multidisciplinary team-training programme designed to enhance patient safety by improving communication and teamwork in operating theatres (OTs). In partnership with the Accident Compensation Corporation, its implementation across New Zealand (NZ) began in 2017. Our aim was to explore the experiences of staff - including the challenges they faced - in implementing NetworkZ in NZ hospitals, so that we could improve the processes necessary for subsequent implementation. METHOD: We interviewed staff from five hospitals involved in the initial implementation of NetworkZ, using the Organising for Quality model as the framework for analysis. This model describes embedding successful quality improvement as a process of overcoming six universal challenges: structure, infrastructure, politics, culture, motivation and learning. RESULTS: Thirty-one people participated. Structural support within the hospital was considered essential to maintain staff enthusiasm, momentum and to embed the programme. The multidisciplinary, simulation-based approach to team training was deemed a fundamental infrastructure for learning, with participants especially valuing the realistic in situ simulations and educational support. Participants reported positive changes to the OT culture as a result of NetworkZ and this realisation motivated its implementation. In sites with good structural support, NetworkZ implementation proceeded quickly and participants reported rapid cultural change towards improved teamwork and communication in their OTs. CONCLUSION: Implementation challenges exist and strategies to overcome these are informing future implementation of NetworkZ. Embedding the programme as business as usual across a nation requires significant and sustained support at all levels. However, the potential gains in patient safety and workplace culture from widespread multidisciplinary team training are substantial. Trial registration number ACTRN12617000017325.

Examining reliability of WHOBARS: a tool to measure the quality of administration of WHO surgical safety checklist using generalisability theory with surgical teams from three New Zealand hospitals.

OBJECTIVES: To extend reliability of WHO Behaviourally Anchored Rating Scale (WHOBARS) to measure the quality of WHO Surgical Safety Checklist administration using generalisability theory. In this context, extending reliability refers to establishing generalisability of the tool scores across populations of teams and raters by accounting for the relevant sources of measurement errors. DESIGN: Cross-sectional random effect measurement design assessing surgical teams by the five items on the three Checklist phases, and at three sites by two trained raters simultaneously. SETTING: The data were collected in three tertiary hospitals in Auckland, New Zealand in 2016 and included 60 teams observed in 60 different cases with an equal number of teams (n=20) per site. All elective and acute cases (adults and children) involving surgery under general anaesthesia during normal working hours were eligible. PARTICIPANTS: The study included 243 surgical staff members, 138 (50.12%) women. MAIN OUTCOME MEASURE: Absolute generalisability coefficient that accounts for variance due to items, phases, sites and raters for the WHOBARS measure of the quality of WHO Surgical Safety Checklist administration. RESULTS: The WHOBARS in its present form has demonstrated good generalisability of scores across teams and raters (G absolute=0.83). The largest source of measurement error was the interaction between the surgical team and the rater, accounting for 16.7% (95% CI 16.4 to 16.9) of the total variance in the data. Removing any items from the WHOBARS led to a decrease in the overall reliability of the instrument. CONCLUSIONS: Assessing checklist administration quality is important for promoting improvement in its use, and WHOBARS offers a reliable approach for doing this.

Improving the quality of administration of the Surgical Safety Checklist: a mixed methods study in New Zealand hospitals.

While the WHO Surgical Safety Checklist (the Checklist) can improve patient outcomes, variable administration can erode benefits. We sought to understand and improve how operating room (OR) staff use the Checklist. Our specific aims were to: determine if OR staff can discriminate between good and poor quality of Checklist administration using a validated audit tool (WHOBARS); to determine reliability and accuracy of WHOBARS self-ratings; determine the influence of demographic variables on ratings and explore OR staff attitudes to Checklist administration. DESIGN: Mixed methods study using WHOBARS ratings of surgical cases by OR staff and two independent observers, thematic analysis of staff interviews. PARTICIPANTS: OR staff in three New Zealand hospitals. OUTCOME MEASURES: Reliability of WHOBARS for self-audit; staff attitudes to Checklist administration. RESULTS: Analysis of scores (243 participants, 2 observers, 59 cases) supported tool reliability, with 87% of WHOBARS score variance attributable to differences in Checklist administration between cases. Self-ratings were significantly higher than observer ratings, with some differences between professional groups but error variance from all raters was less than 10%. Key interview themes (33 interviewees) were: Team culture and embedding the Checklist, Information transfer and obstacles, Raising concerns and 'A tick-box exercise'. Interviewees felt the Checklist could promote teamwork and a safety culture, particularly enabling speaking up. Senior staff were of key importance in setting the appropriate tone. CONCLUSIONS: The WHOBARS tool could be useful for self-audit and quality improvement as OR staff can reliably discriminate between good and poor Checklist administration. OR staff self-ratings were lenient compared with external observers suggesting the value of external audit for benchmarking. Small differences between ratings from professional groups underpin the value of including all members of the team in scoring. We identified factors explaining staff perceptions of the Checklist that should inform quality improvement interventions.

Information transfer in multidisciplinary operating room teams: a simulation-based observational study.

BACKGROUND: Communication of clinically relevant information between members of the operating room (OR) team is critical for safe patient care. Formal communication processes, such as briefing, sign in and time out, are designed to promote this. AIMS: We investigated patterns of communication of clinically relevant information between OR staff in simulated surgical scenarios, to identify factors associated with effective information sharing. We focused on the influence of precase briefing, sign in and time out, which we defined as formal team communications. METHOD: Twenty teams of six participated in two scenarios during a day-long course. Participants each received unique, clinically relevant items of information (information probes) prior to simulations and were tested postscenario on recall of the information in the probe. Using videos of the simulations, we coded each time an information probe was mentioned against a structured framework. RESULTS: Of the 145 instances where a probe was mentioned at least once, 75 (51.7%) were mentioned during a formal team communication. However, there were 89 instances of a possible 234 (38%) where a probe was never mentioned. Some team members were more likely to mention the information than others. When probes were mentioned during formal team communications, significantly more team members were attentive (1.4 vs 2.3; p<0.001), the information was significantly more likely to be recalled and the team was five times more likely (p=0.01) to recall the information than if the information was only mentioned outside of a formal communication. CONCLUSIONS: While our study supports the value of formal team communications during precase briefing, sign in and time out in the Surgical Safety Checklist, our findings suggest suboptimal transmission of information between team members and unequal contributions of information by different professional groups.