Trainee anaesthetist self-assessment using an entrustment scale in workplace-based assessment.

The role of self-assessment in workplace-based assessment remains contested. However, anaesthesia trainees need to learn to judge the quality of their own work. Entrustment scales have facilitated a shared understanding of performance standards among supervisors by aligning assessment ratings with everyday clinical supervisory decisions. We hypothesised that if the entrustment scale similarly helped trainees in their self-assessment, there would be substantial agreement between supervisor and trainee ratings. We collected separate mini-clinical evaluation exercises forms from 113 anaesthesia trainee-supervisor pairs from three hospitals in Australia and New Zealand. We calculated the agreement between trainee and supervisor ratings using Pearson and intraclass correlation coefficients. We also tested for associations with demographic variables and examined narrative comments for factors influencing rating. We found ratings agreed in 32% of cases, with 66% of trainee ratings within one point of the supervisor rating on a nine-point scale. The correlation between trainee and supervisor ratings was 0.71, and the degree of agreement measured by the intraclass correlation coefficient was 0.67. With higher supervisor ratings, trainee ratings better correlated with supervisor ratings. We found no strong association with demographic variables. Possible explanations of divergent ratings included one party being unaware of a vital aspect of the performance and different interpretations of the prospective nature of the scale. The substantial concordance between trainee and supervisor ratings supports the contention that the entrustment scale helped produce a shared understanding of the desired performance standard. Discussion between trainees and supervisors on the reasoning underlying their respective judgements would provide further opportunities to enhance this shared understanding.

Teamwork matters: team situation awareness to build high-performing healthcare teams, a narrative review.

Healthcare today is the prerogative of teams rather than of individuals. In acute care domains such as anaesthesia, intensive care, and emergency medicine, the work is complex and fast-paced, and the team members are diverse and interdependent. Three decades of research into the behaviours of high-performing teams provides us with clear guidance on team training, demonstrating positive effects on patient safety and staff wellbeing. Here we consider team performance through the lens of situation awareness. Maintaining situation awareness is an absolute requirement for safe and effective patient management. Situation awareness is a dynamic process of perceiving cues in the environment, understanding what they mean, and predicting how the situation may evolve. In the context of acute clinical care, situation awareness can be improved if the whole team actively contributes to monitoring the environment, processing information, and planning next steps. In this narrative review, we explore the concept of situation awareness at the level of the team, the conditions required to maintain team situation awareness, and the relationship between team situation awareness, shared mental models, and team performance. Our ultimate goal is to help clinicians create the conditions required for high-functioning teams, and ultimately improve the safety of clinical care.

Anaesthesia and patient safety in the socio-technical operating theatre: a narrative review spanning a century.

We review the development of technology in anaesthesia over the course of the past century, from the invention of the Boyle apparatus to the modern anaesthetic workstation with artificial intelligence assistance. We define the operating theatre as a socio-technical system, being necessarily comprised of human and technological parts, the ongoing development of which has led to a reduction in mortality during anaesthesia by an order of four magnitudes over a century. The remarkable technological advances in anaesthesia have been accompanied by important paradigm shifts in the approach to patient safety, and we describe the inter-relationship between technology and the human work environment in the development of such paradigm shifts, including the systems approach and organisational resilience. A better understanding of emerging technological advances and their effects on patient safety will allow anaesthesia to continue to be a leader in both patient safety and in the design of equipment and workspaces.

Desire paths for workplace assessment in postgraduate anaesthesia training: analysing informal processes to inform assessment redesign.

BACKGROUND: In postgraduate specialist training, workplace assessments are expected to provide the information required for decisions on trainee progression. Research suggests that meeting this expectation can be difficult in practice, which has led to the development of informal processes, or 'shadow systems' of assessment. Rather than rejecting these informal approaches to workplace assessment, we propose borrowing from sociology the concept of 'desire paths' to legitimise and strengthen these well-trodden approaches. We asked what information about trainees is currently used or desired by those charged with making decisions on trainee progression, and how is it obtained? METHODS: We undertook a qualitative study with thematic analysis of semi-structured interviews of supervisors of training across Australia and New Zealand. RESULTS: From 21 interviews, we identified four interrelated themes, the first being the local context of training sites. The other three themes represent dilemmas in the desire for authentic and representative information about the trainee: 1) how the process of gathering and documenting information can filter, transform, or limit the original message; 2) deciding when possible trainee deviation from performance norms warrants a closer look; and 3) how transparent vs covert information gathering affects the information supervisors will provide, and how control over assessment is distributed between trainee and supervisor. CONCLUSION: From these themes, we propose a set of design principles for future workplace assessment. Understanding the reasons desire paths exist can inform future assessment redesign, and may address the current disjunct between the formal workplace assessment system and what happens in practice.

Latent Safety Threats and Countermeasures in the Operating Theater: A National In Situ Simulation-Based Observational Study.

INTRODUCTION: In situ simulation provides a valuable opportunity to identify latent safety threats (LSTs) in real clinical environments. Using a national simulation program, we explored latent safety threats (LSTs) identified during in situ multidisciplinary simulation-based training in operating theaters in hospitals across New Zealand. METHOD: Surgical simulations lasting between 15 and 45 minutes each were run as part of a team training course delivered in 21 hospitals in New Zealand. After surgical in situ simulations, instructors used a template to record identified LSTs in a postcourse report. We analyzed these reports using the contributory factors framework from the London Protocol to categorize LSTs. RESULTS: Of 103 postcourse reports across 21 hospitals, 77 contained LSTs ranging across all factors in the London Protocol. Common threats included staff knowledge and skills in emergencies, team factors, factors related to task or technology, and work environment threats. Team factors were also commonly reported as protecting against adverse events, in particular, creating a shared mental model. Examples of actions taken to address threats included replacing or repairing faulty equipment, clarifying emergency processes, correcting written information, and staff training for clinical emergencies. CONCLUSIONS: The pervasiveness of LSTs suggests that our results have widespread relevance to surgical departments throughout New Zealand and elsewhere and that collective solutions would be valuable. In situ simulation is an effective mechanism both for identifying threats to patient safety and to prompt initiatives for improvement, supporting the use of in situ simulation in the quality improvement cycle in healthcare.

Hospital accreditation processes in Saudi Arabia: a thematic analysis of hospital staff experiences.

BACKGROUND: Hospital accreditation by an international organisation can play an important role in health quality and safety. However, little is known about how managers and front-line employees experience and perceive the effects of accreditation. Their views could inform quality improvement processes and procedures. OBJECTIVE: To explore perceptions of employees at the managerial level on the Joint Commission International (JCI) accreditation process and its impact on quality of patient care in Saudi Arabian JCI-accredited hospitals. METHODS: We undertook a qualitative study using semi-structured interviews to explore the perspectives of senior staff from three accredited public hospitals in Saudi Arabia. Interviews were transcribed prior to thematic analysis. RESULTS: Twenty managers participated in the interviews. The following inter-related themes emerged concerning the JCI accreditation process and its impact on quality of patient care: drivers for the change; the plan for the change; the process of the change; maintaining changes post-accreditation and patients' issues. Participants were positive in their accounts of: drivers for the change; planning for the change needed to achieve accreditation and managing patients' issues. However, participants reported less favourably on: the process of the change; and maintaining changes post-accreditation. CONCLUSION: The planning stage was perceived as the easiest component of JCI accreditation. Implementing and maintaining changes post-accreditation that demonstrably promote patient safety and quality of care was perceived as more difficult. When planning for accreditation, institutions need to incorporate strategies to ensure that improvements to care continue beyond the accreditation period.

Trust, power and learning in workplace-based assessment: The trainee perspective.

For trainees to participate meaningfully in workplace-based assessment (WBA), they must have trust in their assessor. However, the trainee's dependent position complicates such trust. Understanding how power and trust influence WBAs may help us make them more effective learning opportunities. We conducted semi-structured interviews with 17 postgraduate anaesthesia trainees across Australia and New Zealand. Sensitised by notions of power, we used constructivist grounded theory methodology to examine trainees' experiences with trusting their supervisors in WBAs. In our trainee accounts, we found that supervisors held significant power to mediate access to learning opportunities and influence trainee progress in training. All episodes where supervisors could observe trainees, from simply working together to formal WBAs, were seen to generate assessment information with potential consequences. In response, trainees actively acquiesced to a deferential role, which helped them access desirable expertise and minimise the risk of reputational harm. Trainees granted trust based on how they anticipated a supervisor would use the power inherent in their role. Trainees learned to ration exposure of their authentic practice to supervisors in proportion to their trust in them. Trainees were more trusting and open to learning when supervisors used their power for the trainee's benefit and avoided WBAs with supervisors they perceived as less trustworthy. If assessment for learning is to flourish, then the trainee-supervisor power dynamic must evolve. Enhancing supervisor behaviour through reflection and professional development to better reward trainee trust would invite more trainee participation in assessment for learning. Modifying the assessment system design to nudge the power balance towards the trainee may also help. Modifications could include designated formative and summative assessments or empowering trainees to select which assessments count towards progress decisions. Attending to power and trust in WBA may stimulate progress towards the previously aspirational goal of assessment for learning in the workplace.

How Trainees Come to Trust Supervisors in Workplace-Based Assessment: A Grounded Theory Study.

PURPOSE: In competency-based medical education, workplace-based assessment provides trainees with an opportunity for guidance and supervisors the opportunity to judge the trainees' clinical practice. Learning from assessment is enhanced when trainees reveal their thinking and are open to critique, which requires trust in the assessor. If supervisors knew more about how trainees come to trust them in workplace-based assessment, they could better engender trainee trust and improve trainees' learning experience. METHOD: From August 2018 to September 2019, semistructured interviews were conducted with 17 postgraduate anesthesia trainees across Australia and New Zealand. The transcripts were analyzed using constructivist grounded theory methods sensitized by a sociocultural view of learning informed by Wenger's communities of practice theory. RESULTS: Participants described a continuum from a necessary initial trust to an experience-informed dynamic trust. Trainees assumed initial trust in supervisors based on accreditation, reputation, and a perceived obligation of trustworthiness inherent in the supervisor's role. With experience and time, trainees' trust evolved based on supervisor actions. Deeper levels of trainee trust arose in response to perceived supervisor investment and allowed trainees to devote more emotional and cognitive resources to patient care and learning rather than impression management. Across the continuum from initial trust to experience-informed trust, trainees made rapid trust judgments that were not preceded by conscious deliberation; instead, they represented a learned "feel for the game." CONCLUSIONS: While other factors are involved, our results indicate that the trainee behavior observed in workplace-based assessment is a product of supervisor invitation. Supervisor trustworthiness and investment in trainee development invite trainees to work and present in authentic ways in workplace-based assessment. This authentic engagement, where learners "show themselves" to supervisors and take risks, creates assessment for learning.

Key dimensions of innovations in workplace-based assessment for postgraduate medical education: a scoping review.

BACKGROUND: Specialist training bodies continue to devise innovative methods of gathering information on trainee workplace performance to meet the requirements of competency-based medical education. We reviewed recent innovations in workplace-based assessment (WBA) tools to identify strengths, weaknesses, and trade-offs inherent in their design and use. METHODS: In this scoping review, using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, we systematically searched databases between 2009 and 2019 for WBA tools with novel characteristics not typically seen in traditional WBAs. These included innovations in rating scales, ways of collecting information, technological innovations, ways of triggering WBAs, and approaches to compiling and using information. RESULTS: We identified 30 innovative WBA tools whose characteristics could be categorised into seven dimensions: frequency of assessment, granularity (unit of performance assessed), coverage of the curriculum, rating method, initiation of the WBA, information use, and incentives. These dimensions had multiple interdependencies and trade-offs, often balancing generating assessment data with available resources. Philosophical stance on assessment also influenced WBA choice, for example prioritising trainee-centred learning (i.e. initiation of WBA and transparency of assessment data), perceptions of assessment and feedback as burdensome or beneficial, and holistic vs reductionist views on assessment of performance. CONCLUSIONS: Our synthesis of the literature on innovative WBAs provides a framework for categorising tool characteristics across seven dimensions, systematically teasing apart the considerations in design and use of workplace assessments. It also draws attention to the trade-offs inherent in tool design and selection, and enables a more deliberate consideration of the tool characteristics most appropriate to the local context.

Communication and team function affect patient outcomes in anaesthesia: getting the message across.

A study in this edition of the Journal has added to data showing that failures in communication in the operating room contribute to patient harm. These data support the view that multidisciplinary teamwork and communication training should be part of the continuous professional development of all members of the perioperative team. Achieving change will require efforts to win the hearts and minds of all concerned, but these data also support an expectation that engagement in initiatives and techniques to enhance communication and teamwork should not be optional.

Sustaining multidisciplinary team training in New Zealand hospitals: a qualitative study of a national simulation-based initiative.

AIM: Healthcare is delivered by teams, but the training of healthcare staff is commonly undertaken in professional silos. This study investigated local perspectives on the sustainability of NetworkZ, a New Zealand national simulation-based multi-disciplinary operating room team training programme. METHOD: Local course instructors and managers were invited to participate in semi-structured interviews. Diffusion of innovations theory was utilised to frame deductive thematic analysis of interview data. RESULTS: Twenty-seven people participated. Interviewees described valuing NetworkZ for its multi-disciplinary orientation, in-situ delivery, scenario realism, relevance to teamwork and communication and potential for generalisability to other settings. Interviewees also identified NetworkZ as generating improvements in teamwork and crisis management. NetworkZ was described as complex, due to multidisciplinary participation and the multiple roles and skillsets of instructors needed to run simulations smoothly, making the programme resource intensive to deliver. CONCLUSION: NetworkZ is appreciated as a valuable and unique programme for developing important teamwork and communication skills. Its sustainability is dependent on adequate resourcing and funding.

Systematic review and narrative synthesis of competency-based medical education in anaesthesia.

BACKGROUND: Competency-based medical education (CBME) addresses the accountability of postgraduate training programmes to graduate specialists capable of independent practice. METHODS: We undertook a systematic review and narrative synthesis of the published CBME literature in anaesthesia training programmes to identify current practices and areas requiring further exploration. RESULTS: We grouped the 23 studies that met our inclusion criteria into the following categories: demonstrating outcomes of CBME, developing a consensus on an achievable CBME curriculum, CBME curriculum framework, design and implementation of workplace-based assessment (WBA) tools, trainee self-assessment, perceptions of trainees and supervisors on WBA tools, and technological solutions for assessment and feedback. Included studies reported variable success in reaching consensus in competency outcome frameworks for sequenced progression and limited research on approaches to curriculum delivery, whilst the majority of studies focused on workplace assessment. Studies supported the use of entrustment scales, where assessors make a judgement on the extent to which the trainee can manage a case independently. While evidence supported the reliability of WBA tools, and predicted the numbers needed for high-stakes decisions, areas of concern related to factors influencing the value WBA tools in promoting trainee learning, and variable perceptions of their value in making decisions on progression. CONCLUSIONS: Evidence on outcomes of CBME was limited to acquisition of specific competencies during training. The large number of unanswered questions and the dearth of studies across the core components of CBME suggest that we need a collaborative approach to create the evidence required to implement CBME wisely and cost effectively, to have positive impacts on patients, trainees, and healthcare systems.

Challenging gender stereotypes and advancing inclusive leadership in the operating theatre.

Modern healthcare is delivered by interprofessional teams, and good leadership of these teams is integral to safe patient care. Good leadership in the operating theatre has traditionally been considered as authoritative, confident and directive, and stereotypically associated with men. We argue that this may not be the best model for team-based patient care and promote the concept of inclusive leadership as a valid alternative. Inclusive leadership encourages all team members to contribute to decision-making, thus engendering more team cohesion, information sharing and speaking up, and ultimately enhancing team effectiveness. However, the relational behaviours associated with inclusive leadership are stereotypically associated with women and may not in fact be recognised as leadership. In this article we provide evidence on the advantages of inclusive leadership over authoritative leadership and explore gender stereotypes and obstacles that limit the recognition of inclusive leadership. We propose that operating teams rise above gender stereotypes of leadership. Inclusive leadership can elicit maximum performance of every team member, thus realising the full potential of interprofessional healthcare teams to provide the best care for patients.

Shadow systems in assessment: how supervisors make progress decisions in practice.

Medical educators are tasked with decisions on trainee progression and credentialing for independent clinical practice, which requires robust evidence from workplace-based assessment. It is unclear how the current promotion of workplace-based assessment as a pedagogical approach to promote learning has impacted this use of assessments for decision-making; meeting both these purposes may present unforeseen challenges. In this study we explored how supervisors make decisions on trainee progress in practice. We conducted semi-structured interviews with 19 supervisors of postgraduate anesthesia training across Australia and New Zealand and undertook thematic analysis of the transcripts. Supervisors looked beyond the formal assessment portfolio when making performance decisions. They instead used assessment 'shadow systems' based on their own observation and confidential judgements from trusted colleagues. Supervisors' decision making involved expert judgement of the perceived salient aspects of performance and the standard to be attained while making allowances for the opportunities and constraints of the local learning environment. Supervisors found making progress decisions an emotional burden. When faced with difficult decisions, they found ways to share the responsibility and balance the potential consequences for the trainee with the need to protect their patients. Viewed through the lens of community of practice theory, the development of assessment 'shadow systems' indicates a lack of alignment between local workplace assessment practices and the prescribed programmatic assessment approach to high-stakes progress decisions. Avenues for improvement include cooperative development of formal assessment processes to better meet local needs or incorporating the information in 'shadow systems' into formal assessment processes.