Evaluating structured assessment of anaesthesiologists' non-technical skills.

BACKGROUND: Non-technical skills (NTS) are essential for safe and efficient anaesthesia. Assessment instruments with appropriate validity evidence can be used to ensure that anaesthesiologists possess the NTS necessary to deliver high-standard patient care. The aims were to collect validity evidence using a contemporary validity framework for the assessment instrument Anaesthesiologists' Non-Technical Skills in Denmark (ANTSdk) regarding response process and internal structure (including reliability), and to investigate the effect of rater training on these properties. METHODS: An explorative study was undertaken at the Danish Institute for Medical Simulation, Copenhagen, Denmark. In a 1-day session, using ANTSdk, a convenience sample of 19 anaesthesiologists rated trainee anaesthesiologists' NTS in nine video-recorded simulation scenarios before and after a 3-h training session. RESULTS: Response process evidence: participants considered ANTSdk useful and feasible for NTS assessment. Internal structure evidence: inter-rater reliability (single measures) largely expressed substantial agreement (ICC ≥ 0.55 and ICC ≥ 0.60 for pre- and post-training ratings respectively). Strong internal consistency of ratings was found (Spearman's correlation coefficient ≥ 0.82). Accuracy of participants' ratings compared with reference ratings (± 1 scale point) was notable (76% and 78% for pre- and post-training ratings, respectively). The results indicate that the elements 'Demonstrating self-awareness', 'Reassessing decisions', 'Assessing competencies', and 'Supporting others' need more attention in future rater training. CONCLUSION: The validity evidence collected on content, response process, and internal structure, suggests that ANTSdk is easy to use on video-recorded simulation scenarios, indicating that ANTSdk is a feasible instrument for NTS assessment during anaesthesia training.

Assessing Nurse Anaesthetists' Non-Technical Skills in the operating room.

BACKGROUND: Incident reporting and fieldwork in operating rooms have shown that some of the errors that arise in anaesthesia relate to inadequate use of non-technical skills. To provide a tool for training and feedback on nurse anaesthetists' non-technical skills, this study aimed to adapt the Anaesthetists' Non-Technical Skills (ANTS) as a behavioural marker system for the formative assessment of nurse anaesthetists' non-technical skills in the operating room. METHODS: A qualitative approach with focus group interviews was used to identify the non-technical skills of nurse anaesthetists in the operating room. The interview data were transcribed verbatim. Directed content analysis was used to code and sort data deductively into the ANTS categories: task management, team working, situation awareness and decision making. The prototype named Nurse Anaesthetists' Non-Technical Skills (N-ANTS) was presented and discussed in a group of subject matter experts to ensure face validity. RESULTS: The N-ANTS system consists of the same four categories as ANTS and 15 underlying elements. Three to five good and poor behavioural markers for each element were identified. The headings and definitions of the categories and elements were adjusted to encompass the behavioural markers in N-ANTS. The differences that emerged mainly reflected statements regarding the establishment of role, competence, and task delegation. CONCLUSION: A behavioural marker system, N-ANTS, for nurse anaesthetists was adapted from a behavioural marker system, ANTS, for anaesthesiologists.