Reliability of the Behaviorally Anchored Rating Scale (BARS) for assessing non-technical skills of medical students in simulated scenarios.

PURPOSE: Caring for critically ill patients requires non-technical skills such as teamwork, communication, and task management. The Behaviorally Anchored Rating Scale (BARS) is a brief tool used to assess non-technical skills. The investigators determined inter- and intra-rater reliability of the BARS when used to assess medical students in simulated scenarios. METHOD: The investigators created simulation scenarios for medical students during their pediatric clerkship. Content experts reviewed video recordings of the simulations and assigned BARS scores for four performance components (Situational Awareness, Decision-Making, Communication, and Teamwork) for the leader and for the team as a whole. Krippendorff's alpha with ordinal difference was calculated to measure inter- and intra-rater reliability. RESULTS: Thirty medical students had recordings available for review. Inter- and intra-rater reliability for performance components were, respectively, Individual Situational Awareness (0.488, 0.638), Individual Decision-Making (0.529, 0.691), Individual Communication (0.347, 0.473), Individual Teamwork (0.414, 0.466), Team Situational Awareness (0.450, 0.593), Team Decision Making (0.423, 0.703), Team Communication (0.256, 0.517), and Team Teamwork (0.415, 0.490). CONCLUSIONS: The BARS demonstrated limited reliability when assessing medical students during their pediatric clerkship. Given the unique needs of this population, a modified or new objective scoring system for assessing non-technical skills may be needed for medical students.

"Sim One, Do One, Teach One": A Simulation-Based Trauma Orientation for Pediatric Residents in the Emergency Department.

OBJECTIVES: The objective of this study was to determine the effectiveness of a simulation-based curriculum in improving confidence in trauma resuscitation skills and increasing attendance during trauma resuscitations for pediatric residents during their emergency medicine rotation. METHODS: A simulation-based orientation curriculum was implemented for the 2017-2018 academic year. Participants completed a qualitative survey before and after each session to assess their comfort level with skills required in a trauma resuscitation. Responses were compared using the Wilcoxon ranked sum test. Nursing documentation was reviewed for the 2016-2017 and 2017-2018 academic years to determine the frequency of resident attendance at trauma resuscitations. Pediatric resident attendance before and after intervention were compared via χ2 analysis. RESULTS: Survey responses showed a significant increase in confidence in all skills assessed, including primary and secondary survey performance, knowledge of pediatric resident role, knowledge of necessary equipment, ability to determine acuity of patient illness or injury, and ability to differentiate between modes of oxygen delivery (P < 0.01). There was no statistically significant change in the frequency of pediatric resident attendance at trauma bay resuscitations before and after curriculum implementation (21.2% vs 25.7%, P = 0.09). CONCLUSIONS: Through the implementation of a simulation-based trauma orientation for pediatric residents, we were able to improve self-reported confidence in trauma resuscitation skills. This improvement did not result in an increased attendance at trauma resuscitations. Next steps include identifying additional barriers to pediatric resident attendance at trauma bay resuscitations.