Reliability of the Behaviorally Anchored Rating Scale (BARS) for assessing non-technical skills of medical students in simulated scenarios.

PURPOSE: Caring for critically ill patients requires non-technical skills such as teamwork, communication, and task management. The Behaviorally Anchored Rating Scale (BARS) is a brief tool used to assess non-technical skills. The investigators determined inter- and intra-rater reliability of the BARS when used to assess medical students in simulated scenarios. METHOD: The investigators created simulation scenarios for medical students during their pediatric clerkship. Content experts reviewed video recordings of the simulations and assigned BARS scores for four performance components (Situational Awareness, Decision-Making, Communication, and Teamwork) for the leader and for the team as a whole. Krippendorff's alpha with ordinal difference was calculated to measure inter- and intra-rater reliability. RESULTS: Thirty medical students had recordings available for review. Inter- and intra-rater reliability for performance components were, respectively, Individual Situational Awareness (0.488, 0.638), Individual Decision-Making (0.529, 0.691), Individual Communication (0.347, 0.473), Individual Teamwork (0.414, 0.466), Team Situational Awareness (0.450, 0.593), Team Decision Making (0.423, 0.703), Team Communication (0.256, 0.517), and Team Teamwork (0.415, 0.490). CONCLUSIONS: The BARS demonstrated limited reliability when assessing medical students during their pediatric clerkship. Given the unique needs of this population, a modified or new objective scoring system for assessing non-technical skills may be needed for medical students.

Do perceptions of effective distractive driving public service announcements differ between adults and teens?

BACKGROUND: Motor vehicle crashes remain the leading cause of death and disability in teenagers. Graduated licensing laws, enforcement of a legal drinking age of 21 years, zero tolerance, and mandatory restraints have been successful in reducing crashes and fatalities. Media safety campaigns have been less successful. This study was designed to analyze whether perceived effectiveness of public service announcements (PSAs) differed between teens and adults. We hypothesized that adult-derived intent differs from teen perception. METHODS: High-school students attending an annual municipal student safety exposition were asked to rank order six PSAs that were previously judged by an almost all adult committee. Additionally, students were asked to rate the PSAs on an agreement Likert scale assessing interest, understandability, and believability and to indicate potential effect on driving behavior. Students further graded their agreement with 10 top-published driving distracters and were asked to list additional perceived driving distracters. RESULTS: Of the 330 surveys collected, 201 students aged 14 years to 19 years selected at least one choice and 181 ranked-ordered >1 PSA. The PSA #3 selected by the original adult judges was ranked second (33%) by teens with PSA #1 in first place (34%). PSA 1 was not considered as effective by the adult judges. Student age, race, grade, or gender did not produce statistically significant differences. A cohort of 186 teens responded to nationally noted driving distracters with >86% in agreement. Eating and applying cosmetics were additional distracters noted by students completing the survey. CONCLUSION: Preventative media messages should include teen stakeholder review. The components of effective adolescent safety messages continue to require further study.