Effect of Grading and Class Rank on Performance in a Surgical Simulation Course.

OBJECTIVE: This study seeks to evaluate the psychological and academic effects of grading a medical school simulation course. This study also seeks to evaluate whether a student's class rank affects these results and whether the academic effects of a simulation course persist long-term. DESIGN: Two separate medical school classes were evaluated. The first class participated in an ungraded surgery simulation course with only formative feedback. The second class participated in a tier-graded course. Qualitative assessment of both courses included daily standardized and self-reported stress surveys, student feedback surveys at the end of the course, and student feedback surveys at the end of the surgery clerkship. Quantitative assessment included pre and postcourse examinations, NBME Surgery Subject Assessment percentile ranks, and COMLEX 2 surgery subscores. Qualitative and quantitative assessments were compared between classes as a whole and between students in the upper and lower half of each class. Results were analyzed using a Student two-tailed t test. SETTING: Rocky Vista University College of Osteopathic Medicine Institute of Simulation, Parker, Colorado. PARTICIPANTS: One hundred forty-eight students from the Rocky Vista University College of Osteopathic Medicine Class of 2017 and one hundred fifty-one students from the Rocky Vista University College of Osteopathic Medicine Class of 2018. RESULTS: Daily standardized stress surveys indicated some increase in stress for students in the graded course, especially for students in the upper half of the graded class. All students adapted to stress significantly by the end of each course, but upper-half students adapted more significantly. Students' self-reported stress levels demonstrated a significant decrease during the ungraded course, but a slight increase during the graded course. Students' self-reported confidence levels rose significantly during the ungraded course, but less significantly during the graded course. Student feedback surveys demonstrated a positive student perception of both courses, but, when stratified, some responses were significantly different between students in the upper and lower half of each class, with upper-half students generally giving a higher rating to the graded course. Clinical knowledge examination scores and improvement in scores within the course were significantly higher for the graded course, with no significant differences between students in the upper and lower half of each class. Students' mean NBME Surgery Subject Assessment percentile rank did not differ significantly between the two classes, but were significantly higher than for previous classes who had not participated in a simulation course. Mean NBME Surgery Subject Assessment percentile rank and COMLEX 2 mean surgery subscore were significantly higher for students in the upper half than for students in the lower half of each class. COMLEX 2 mean surgery subscore did not differ significantly between classes who had participated in a simulation course and previous classes who had not. Students in the upper half of each class in this study performed significantly better than students in the lower half of each class on the mean COMLEX 2 surgery subscore. CONCLUSIONS: This study reaches the following conclusions: (1) objective stress level measurements did not correlate with students' self-reported stress levels; (2) grading a simulation course raises objective stress levels in higher-performing students more than in lower-performing students, but higher-performing students demonstrate the ability to effectively adapt to the increased levels of stress and achieve significant increases in confidence; (3) higher performing students generally demonstrate a greater sense of satisfaction with the simulation course experience when the course is graded; (4) higher-performing and lower-performing students within a class perform in a similar fashion on examinations within the intensely structured environment of a simulation course, whether the course is graded or not; (5) grading a course stimulates all students to perform better on examinations within the course; (6) this higher level of performance and similarity of performance between higher and lower-performing students does not continue when students are not in a structured environment, and students stratify into disparate performance groups in accordance with their class rank; and (7) students exposed to a surgical simulation course perform better on NBME surgery subject examinations than students who have not participated in a simulation course, but this increased level of performance does not extend long-term to national board exams.

Applications of the reflective practice questionnaire in medical education.

BACKGROUND: We sought to determine whether the Reflective Practice Questionnaire (RPQ) is a reliable measure of reflective capacity and related characteristics in medical students. We also planned to learn how the RPQ could be used in medical education. METHODS: The RPQ is a 40 item self-report questionnaire that includes a multi-faceted approach to measuring reflective capacity. It also includes sub-scales on several other theoretically relevant constructs such as desire for improvement, confidence, stress, and job satisfaction. The reliabilities of reflective capacity and other sub-scales were determined by calculating their Cronbach alpha reliability values. In the present study, the RPQ was answered by 98 graduating fourth-year medical students from an American University, and these RPQ scores were compared with general public and mental health practitioner samples from a prior study using ANOVA and Bonferroni adjusted comparisons. RESULTS: Medical students reported a higher reflective capacity than the general public sample, but students were statistically indistinguishable from the mental health practitioner sample. For medical students, reflective capacity was associated with features of confidence, stress, and desire for improvement. Job satisfaction was positively associated with confidence in communication with patients, and negatively associated with stress when interacting with patients. A cluster analysis revealed that around 19% of the medical students exhibited a relatively high level of anxiety interacting with patients, 23% were less engaged, 5% were dissatisfied, and 7% expressed a level of over-confidence in their knowledge and skills that was concerning. CONCLUSIONS: The RPQ is a reliable measure of reflective capacity (Chronbach's alpha value = 0.84) and related characteristics (Cronbach's alpha values from 0.75 to 0.83) in medical students. The RPQ can be used as part of pre-post evaluations of medical education initiatives, to complement student self-reflection activities in the curriculum, and to identify students who might benefit from targeted intervention.

Results of a Surgical Simulation Course Designed to Improve Surgery Clerkship Performance.

BACKGROUND AND OBJECTIVES: During the 2015-2016 academic year, Rocky Vista University College of Osteopathic Medicine (RVUCOM) conducted a required 1-week surgical simulation course as the first week of students' 8-week surgical clerkship. This course was adapted from a pilot RVUCOM surgical simulation course and other surgical simulation courses identified in the literature. The objectives of this course were to teach surgical skills and clinical knowledge, aid students in adjusting to the stress of a surgical clerkship, and improve students' confidence and abilities during the clerkship. METHODS: In all, 148 students participated in the surgical simulation course. Subjective assessment of this course included each student completing a daily standardized stress survey and student feedback surveys at the end of the course and at the end of the surgical clerkship. Objective assessment of this course compared precourse and postcourse tests of surgical knowledge and instrument identification, as well as comparison of NBME Surgery Subject Assessment percentile ranks of this class with 3 prior classes. RESULTS: Daily stress surveys indicated decreased stress on day 5 compared to day 1 (p < 0.01). Students' confidence level increased significantly on day 5 compared to day 1 (p < 0.01). Student feedback surveys demonstrated students' perception that surgical skills and clinical skills had been learned during the course and that the course improved their performance during the surgical clerkship. NBME Surgery Subject Assessment percentile rank improved significantly (p < 0.01) from a mean percentile rank of 36.94 for 3 previous classes to a mean percentile rank of 43.82 for the class who completed the simulation course. Surgical knowledge improved significantly (p < 0.001) from a pretest mean score of 47.02% to a posttest mean score of 55.14%. Surgical instrument identification improved significantly (p < 0.001) from a pretest mean of 13.81% to a posttest mean of 95.28%. CONCLUSION: The RVUCOM surgery simulation course significantly improved student performance academically and helped students acclimate to stress, increase their confidence level, and acquire surgical skills needed during the third-year surgery clerkship.

From trauma in austere environments to combat or medical school: how blended hyper-realism in the real and virtual worlds can better prepare surgeons.

Surgical and simulation development have always been closely associated with military activity. The last ten years have continued that trend, allowing for training in real time, under reality-based conditions, learning technical and clinical skills with the dynamic of true human factors and team training in the actual environment. We present data from diverse activities in three separate scenarios: second-year medical students in clinical scenarios; the U.S. Ski Team physicians training in austere conditions; the U.S. Navy Fleet Surgical Team training for sea and land deployment.