Response Process Validity Evidence for Video Commentary Assessment in Surgery: A Qualitative Study.

OBJECTIVES: Well-developed mental representations of a task are fundamental to proficient performance. 'Video Commentary' (VC) is a novel assessment intended to measure mental representations of surgical tasks that would reflect an important aspect of task proficiency. Whether examinees' actual response processes align with this intent remains unknown. As part of ongoing validation of the assessment, we sought to understand examinees' response processes in VC. DESIGN: Grounded theory qualitative study. In 2019, residents were interviewed about their understanding of and approach to VC. Using grounded theory, we created a theoretical model explaining relationships among factors that influence residents' response processes and performance. Residents' perceived purpose of VC was also explored using Likert-type questions. SETTING: Academic surgical residency program. PARTICIPANTS: Forty-eight surgical residents (PGY-1 to PGY-5). RESULTS: Analysis of narrative comments indicated that residents' perceived purposes of VC generally align with the educator's intent. Resident response processes are influenced by test characteristics, residents' perception and understanding of VC, and residents' personal characteristics. Four strategies seem to guide how residents respond, namely a focus on speed, points, logic, and relevance. Quantitative results indicated residents believe VC scores reflect their ability to speak quickly, ability to think quickly, and knowledge of anatomy (mean = 5.0, 4.5, and 4.4 respectively [1 = strongly disagree, 6 = strongly agree]). PGY-1 and PGY-2 residents tend to focus on naming facts whereas PGY-4 and PGY-5 residents focus on providing comprehensive descriptions. CONCLUSIONS: Residents generally have an accurate understanding of the purpose of VC. However, their use of different approaches could represent a threat to validity. The response strategies of speed, points, logic, and relevance may inform other clinical skills assessments.

Donning and Doffing of Personal Protective Equipment: Perceived Effectiveness of Virtual Simulation Training to Decrease COVID-19 Transmission and Contraction.

Introduction The COVID-19 pandemic exposed gaps in the knowledge of correct donning and doffing of personal protective equipment (PPE) among healthcare workers, causing hospitals to ramp up training. However, social distancing measures forced most institutions and workplaces to shift to remote operations, allowing only essential personnel onsite. Virtual simulation is a growing trend in healthcare simulation education, even more so in this pandemic era. Yet, we have found no evidence of the perceived effectiveness of virtual simulation for training healthcare providers in the proper donning and doffing of PPE. This study aims to determine learner perceptions of the effectiveness of a virtual simulation PPE training module. Methods To address this gap, we used a virtual simulation training module in an online format to determine the perceived efficacy of this method of instruction with the contribution of a variety of healthcare providers and trainees, including physicians, surgeons, pharmacists, dentists, and nurses. Results We found a statistically significant difference in the confidence level of observing best practices of donning and doffing PPE before and after the training sessions. We also found that participants believe virtual simulation can be an effective educational tool for clinical skills. Conclusions This paper presents an international, guideline-based virtual simulation training module that can serve to educate, train, and assess healthcare workers in the proper sequence and technique of donning (putting on), doffing (removing), and disposing of PPE without contaminating themselves or others.

Video Commentary & Machine Learning: Tell Me What You See, I Tell You Who You Are.

BACKGROUND & OBJECTIVE: Teaching and assessment of complex problem solving are a challenge for medical education. Integrating Machine Learning (ML) into medical education has the potential to revolutionize teaching and assessment of these problem-solving processes. In order to demonstrate possible applications of ML to education, we sought to apply ML in the context of a structured Video Commentary (VC) assessment, using ML to predict residents' training level. SETTING: A secondary analysis of multi-institutional, IRB approved study. Participants had completed the VC assessment consisting of 13 short (20-40 seconds) operative video clips. They were scored in real-time using an extensive checklist by an experienced proctor in the assessment. A ML model was developed using TensorFlow and Keras. The individual scores of the 13 video clips from the VC assessment were used as the inputs for the ML model as well as for regression analysis. PARTICIPANTS: A total of 81 surgical residents of all postgraduate years (PGY) 1-5 from 7 institutions constituted the study sample. RESULTS: Scores from individual VC clips were strongly positively correlated with PGY level (p = 0.001). Some video clips were identified to be strongly correlated with a higher total score on the assessment; others had significant influence when used to predict trainees' PGY levels. Using a supervised machine learning model to predict trainees' PGY resulted in a 40% improvement over traditional statistical analysis. CONCLUSIONS: Performing better in a few select video clips was key to obtaining a higher total score but not necessarily foretelling of a higher PGY level. The use of the total score as a sole measure may fail to detect deeper relationships. Our ML model is a promising tool in gauging learners' levels on an assessment as extensive as VC. The model managed to approximate residents' PGY levels with a lower MAE than using traditional statistics. Further investigations with larger datasets are needed.

Video Commentary: Rapid-Fire Assessment of Surgical Cognitive Skills.

OBJECTIVE: We describe the use of "Video Commentary", an assessment using a set of operative videos, to assess trainees' surgical cognitive skills (operative knowledge, spatial awareness, and surgical insight). DESIGN & SETTING: The Video Commentary assessment has been routinely administered to Postgraduate Year (PGY) 1-5 general surgery residents since 2014 as part of a biannual multistation, OSCE-type exam at Mayo Clinic, Rochester, MN. Video Commentary is a rapid-fire, 6-minute assessment, where trainees watch a series of 20 to 30 second operative video clips and comment on them as they play. Each clip varies in procedure, approach, difficulty, and complexity. The combination of video clips differs according to trainees' PGY level except for a few videos that overlap among PGY groups. The name of the procedure is provided at the beginning of each clip with a countdown timer showing in the corner of the screen. A comprehensive checklist is used to score trainees' performance in real-time. DISCUSSION: Assessment of trainees of different levels and staff surgeons show a positive correlation with the experience level (p = 0.0001). The assessment provides a safe alternative to assess trainees in the operating room and encourages them to become more effective communicators. With the use of technology, large video databases can be created to provide just-in-time tailored feedback to the trainees. CONCLUSIONS: Video Commentary can serve as a time and resource-efficient assessment of trainees' surgical cognitive skills and insight. The use and demand of real-time commentary on operative videos may provide a viable approach to help surgeon educators determine trainees' baseline, progression, and readiness to advance.

Inspiring Young Learners Toward a Career in Healthcare Using Simulation.

STATEMENT: To inspire young learners toward a career in healthcare and provide them opportunities to learn lifesaving skills, a hospital-based simulation center collaborated with a local middle school to develop a 88-minute simulation-based educational activity. The activity consisted of eight 10-minute stations on lifesaving and basic medical skills. One hundred fifty students participated. Evaluation surveys showed students favored stations with extensive hands-on activities and valued the opportunity to interact with health professionals. Students also reported more interest in science careers after the visit. This collaborative effort is time-efficient and low-budget simulation-based learning experience that had an immediate impact on middle school students.

Towards Development of an Open Surgery Competency Assessment for Residents (OSCAR) Tool - A Systematic Review of the Literature and Delphi Consensus.

OBJECTIVE: Competency-based education has mandated accurate intra-operative assessment tools. We aimed to define consensus-based open surgical skills perceived by experts as critical for assessment. DESIGN: A mixed-method design was employed: systematic review and e-Delphi methodology. SETTING: The study was performed at McGill University-affiliated large tertiary academic centers in Montreal, Quebec, Canada. PARTICIPANTS: Per PRISMA guidelines, a peer-reviewed search strategy was employed. Studies published in English and those describing technical skill assessment of open abdominal surgery were included; subspecialty-specific skills, conference abstracts, academic memoirs were excluded. Most-cited skills were subjected to e-Delphi methodology to identify those deemed essential by experts, based a 3-point Likert scale. Eighteen McGill University-affiliated general surgeons, representing a variety of subspecialties of General Surgery, were invited to answer the questionnaire. RESULTS: Around 120 of 4285 references were retained for analysis. The 12 most cited skills included suturing, tissue and instrument handling, movement economy, instrument knowledge, knot tying, flow, knowledge of procedure, completion time, dissection technique, knowledge of anatomy and sterile technique; 6 of these achieved high or perfect scores and agreement after 2 rounds of survey: suturing, sterile technique, knot tying, knowledge of anatomy, knowledge of procedure, and tissue handling. Median standard deviation decreased (0.495 to 0.450) from first to second round, indicating improvement in consensus. CONCLUSION: These results will help develop and validate the OSCAR (objective structured clinical assessment rubric) assessment tool for immediate intra-operative feedback of open technical skills for surgical trainees.

Relations between Psychological Needs Satisfaction, Motivation, and Self-Regulated Learning Strategies in Medical Residents: A cross-sectional Study.

This article was migrated. The article was marked as recommended. Residents in the medical field work to fulfil their clinical duties and study to pass exams at the same time. Thus, they need to continuously learn and acquire knowledge in a self-regulated manner that accommodates their busy work schedule. The importance of self-regulated learning (SRL) and its relation to motivation is widely recognized in educational literature, yet it is still not sufficiently explored in medical education literature. The relationship between self-regulated learning (SRL) and motivation has not been sufficiently explored among medical residents. A total of 160 residents from different medical departments at McGill University were asked to complete a questionnaire about their psychological needs satisfaction, motivation to learn, and use of SRL strategies. Our results showed that residents who are more intrinsically motivated reported more utilization of SRL strategies. Results are discussed in terms of their impact on medical education practice as well as their theoretical implications.

Causes and patterns of spine trauma in children and adolescents in Saudi Arabia: implications for injury prevention.

BACKGROUND AND OBJECTIVES: Knowledge regarding traumatic spine injuries (TSIs) is essential for effective prevention strategies, particularly in the developing world, where majority of the population is younger and organized prevention programs are scarce. Therefore, our objective was to describe TSI mechanisms, demographics, patterns, and outcomes in children and adolescents. DESIGN AND SETTINGS: Retrospective chart review in a major trauma center from May 2001 to May 2009 in Riyadh, Saudi Arabia. PATIENTS AND METHODS: Detailed chart reviews were done for all consecutive TSI patients ≤18 years old.Cases were identified through the trauma database registry that included admitted patients. RESULTS: Of the 3796 cases identified, 120 cases (3.2%) sustained 141 TSIs (mean age: 13.5 years; males:83.8%). TSI was most common among children from 16 to 18 years old. Overall, motor vehicle collision (MVC)was the most common injury mechanism (60.8%). However, younger patients (<12 years) sustained more pedestrian injuries (40.6%). Among MVC cases with known seat belt statuses (43.8%), 90.6% were not wearing seat belts. The cervical spine level was the most commonly affected (55.8%) region, especially in children <12 years old (88%). More than 1 affected spinal level was found in 23.3% cases. Spinal cord injuries were found in 19.2% cases. Overall, mortality was 8.3%, and half of these mortalities were secondary to pedestrian injuries. A total of 22.7% of cases were discharged with neurological deficits. CONCLUSION: The high frequency and severity of MVC and pedestrian injuries observed in the present study raise significant concerns regarding the safety of children on the roads. Spine involvement was age specific;younger patients tended to have more cervical injuries, and older patients exhibited more thoracic spine involvement.