ABSTRACT
PURPOSE: The United States Medical Licensing Examination (USMLE) and Comprehensive Osteopathic Medical Licensing Examination (COMLEX) scores are standard methods used to determine residency candidates' medical knowledge. The authors were interested in using the USMLE and COMLEX part 2 scores in our emergency medicine (EM) residency program to identify at-risk residents who may have difficulty on the in-training exam (ITE) and to determine the cutoff values under which an intern could be given an individualized study plan to ensure medical knowledge competency. METHODS: The authors abstracted the USMLE and COMLEX part 2 scores and the American Board of Emergency Medicine (ABEM) ITE scores for a cohort of first-year EM residents graduating years 2010-2022, converting raw scores to percentiles, and compared part 2 and ABEM ITE scores with Pearson's correlation, a Bland-Altman analysis of bias and 95% limits of agreement, and ROC analysis to determine optimal the cut-off values for predicting ABEM ITE < 50th percentile and the estimated test characteristics. RESULTS: Scores were available for 152 residents, including 93 USMLE and 88 COMLEX exams. The correlations between part 2 scores and ABEM ITE were r = 0.36 (95%CI: 0.17, 0.52; p < 0.001) for USMLE and r = 0.50 (95%CI: 0.33, 0.64; p < 0.001) for COMLEX. Bias and limits of agreement for both part 2 scores were -14 ± 63% for USMLE and 13 ± 50% for COMLEX in predicting the ABEM ITE scores. USMLE < 37th percentile and COMLEX < 53rd percentile identified 42% (N = 39) and 27% (N = 24) of EM residents, respectively, as at risk, with a sensitivity of 61% and 49% and specificity of 71% and 92%, respectively. CONCLUSION: USMLE and COMLEX part 2 scores have a very limited role in identifying those at risk of low ITE performance, suggesting that other factors should be considered to identify interns in need of medical knowledge remediation.
ABSTRACT
OBJECTIVE: The aim of this study was to evaluate the effectiveness of a fishhook removal simulation workshop using investigator-developed diagrams, practice models, and a teaching video. METHODS: This was a descriptive, prospective educational study with Institutional Review Board approval. The primary outcomes were the learner's perception of ease of learning, performance ability, and amount of tissue damage for each technique. A 2¾-minute educational video, instructional visual diagrams, and a simulated model were created to teach 4 techniques: simple retrograde, string pull, advance and cut, and needle cover. Learners performed each technique on a model to assess whether they could remove the hook on the first attempt for each technique. They then rank ordered their technique preferences for ease of learning, performance, perceived tissue damage, and overall choice. RESULTS: Of a total of 34 participants who completed the study, 71% of learners were emergency medicine residents or faculty, 65% were male, 42% were recreational fishers, and 68% had previous fishhook removal experience. On first attempt, more than 88% of participants demonstrated successful fishhook removal using all techniques except needle cover (47%). Simple retrograde was rated easiest to learn (74%) and perform (59%), was perceived to cause the least tissue damage (44%), and was the overall preferred technique. Needle cover was ranked hardest to learn (88%) and perform (82%), was perceived to cause the worst tissue damage (41%), and was the overall least preferred technique. CONCLUSIONS: This study is the first to describe a simulation training program for uncomplicated fishhook removal, and to experimentally evaluate physician learning and preferences for fishhook removal techniques. After a brief educational session, physicians could effectively use all techniques except needle cover. Simple retrograde was the overall preferred technique.
