Which Internal Medicine Clerkship Characteristics Are Associated With Students' Performance on the NBME Medicine Subject Exam? A Multi-Institutional Analysis.

PURPOSE: To identify which internal medicine clerkship characteristics may relate to NBME Medicine Subject Examination scores, given the growing trend toward earlier clerkship start dates. METHOD: The authors used linear mixed effects models (univariable and multivariable) to determine associations between medicine exam performance and clerkship characteristics (longitudinal status, clerkship length, academic start month, ambulatory clinical experience, presence of a study day, involvement in a combined clerkship, preclinical curriculum type, medicine exam timing). Additional covariates included number of NBME clinical subject exams used, number of didactic hours, use of a criterion score for passing the medicine exam, whether medicine exam performance was used to designate clerkship honors, and United States Medical Licensing Examination Step 1 performance. The sample included 24,542 examinees from 62 medical schools spanning 3 academic years (2011-2014). RESULTS: The multivariable analysis found no significant association between clerkship length and medicine exam performance (all pairwise P > .05). However, a small number of examinees beginning their academic term in January scored marginally lower than those starting in July (P < .001). Conversely, examinees scored higher on the medicine exam later in the academic year (all pairwise P < .001). Examinees from schools that used a criterion score for passing the medicine exam also scored higher than those at schools that did not (P < .05). Step 1 performance remained positively associated with medicine exam performance even after controlling for all other variables in the model (P < .001). CONCLUSIONS: In this sample, the authors found no association between many clerkship variables and medicine exam performance. Instead, Step 1 performance was the most powerful predictor of medicine exam performance. These findings suggest that medicine exam performance reflects the overall medical knowledge students accrue during their education rather than any specific internal medicine clerkship characteristics.

Validity of the Health Systems Science Examination: Relationship Between Examinee Performance and Time of Training.

The health systems science (HSS) framework articulates systems-relevant topics that medical trainees must learn to be prepared for physician practice. As new HSS-related curricula are developed, measures demonstrating appropriate levels of reliability and validity are needed. The authors describe a collaborative effort between a consortium of medical schools and the National Board of Medical Examiners to create a multiple-choice HSS examination in the areas of evidence-based medicine/population health, patient safety, quality improvement, and teamwork. Fifteen schools administered the 100-question examination through 2 academic years a total of 1887 times to 1837 first-time takers. Total test score mean was 67% (SD 11%). Total test reliability as measured by coefficient α was .83. This examination differentiated between medical students who completed the examination before, during, and after relevant training/instruction. This new HSS examination can support and inform the efforts of institutions as they integrate HSS-related content into their curricula.

Comparability of the national board of medical examiners comprehensive clinical science examination and a set of five clinical science subject examinations.

PURPOSE: Accreditation standards require medical schools to use comparable assessment methods to ensure students in rotation-based clerkships and longitudinal integrated clerkships (LICs) achieve the same learning objectives. The National Board of Medical Examiners (NBME) Clinical Science Subject Examinations (subject exams) are commonly used, but an integrated examination like the NBME Comprehensive Clinical Science Examination (CCSE) may be better suited for LICs. This study examined the comparability of the CCSE and five commonly required subject exams. METHOD: In 2009-2010, third-year medical students in rotation-based clerkships at the University of British Columbia Faculty of Medicine completed subject exams in medicine, obstetrics-gynecology, pediatrics, psychiatry, and surgery for summative purposes following each rotation and a year-end CCSE for formative purposes. Data for 205 students were analyzed to determine the relationship between scores on the CCSE (and its five discipline subscales) and the five subject exams and the impact of clerkship rotation order. RESULTS: The correlation between the CCSE score and the average score on the five subject exams was high (0.80-0.93). Four subject exam scores were significant predictors of the CCSE score, and scores on the subject exams explained 65%-87% of CCSE score variance. Scores on each subject exam-but not rotation order-were statistically significant in predicting corresponding CCSE discipline subscale scores. CONCLUSIONS: The results provide evidence that these five subject exams and the CCSE measure similar constructs. This suggests that assessment of clerkship-year students' knowledge using the CCSE is comparable to assessment using this set of subject exams.

Relationship between performance on the NBME® Comprehensive Clinical Science Self-Assessment and USMLE® Step 2 Clinical Knowledge for USMGs and IMGs.

BACKGROUND: The Comprehensive Clinical Science Self-Assessment (CCSSA) is a web-administered multiple-choice examination that includes content that is typically covered during the core clinical clerkships in medical school. Because the content of CCSSA items resembles the content of the items on Step 2 Clinical Knowledge (CK), CCSSA is intended to be a tool for students to help assess whether they are prepared for Step 2 CK and to become familiar with its content, format, and pacing. PURPOSES: This study examined the relationship between performance on the National Board of Medical Examiners® CCSSA and performance on the United States Medical Licensing Examination® Step 2 CK for U.S./Canadian (USMGs) and international medical school students/graduates (IMGs). METHODS: The study included 9,789 participants who took CCSSA prior to their first Step 2 CK attempt. Linear and logistic regression analyses investigated the relationship between CCSSA performance and performance on Step 2 CK for both USMGs and IMGs. RESULTS: CCSSA scores explained 58% of the variation in first Step 2 CK scores for USMGs and 60% of the variation for IMGs; the relationship was somewhat different for the two groups as indicated by statistically different intercepts and slopes for the regression lines based on each group. Logistic regression results showed that examinees in both groups with low scores on CCSSA were at a higher risk of failing their first Step 2 CK attempt. CONCLUSIONS: RESULTS suggest that CCSSA can provide students with a valuable practice tool and a realistic self-assessment of their readiness to take Step 2 CK.

Relationship between performance on the NBME Comprehensive Basic Sciences Self-Assessment and USMLE Step 1 for U.S. and Canadian medical school students.

BACKGROUND: This study examined the relationship between performance on the National Board of Medical Examiners Comprehensive Basic Science Self-Assessment (CBSSA) and performance on United States Medical Licensing Examination Step 1. METHOD: The study included 12,224 U.S. and Canadian medical school students who took CBSSA prior to their first Step 1 attempt. Linear and logistic regression analyses investigated the relationship between CBSSA performance and performance on Step 1, and how that relationship was related to interval between exams. RESULTS: CBSSA scores explained 67% of the variation in first Step 1 scores as the sole predictor variable and 69% of the variation when time between CBSSA attempt and first Step 1 attempt was also included as a predictor. Logistic regression results showed that examinees with low scores on CBSSA were at higher risk of failing their first Step 1 attempt. CONCLUSIONS: Results suggest that CBSSA can provide students with a realistic self-assessment of their readiness to take Step 1.