Investigating Group Differences in Examinees' Preparation for and Performance on the New MCAT Exam.

In 2015, the Medical College Admission Test (MCAT) was redesigned to better assess the concepts and reasoning skills students need to be ready for the medical school curriculum. During the new exam's design and rollout, careful attention was paid to the opportunities examinees had to learn the new content and their access to free and low-cost preparation resources. The design committee aimed to mitigate possible unintended effects of the redesign, specifically increasing historical mean group differences in MCAT scores for examinees from lower socioeconomic status (SES) backgrounds and races/ethnicities underrepresented in medicine compared with those from higher SES backgrounds and races/ethnicities not underrepresented in medicine.In this article, the authors describe the characteristics and scores of examinees who took the new MCAT exam in 2017 and compare those trends with historical ones from 2013, presenting evidence that the diversity and performance of examinees has remained stable even with the exam's redesign. They also describe the use of free and low-cost MCAT preparation resources and MCAT preparation courses for examinees from higher and lower SES backgrounds and who are enrolled in undergraduate institutions with more and fewer resources, showing that examinees from lower SES backgrounds and who attend institutions with fewer resources use many free and low-cost test preparation resources at lower rates than their peers. The authors conclude with a description of the next phase of this research: to gather qualitative and quantitative data about the preparation strategies, barriers, and needs of all examinees, but especially those from lower SES and underrepresented racial/ethnic backgrounds.

Association of MCAT scores obtained with standard vs extra administration time with medical school admission, medical student performance, and time to graduation.

IMPORTANCE: Individuals with documented disabilities may receive accommodations on the Medical College Admission Test (MCAT). Whether such accommodations are associated with MCAT scores, medical school admission, and medical school performance is unclear. OBJECTIVE: To determine the comparability of MCAT scores obtained with standard vs extra administration time with respect to likelihood of acceptance to medical school and future medical student performance. DESIGN, SETTING, AND PARTICIPANTS: Retrospective cohort study of applicants to US medical schools for the 2011-2013 entering classes who reported MCAT scores obtained with standard time (n = 133,962) vs extra time (n = 435), and of students who matriculated in US medical schools from 2000-2004 who reported MCAT scores obtained with standard time (n = 76,262) vs extra time (n = 449). EXPOSURES: Standard or extra administration time during MCAT. MAIN OUTCOMES AND MEASURES: Primary outcome measures were acceptance rates at US medical schools and graduation rates within 4 or 5 years after matriculation. Secondary outcome measures were pass rates on the United States Medical Licensing Examination (USMLE) Step examinations and graduation rates within 6 to 8 years after matriculation. RESULTS: Acceptance rates were not significantly different for applicants who had MCAT scores obtained with standard vs extra time (44.5% [59,585/133,962] vs 43.9% [191/435]; difference, 0.6% [95% CI, -4.1 to 5.3]). Students who tested with extra time passed the Step examinations on first attempt at significantly lower rates (Step 1, 82.1% [344/419] vs 94.0% [70,188/74,668]; difference, 11.9% [95% CI, 9.6% to 14.2%]; Step 2 CK, 85.5% [349/408] vs 95.4% [70,476/73,866]; difference, 9.9% [95% CI, 7.8% to 11.9%]; Step 2 CS, 92.0% [288/313] vs 97.0% [60,039/61,882]; difference, 5.0% [95% CI, 3.1% to 6.9%]). They also graduated from medical school at significantly lower rates at different times (4 years, 67.2% [285/424] vs 86.1% [60,547/70,305]; difference, 18.9% [95% CI, 15.6% to 22.2%]; 5 years, 81.6% [346/424] vs 94.4% [66,369/70,305]; difference, 12.8% [95% CI, 10.6% to 15.0%]; 6 years, 85.4% [362/424] vs 95.8% [67,351/70,305]; difference, 10.4% [95% CI, 8.5% to 12.4%]; 7 years, 88.0% [373/424] vs 96.2% [67,639/70,305]; difference, 8.2% [95% CI, 6.4% to 10.1%]; 8 years, 88.4% [375/424] vs 96.5% [67,847/70,305]; difference, 8.1% [95% CI, 6.3% to 9.8%]). These differences remained after controlling for MCAT scores and undergraduate grade point averages. CONCLUSIONS AND RELEVANCE: Among applicants to US medical schools, those with MCAT scores obtained with extra test administration time, compared with standard administration time, had no significant difference in rate of medical school admission but had lower rates of passing the USMLE Step examinations and of medical school graduation within 4 to 8 years after matriculation. These findings raise questions about the types of learning environments and support systems needed by students who test with extra time on the MCAT to enable them to succeed in medical school.

The predictive validity of the MCAT exam in relation to academic performance through medical school: a national cohort study of 2001-2004 matriculants.

PURPOSE: Most research examining the predictive validity of the Medical College Admission Test (MCAT) has focused on the relationship between MCAT scores and scores on the United States Medical Licensing Examination Step exams. This study examined whether MCAT scores predict students' unimpeded progress toward graduation (UP), which the authors defined as not withdrawing or being dismissed for academic reasons, graduating within five years of matriculation, and passing the Step 1, Step 2 Clinical Knowledge, and Step 2 Clinical Skills exams on the first attempt. METHOD: Students who matriculated during 2001-2004 at 119 U.S. medical schools were included in the analyses. Logistic regression analyses were used to estimate the relationships between UP and MCAT total scores alone, undergraduate grade point averages (UGPAs) alone, and UGPAs and MCAT total scores together. All analyses were conducted at the school level and were considered together to evaluate relationships across schools. RESULTS: The majority of matriculants experienced UP. Together, UGPAs and MCAT total scores predicted UP well. MCAT total scores alone were a better predictor than UGPAs alone. Relationships were similar across schools; however, there was more variability across schools in the relationship between UP and UGPAs than between UP and MCAT total scores. CONCLUSIONS: The combination of UGPAs and MCAT total scores performs well as a predictor of UP. Both UGPAs and MCAT total scores are strong predictors of academic performance in medical school through graduation, not just the first two years. Further, these relationships generalize across medical schools.

Do racial and ethnic group differences in performance on the MCAT exam reflect test bias?

The Medical College Admission Test (MCAT) is a standardized examination that assesses fundamental knowledge of scientific concepts, critical reasoning ability, and written communication skills. Medical school admission officers use MCAT scores, along with other measures of academic preparation and personal attributes, to select the applicants they consider the most likely to succeed in medical school. In 2008-2011, the committee charged with conducting a comprehensive review of the MCAT exam examined four issues: (1) whether racial and ethnic groups differ in mean MCAT scores, (2) whether any score differences are due to test bias, (3) how group differences may be explained, and (4) whether the MCAT exam is a barrier to medical school admission for black or Latino applicants. This analysis showed that black and Latino examinees' mean MCAT scores are lower than white examinees', mirroring differences on other standardized admission tests and in the average undergraduate grades of medical school applicants. However, there was no evidence that the MCAT exam is biased against black and Latino applicants as determined by their subsequent performance on selected medical school performance indicators. Among other factors which could contribute to mean differences in MCAT performance, whites, blacks, and Latinos interested in medicine differ with respect to parents' education and income. Admission data indicate that admission committees accept majority and minority applicants at similar rates, which suggests that medical students are selected on the basis of a combination of attributes and competencies rather than on MCAT scores alone.