Racial Bias on the Emergency Medicine Standardized Letter of Evaluation.

Background: Studies on components of residency applications have shown evidence of racial bias. The Standardized Letter of Evaluation (SLOE) is an assessment measure for emergency medicine (EM) residency applications and, as more specialties opt to use SLOEs in place of narrative letters of recommendation, understanding bias on standardized assessments is essential. Objective: To determine whether there is a difference in rankings on the EM SLOE between underrepresented in medicine (UIM) and non-UIM applicants, White and non-White applicants, and to examine whether differences persist after controlling for other characteristics. Methods: The sample was drawn from medical students who applied to EM residency at the study institution in 2019. We compared rankings between UIM and non-UIM students and between students of each individual race/ethnicity and White students, after controlling for United States Medical Licensing Examination Step scores, Alpha Omega Alpha status, type of school (US MD, US DO, internation medical graduate), Medical Student Performance Evaluation class percentile, affiliated program vs visiting clerkship SLOE, gender and the interaction of race/ethnicity and gender, and adjusted for students submitting multiple SLOEs, using ordinal regression. Results: There were 1555 applicants to the study institution in 2019; 1418 (91.2%) had a SLOE and self-identified race/ethnicity. After controlling for applicant characteristics, non-UIM students were significantly more likely to be ranked higher than UIM students on "Rank Against Peers," (OR 1.46, 95% CI 1.03-2.07) and Grade (OR 1.46, 95% CI 1.05-2.04). Conclusions: Analysis of EM SLOEs submitted to our institution demonstrates racial bias on this standardized assessment tool, which persists after controlling for other performance predictors.

Growth of for-profit involvement in emergency medicine graduate medical education and association between for-profit affiliation and resident salary.

Background: Following projections of an emergency medicine (EM) physician oversupply, the growth of EM residency programs affiliated with for-profit hospitals has been subject to increased attention and speculation. However, essentially no literature exists regarding these programs. Resident pay is one area where these programs could differ from nonprofit-affiliated programs, as investor obligations could make for-profit corporations more likely to reduce resident salaries to increase profit margins. Here, we aim to quantify the growth of EM for-profit affiliated residency programs from 2001-2021 and determine if PGY1 salaries differ between these program types. Methods: Medicare and ACGME accreditation data were used to determine the profit status of hospitals affiliated with EM residency programs. ACGME new accreditation data from 2001-2021 were used to quantify the growth of both for-profit and nonprofit affiliated programs over this period. We searched program websites and called programs to determine 2021-2022 PGY1 salary. Multiple regression was used to model the relationship between profit status and salary using program characteristic covariates to control for confounding variables. Results: The number of EM programs increased from 117 to 276 from 2001-2021 while the number of for-profit affiliated EM residency programs increased from 1 to 29 during this period. Most (85.7%, [24/29]) for-profit affiliated programs were accredited from 2016-2021. Mean for-profit affiliated program salary ($55,658, n  = 24) was $3840 lower than mean nonprofit affiliated program salary ($59,498, n  = 203). For-profit affiliation was a significant predictor of lower 2021-2022 PGY1 salary after controlling for other program characteristics using multiple regression ( ß  = -1919.88, P = 0.010). Conclusions: We found a substantial growth of newly ACGME accredited for-profit affiliated EM residency programs from 2016-2021. We also found for-profit affiliated programs pay lower PGY1 salaries than nonprofit-affiliated programs after controlling for potential confounding variables, which suggests more oversight over the salary determination process could be necessary to prevent resident underpayment.

Quantifying For-Profit Outcomes in GME: A Multispecialty Analysis of Board Certifying Examination Pass Rates in For-Profit Affiliated Residency Programs.

Background: The number of for-profit hospitals has increased in the United States, but their role in and outcomes for graduate medical education (GME) are unclear. Objectives: To describe for-profit involvement in internal medicine (IM), general surgery (GS), and pediatrics GME by quantifying change in for-profit affiliated residency programs and comparing for-profit and nonprofit affiliated program board certifying examination pass rates. Methods: We used Accreditation Council for Graduate Medical Education and Medicare data to quantify for-profit prevalence in IM, GS, and pediatrics GME from 2001 to 2021. We used public pass rate data from the American Board of Surgeons (2017-2019; n=242 programs; 6562 examinees), American Board of Internal Medicine (2018-2020; n=465 programs; 23 922 examinees), and American Board of Pediatrics (2018-2020; n=202 programs; 9819 examinees) to model the relationship between profit status and pass rate within each specialty and across specialties combined using linear regression. Results: The proportion of for-profit affiliated residency programs increased 400.0% in IM, 334.4% in GS, and 23.2% in pediatrics from 2001 to 2021. Bivariate linear regression revealed significantly lower pass rate in for-profit affiliated programs in IM ß =-7.73, P<.001), pediatrics (ß =-14.6, P<.001), and the 3 specialties combined (ß =-5.45, P<.001). Upon multiple regression with addition of program characteristic covariates, this relationship remained significant in pediatrics (ß =-10.04, P=.006). Conclusions: The proportion of for-profit affiliated residency programs has increased in IM, GS, and pediatrics from 2001 to 2021. After controlling for covariates, for-profit affiliated programs were associated with lower board examination pass rates in pediatrics with no association in IM, GS, or the combined measure.

Validity Evidence for the Emergency Medicine Standardized Letter of Evaluation.

BACKGROUND: The standardized letter of evaluation (SLOE) is the application component that program directors value most when evaluating candidates to interview and rank for emergency medicine (EM) residency. Given its successful implementation, other specialties, including otolaryngology, dermatology, and orthopedics, have adopted similar SLOEs of their own, and more specialties are considering creating one. Unfortunately, for such a significant assessment tool, no study to date has comprehensively examined the validity evidence for the EM SLOE. OBJECTIVE: We summarized the published evidence for validity for the EM SLOE using Messick's framework for validity evidence. METHODS: A scoping review of the validity evidence of the EM SLOE was performed in 2020. A scoping review was chosen to identify gaps and future directions, and because the heterogeneity of the literature makes a systematic review difficult. Included articles were assigned to an aspect of Messick's framework and determined to provide evidence for or against validity. RESULTS: There have been 22 articles published relating to validity evidence for the EM SLOE. There is evidence for content validity; however, there is a lack of evidence for internal structure, relation to other variables, and consequences. Additionally, the literature regarding response process demonstrates evidence against validity. CONCLUSIONS: Overall, there is little published evidence in support of validity for the EM SLOE. Stakeholders need to consider changing the ranking system, improving standardization of clerkships, and further studying relation to other variables to improve validity. This will be important across GME as more specialties adopt a standardized letter.

Medical student self-assessment as emergency medicine residency applicants.

BACKGROUND: Emergency medicine (EM) applicants are encouraged to consider their own "competitiveness" when deciding on the number of applications to submit. Program directors rank the Standardized Letter of Evaluation (SLOE) as the most important factor when reviewing an applicant. Accurate insight into how clinical performance is reflected on the SLOE could improve medical students' ability to gauge their own competitiveness. OBJECTIVE: This study aims to determine the accuracy of students' self-assessment by SLOE evaluation measures when compared to the SLOE completed by faculty after their EM clerkship. METHODS: Participants of this multicenter study included fourth-year medical students who had completed their EM clerkship and were applying to EM residency. Students completed a modified SLOE to reflect rankings they believed they would receive on their official SLOE. Additionally, students completed a survey assessing their knowledge of the SLOE, their perception of feedback during the clerkship, and their self-perceived competitiveness as an EM applicant. Correlation between the rankings on the student-completed SLOE and the official SLOE was analyzed using the Kendall correlation. RESULTS: Of the 49 eligible students, 42 (85.7%) completed the study. The correlation between scores on the student-completed and official SLOE were significantly low (r < 0.68) for each item. The majority of students agreed that they were satisfied by the quantity and quality of feedback they received (31/42, 73.8%). Few students agreed that they knew how many applications to submit to ensure a match in EM (7/42, 16.7%). CONCLUSION: This study demonstrates that students did not accurately predict their rankings on the official SLOE at the end of an EM rotation and had little insight into their competitiveness as an applicant. These findings highlight opportunities to mitigate the burden on students and programs caused by the increasing number of applications per applicant. Further research is needed as to whether strategies to increase insight into competitiveness are effective.

Ultrasound for Volume Assessment in Patients with Shock: Effectiveness of an Educational Intervention for Fourth-year Medical Students.

Objective Shock is a common emergency condition with high morbidity and mortality, and judicious fluid resuscitation can significantly affect outcomes. The use of a bedside echocardiogram and evaluation of the inferior vena cava (IVC) via ultrasound (US) for collapsibility can predict volume status. Additionally, the Association of American Medical Colleges (AAMC) Entrustable Professional Activities (EPA) 10 states that residents need to be able to address a patient with a critical illness, including hypotension, on Day 1 of residency. Existing literature revealed no published curriculum to teach medical students these skills. We aimed to determine the effectiveness of an educational intervention to teach fourth-year medical students how to utilize IVC US measurement and echocardiography to assist in volume assessment of patients presenting with shock. Methods Students participated in an hour session on the first day of the emergency medicine (EM) clerkship. Didactic effectiveness was evaluated by comparing results on a pre-test and post-test. The test was administered to residents and attendings during the first week of the academic year to gain evidence for content validity. Students also responded to a survey to evaluate learner satisfaction. Results The average score on the validation test was 68.4% (standard deviation (SD): 21.6%, number (n) = 38) for residents and attendings, and 47.4% (SD: 19.4, n = 13) for interns. Students scored an average of 45.6% (SD: 23.6, n = 83) on the pre-test and 66.4% (SD: 22.1 n = 72) on the post-test, p < 0.01 (degrees of freedom (df) = 153, t = 5.7), Cohen's d = 0.92. The satisfaction survey showed 97.6% of students felt the session was worthwhile, 96.4% would recommend it to other students, and 83.1% felt it taught new information. Conclusion These results show that the educational intervention provides a significant increase in knowledge regarding volume assessment and the use of echocardiogram and IVC US. Additionally, students rated the course highly and felt that it provided information not otherwise taught in medical school. This curriculum addresses the AAMC EPA 10, as it increases students' readiness to address hypotension and could add significant value to the medical school curriculum.