Racial and Ethnic Bias in Letters of Recommendation and Personal Statements for Application to Otolaryngology Residency.

INTRODUCTION: The persistent lack of racial and ethnic diversity within the field of otolaryngology calls for an analysis of potential bias within the residency application system. Letters of recommendation (LORs) and personal statements (PSs) are the most important subjective application constituents. This subjectivity predisposes these components to implicit bias. In applications to various surgical subspecialties, prior linguistic studies assessing bias in reviews of LOR show race-based differences. Thus far, racial and ethnic linguistic differences in LORs for otolaryngology applicants have not been analyzed in the literature. METHODS: LORs and PSs were abstracted from otolaryngology - head and neck surgery applications in the Electronic Residency Application Service for the 2019-20 and 2020-21 application cycles. Linguistic Inquiry and Word Count 2015 was used for quantitative analysis of emotional, cognitive, and structural components of written text. RESULTS: Race-pair analysis of the 2019-2021 application cycles revealed higher mean "teaching" scores for LORs for Asian, black, Hispanic, and white applicants when compared to applicants who self-identified as Other. White applicants had lower scores for the terms "research" and "analytic" when compared to Asian and black applicants, respectively. Analysis of PSs revealed greater scores for an "authentic" writing style for white versus Asian applicants. White applicants were found to have higher scores for "tone" compared to black applicants. CONCLUSION: Minor racial and ethnic language differences exist in both LORs and PSs. A statistically significant difference was observed among LORs, with the "teaching" term used more frequently for Asian, black, Hispanic, and white applicants compared to self-identified Other individuals. For PSs, statistically significant differences were observed among white applicants, who wrote about themselves using more "authentic" language when compared with Asian applicants and who also had higher scores for "tone" compared to black applicants. Although these differences were statistically significant, the practical impact of the variances is likely small.

The next step: Replacing step 1 as a metric for residency application.

Objective: As of January 2022, USMLE Step 1 scores are reported as pass/fail. Historically, Step 1 scores have been a critical component of residency applications, representing one of the few metrics standardized across all applicants independent of the school they attended. In competitive specialties, such as otolaryngology, programs routinely get 100+ applicants for each residency spot and use Step 1 as a screening tool. This study seeks to assess quantifiable metrics in the application that highly competitive residency programs could use for screening in place of Step 1 scores. Methods: Otolaryngology applications to an academic medical center for the 2019-20 and 2020-21 ERAS cycles were reviewed. Board scores and quantitative research data were extracted. The relationships between Step 1 score and the other metrics were examined by computing Pearson's correlation coefficients and building regression models. Similar analyses were done separately for three different score tiers defined by Step 1 cutoffs at 220 points and 250 points. Results: Step 2 score was the only variable that had meaningful correlation with Step 1 score (R = .67, p < 2.2e-16). No other objective metric such as journal articles, posters, or oral presentations correlated with Step 1 scores. Conclusion: Step 1 scores were moderately correlated with Step 2 scores; however, using a Step 2 cutoff instead of a Step 1 cutoff would identify a different cohort of applicants for interview. No other quantifiable application metric had a positive correlation. In future match cycles, highly competitive residency programs will need to adopt new methods to screen candidates.Level of Evidence: Level 3.

Sex bias in letters of recommendation and personal statements for otolaryngology residency.

Objective: Application for otolaryngology residency is highly competitive, with letters of recommendation (LORs) and applicant personal statements (PSs) representing important components of the application process. However, their inherently subjective nature predisposes them to potential implicit bias. Otolaryngology has historically been predominated by male physicians and while implicit sex bias has been demonstrated in LORs for application to residency of multiple specialties, data is limited for otolaryngology. Methods: LORs and PSs for all otolaryngology applicants to an academic medical center during the 2019-20 and 2020-21 cycles were abstracted. Quantitative analysis was performed using Linguistic Inquiry and Word Count 2015 (LIWC2015), a validated software application designed to analyze various emotional, cognitive, and structural components of written text. Results: LORs written for females were found to be written from a perspective of higher expertise and confidence while LORs written for males were associated with a more honest, personal, and disclosing tone. Moreover, LORs written for female applicants were found to reference achievement and "grindstone" terminology more than those written for men. No differences were observed in any word category between PSs written by male and female applicants. Conclusion: Minor linguistic differences exist in multiple domains between LORs written for male and female applicants for otolaryngology residency. These tended to favor female applicants, with their letters demonstrating higher clout, achievement, and grindstone scores. This trend was unexpected in this historically predominantly male specialty. While differences were statistically significant, the overall difference in an entire letter of recommendation is likely subtle. Level of Evidence: 3.