Does "Hoarding" Impact Interview Distribution Among Orthopaedic Surgery Residency Applicants? A Modeling Study Based on a Large Database.

BACKGROUND: "Interview hoarding" is commonly used to refer to the concentration of interview offers among a small number of high-performing residency applicants. Theoretically, if the same applicants interview at every program, fewer rank lists will be submitted than open residency positions, leading to a "match crisis" with unfilled positions after the match. There are no published studies we are aware of that describe the observed distribution of residency interview offers among orthopaedic surgery applicants or the potential impact of "hoarding" on that distribution. QUESTIONS/PURPOSES: We examined the distribution of interview invitations extended to orthopaedic surgery residency applicants in the 2020 to 2021 and 2021 to 2022 application cycles. The change in the shape of the interview invitation distribution was the primary outcome for two central questions: (1) Does the interview offer distribution curve among orthopaedic surgery applicants change meaningfully from baseline with implementation of an interview cap (Model 1)? (2) What is the impact on the distribution of invitations with a reduction in the number of applicants in the field (Model 2)? METHODS: This was a retrospective study of orthopaedic surgery interview invitations extended to applicants via the Thalamus interview management program during the 2020 to 2021 and 2021 to 2022 residency application cycles. The Thalamus database was chosen because it contains data on interview invitations for orthopaedic surgery residency positions and has the largest market share in orthopaedics compared with similar databases. Thalamus data represent 1565 applicants and 53 residency programs (90% and 25% of the national total, respectively) in 2021 to 2022 and 993 applicants and 46 programs (77% and 23%, respectively) in 2020 to 2021. It has been shown to contain a representative sample of orthopaedic residency programs. An interview cap (Model 1) was simulated by removing excess interviews held by applicants above the 75th and 95th percentiles, which were chosen to represent a formal cap and an informal cap, respectively. A reduction in the size of the applicant pool was similarly modeled by randomly removing 5% and 25% of applicants, chosen to simulate informal and formal application requirements, respectively. In both models, the excess interviews were redistributed among the remaining applicants. RESULTS: Applicants received a mean of 1.8 ± 2.2 Thalamus interview invitations in 2020 to 2021 and 1.7 ± 2.4 invitations in 2021 to 2022, with no change to the overall distribution curve. A total of 39% (606 of 1565) of applicants received no Thalamus interview invitations in 2021 to 2022, 75% (1176 of 1565) received two or fewer, and < 1% (14 of 1565) of applicants received 10 or more invitations. Redistributing excess interviews held by the top 5% of applicants resulted in 2% (61 of 2651) of interviews being redistributed (Model 1). Removing 5% of the total applicant pool resulted in a redistribution of 3% (87 of 2651) of the interview invitations (Model 2). CONCLUSION: Orthopaedic surgery interview data demonstrated an expected uneven distribution of interview invitations, with a small proportion of highly competitive applicants receiving a higher number of interview offers as well as a large group of applicants receiving no interview invitations in Thalamus. Concerns that "hoarding" would lead to a crisis resulting in many unmatched residency positions seemed unfounded, given the excess of applicants relative to positions and the minimal change in the distribution of interviews in the cap model. CLINICAL RELEVANCE: Medical students applying to orthopaedic residency should seek individual advising to improve their individual odds of matching, while understanding that interview hoarding does not seem to alter the distribution of interviews. Program directors and medical students' advisors should be cognizant that a small proportion of applicants are broadly interviewed and may benefit from steps taken to ensure applicants have genuine interest in the program.

Over-Application and Interviewing in the 2021 United States Primary Care Virtual Recruitment Season.

Importance Over-application and interviewing are believed to be widespread in residency recruitment. These may have increased during the 2021 virtual recruitment season. The increase does not correspond to an increase in available residency positions and likely results in more interviews with low probabilities of yielding matches. Prior work demonstrates that such marginal interviews are identifiable ­ from key explanatory factors like same-state for interviewee and program ­ in sufficient volume to allow programs to substantially decrease interviews. Objective To evaluate the importance of same-state relationships in primary care and to determine the extent of over-interviewing in the 2021 virtual recruitment season. Design The National Resident Matching Program and Thalamus merged match (outcomes) and interview (explanatory variables) data from primary care specialties (family medicine, internal medicine, pediatrics). Data were analyzed by logistic regression, trained on the 2017-2020 seasons, and projected on the 2021 season for testing. Setting The setting was the 2017-2021 main residency matches. Participants This comprised 4,442 interviewees applying to 167 residency programs in primary care. Intervention This included the transition to virtual recruitment from in-person recruitment in the 2021 residency recruitment season. Measurements A total of 20,415 interviews and 20,791 preferred programs with program and interviewee characteristics and match outcomes were included. Results Same-state geographic relations predicted match probability in primary care residency interviews better than medical school/residency affiliation, with 86.0% of interviewees matching consistently with their preferences for the same state. Same-state was more effective than medical school affiliations with programs in predicting matching. Eliminating interviews with less than a 5% probability of matching (upper 95% prediction limit) removed 31.5% of interviews. Conclusions and relevance The large number of low-match probability interviews demonstrates over-interviewing in primary care. We suggest that programs eliminate interview offers to applications falling below their chosen match probability threshold.

Orthopaedic Surgery Residency Program Adherence to Universal Interview Offer Day Guidelines: A Retrospective Analysis.

INTRODUCTION: The 2020 to 2021 application cycle was marked by structural changes due to the COVID-19 pandemic. Adaptations included the American Orthopaedic Association Council of Residency Program Directors recommendations for a universal interview offer day (UIOD), synchronizing applicant interview offer release. As a novel process within orthopaedics, there are limited data on the execution and effect of a UIOD. Study goals include determining (1) residency program adherence to the Council of Residency Program Directors UIOD guidelines during the 2020 to 2021 cycle, (2) how quickly programs filled available interviews, and (3) any difference in time from release to booking between applicants using edu and com e-mail domains. METHODS: Orthopaedic residency programs and applicants using the Thalamus interview management software platform during the 2020 to 2021 residency application cycle were analyzed, representing 46 residency programs and 993 interviewees (22% and 77% of the national total, respectively). RESULTS: Of the programs included in this study, 19 (41%) were strictly adherent to the UIOD and time, 14 (30%) were weakly adherent and sent out offers outside of the assigned time, and 13 (28%) were nonadherent and sent out offers on dates other than the UIOD. The average time to fill to 80% capacity was 26 ± 14 minutes (range 3 to 77 minutes) for the 33 programs that released on the UIOD. Applicants with edu e-mail domains scheduled their first interview an average of 1.8 minutes after those with com e-mail domains (14.8 versus 13.0 minutes, P < 0.05). CONCLUSION: Despite more than 60% of the residency programs committing to participate, less than half of the programs that initially agreed to participate were strictly adherent to guidelines during the first UIOD in orthopaedic surgery. Although additional research is needed to analyze the unique, fully virtual 2020 to 2021 recruitment season, a phased or waved approach to the UIOD may improve the process for all stakeholders in future cycles. LEVEL OF EVIDENCE: Level V.

Reducing Over-Interviewing in the Anesthesiology Residency Match.

Background The U.S. residency recruitment process is expensive and time-consuming because of application inflation and over-invitation. Objective Using interview and match data, we quantify the predicted effects if anesthesiology residency programs excluded interviews for applicants who are very unlikely to match. Methods We previously published the validity and accuracy of the logistic regression model based on data from interview scheduling software used by 32 U.S. anesthesiology residency programs and 1300 applicants from 2015-18. Data used were program region, applicant address, numbers of interviews of the interviewee, medical school US News and World Report (USNWR) rank, the difference between United States Medical Licensing Exam (USMLE) Step 1 and 2 Clinical Knowledge (CK) scores, and the historical average of USMLE scores of program residents. In the current study completed in 2020, the predicted probabilities and their variances were summed among interviewees for 30 deidentified programs. Results For anesthesiology, the median residency program could reduce their interviews by 16.9% (97.5% confidence interval 8.5%-24.1%) supposing they would not invite applicants if the 99% upper prediction limit for the probability of matching was less than 10.0%. The corresponding median savings would be 0.80 interviews per matched spot (0.34-1.33). In doing so, the median program would sustain a risk of 5.3% (97.5% confidence interval 2.3%-7.9%) of having at least one interviewee removed from their final rank-to-match list. Conclusion Using novel interview data and analyses, we demonstrate that residency programs can substantively reduce interviews with less effect on rank-to-match lists. The data-driven approach to manage marginal interviews allows program leadership to better weigh costs and benefits when composing their annual list of interviewees.

Interview Data Highlight Importance of "Same-State" on Anesthesiology Residency Match.

BACKGROUND: The US residency application, interview, and match processes are costly and time-intensive. We sought to quantify the importance of an applicant being from the same-state as a residency program in terms of how this impacted the number of interviews needed to match. METHODS: We examined data from interview scheduling software used by 32 programs located in 31 US states and 1300 applicants for the US anesthesiology recruitment cycles from 2015 to 2018. Interviewee data (distance from program, region, numbers of interviews, and program at which interview occurred) were analyzed to quantify the effect of the interviewee being from the same state as the residency program on the odds of matching to that program. Other variables of interest (medical school, current address, US Medical Licensing Exam [USMLE] Step 1 and 2 clinical knowledge [CK] scores, Alpha Omega Alpha [AOA] status, medical school ranking) were also examined as controls. Confidence intervals (CI) were calculated for the ratios of odds ratios. RESULTS: An interviewee living in the same state as the interviewing program could have 5.42 fewer total interviews (97.5% CI, 3.02-7.81) while having the same odds of matching. The same state effect had an equivalent value as an approximately 4.14 USMLE points-difference from the program's mean (97.5% CI was 2.34-5.94 USMLE points). Addition of whether the interviewee belonged to an affiliated medical school did not significantly improve the model; same-state remained significant (P < .0001) while affiliated medical school was not (P = .40). CONCLUSIONS: Our analysis of anesthesiology residency recruitment using previously unstudied interview data shows that same-state locality is a viable predictor of residency matching and should be strongly considered when evaluating whether to interview an applicant.