Evaluation and Assessment of the ABATE Framework to Enhance Implicit Bias Training for Virtual Interviews in Medical Schools.

Introduction: The influence of implicit biases in virtual interviews must be addressed to ensure equity within the admissions process. ABATE is a mnemonic framework of five specific categories of implicit bias (affinity-based, backdrop-based, appearance-based, technology and media-based, and enunciation-based biases) that should be anticipated and mitigated for faculty, staff, health professionals, and medical students who conduct virtual interviews at medical schools. Methods: A 60-minute workshop was developed to educate medical school admissions interviewers about the ABATE model and strategies to mitigate implicit bias during virtual interviews. Four workshops were held over 1 year totaling 217 individual attendees. The workshops were evaluated using a single-group, pre-post questionnaire designed with the Kirkpatrick evaluation model. Results: Attendees reported that they found the ABATE workshop useful and relevant to improving their ability to minimize implicit bias during virtual interviews. Significant improvements were found in attendee reactions to the utility of implicit bias training (M pre = 2.6, M post = 3.1, p = .002). Significant changes were also reported in attendees' attitudes about interviewing confidence (M pre = 3.0, M post = 3.2, p = .04), bias awareness (M pre = 3.0, M post = 3.4, p = .002), and identifying and applying bias mitigation solutions (M pre = 2.5, M post = 3.0, p = .003). Knowledge specific to backdrop-based biases also significantly increased (M pre = 3.2, M post = 3.4, p = .04). Discussion: The ABATE workshop demonstrates promise in mitigating implicit bias in virtual medical school interviews.

Evidence-Based Practices for Interviewing Graduate Medical Education Applicants: A Systematic Review.

Background Although the selection interview is a standard admission practice for graduate medical education (GME) programs in the United States, there is a dearth of recent reviews on optimizing the trainee interview process, which has low reliability, high cost, and major risk of bias. Objective To investigate the evidence base for different selection interview practices in GME. Methods We searched 4 literature databases from inception through September 2022. Two investigators independently conducted title/abstract screening, full-text review, data extraction, and quality assessment. Disagreements were mediated by discussion. We used backward reference searching of included articles to identify additional studies. We included studies of different interview methods and excluded literature reviews, non-GME related publications, and studies comparing different applicant populations. We examined study characteristics, applicant and interviewer preferences, and interview format. We evaluated study quality using the Medical Education Research Study Quality Instrument (MERSQI). Results Of 2192 studies, 39 (2%) met our inclusion criteria. The evidence base was rated as moderately low quality using MERSQI criteria. Applicants reported preferences for several one-on-one interviews lasting 15 to 20 minutes, interviews by current trainees, and interviews including social events with only trainees. Applicants had mixed perceptions of virtual versus in-person interviews and reported that virtual interviews saved costs. The multiple mini interview (MMI) required more applicant and interviewer time than individual interviews but demonstrated construct and predictive validity and was preferred by applicants and interviewers. Conclusions Based on moderately low-quality evidence, using the MMI, training interviewers, and providing applicants with basic program information in advance should be considered for GME selection interviews.

Racial and Ethnic Disparities in Situational Judgment Testing Among Applicants to an Anesthesiology Residency Program.

Background The Computer-Based Assessment for Sampling Personal Characteristics (CASPer) is a situational judgment test (SJT) that assesses noncognitive skills like professionalism, communication, and empathy. There are no reports of the effects of race/ethnicity and sex on CASPer scores among residency applicants. Objective We examined the effects of race/ethnicity, sex, and United States vs international medical school attendance on CASPer performance. Methods Our anesthesiology residency program required all applicants for the 2021-2022 Match cycle to complete an online video and text-based SJT (CASPer). We compared these results, reported as z-scores, with self-identified race/ethnicity, sex, United States vs international medical school attendance, and United States Medical Licensing Examination (USMLE) Step 1 scores. Results Of the 1245 applicants who completed CASPer, 783 identified as male. The racial/ethnic distribution was 512 White, 412 Asian, 106 Black, 126 Hispanic, and 89 Other/No Answer. CASPer z-scores did not differ by sex. White candidates scored higher than Black (0.18 vs -0.57, P<.001) and Hispanic (0.18 vs -0.52, P<.001) candidates. Applicants attending US medical schools scored higher than those attending international medical schools (z-scores: 0.15 vs -0.68, P<.001). There was no correlation between CASPer z-scores and USMLE Step 1 scores. Conclusions Our results suggest that CASPer scores favor White applicants over Black and Hispanic ones and applicants attending US medical schools over those attending international medical schools.

Impact of corrupt admission on the mental health of Chinese adolescents.

Through preferential treatment by education officials or through bribery, some adolescents can obtain admission to a junior high school. However, it is unclear whether it affects the mental health of adolescents. This study used Propensity Score Matching to examine the effects of corruption on adolescent mental health. A total of 17,254 junior high school students sample (11-18 years old; 48.7% girls and 53.1% boys) were used from the China Education Panel Survey. 14.1% of adolescents attended a junior high school by corrupt means, corruption had a significantly negative effect on the mental health of these adolescents (ATT = -0.388, p < 0.01), the reasons grounded in the fact that they received more criticisms from teachers and wanted to leave their current school. In general, corruption in the admissions process can have detrimental effects on the mental health of adolescents. This study extends the previous articles on how to improve adolescent mental health and complements the application of cognitive dissonance theory. Findings from this study revealed that anti-corruption in the education sector is necessary, and the institutional design to ensure fair enrolment in basic education will contribute to the mental health of adolescents.

Dear admission committee : Which moves in application essays predict student master grades?

Application essays are a commonly used admission instrument for students entering higher education. The quality of the essay is usually scored, but this score is often subjective and has poor interrater reliability due to the unstructured format of the essays. This results in mixed findings on the validity of application essays as an admission instrument. We propose a more objective method of using application essays, using Latent Dirichlet Allocation (LDA), a text mining method, to distinguish seven moves occurring in application essays written by students who apply to a master degree program. We use the probability that these moves occur in the essay to predict study success in the master. Thereby we answer the following research question: What is the effect of discussing different moves in students' application essays on the student grades in a master program? From the seven different moves (functional unit of text) we distinguished, five of which have a significant effect on student grades. The moves we labeled as 'master specific' and 'interest to learn' have a positive effect on student grades, and the moves we labeled as 'research skills', 'societal impact' and 'city and university' have a negative effect. Our interpretation of this finding is that topics related to intrinsic motivation and specific knowledge, as opposed to generic knowledge, are positively related with study success. We thereby demonstrate that application essays can be a valid predictor of study success. This contributes to justifying their use as admission instruments.

Differences in fear of failure and college adjustment by type of medical school entrance extrinsic motivation using a latent profile analysis.

PURPOSE: The decision to enroll in medical school is largely influenced by extrinsic motivation factors. It is necessary to explore the factors that affect pre-med students' motivation to enter medical school and their college adjustment, and to develop measures to help them adjust. METHODS: A total of 407 pre-med students were surveyed regarding their motivation to enter medical school, fear of failure, and college adjustment. We analyzed the latent profiles of extrinsic motivation factors using latent profile analysis. One-way analysis of variance was conducted to examine the differences in fear of failure and adaptation to university life according to the latent groups. RESULTS: After analyzing the latent profiles of entrance motivation, three latent profiles were selected. They were divided into high, medium, and low extrinsic motivation groups. Three profiles scored the highest on job security, followed by good grades and social status. Sophomores were more likely to be high extrinsic motivators than freshmen were. Fear of failure was high in the group with high extrinsic motivation, and adaptation to college life was highest in the group with low extrinsic motivation. CONCLUSION: Job security was the most important extrinsic motivator for entering medical school, and extrinsic entrance motivation influenced fear of failure and college adjustment. Given the high level of extrinsic motivation among medical students, it is meaningful to analyze the extrinsic motivation profile of entering medical students and how it affects failure motivation and college adjustment.

Privileges, Immunities, and Affirmative Action in Medical Education.

In Students for Fair Admissions v. President and Fellows of Harvard College and Students for Fair Admissions v. University of North Carolina, the Supreme Court ruled that affirmative action in university admissions, in which an applicant of a particular race or ethnicity receives a plus factor, is unconstitutional. This ruling was based on both the Equal Protection Clause of the Fourteenth Amendment and Title VI of the Civil Rights Act of 1964. This article argues that a more natural fit as the basis for constitutional analysis would be a different clause in the Fourteenth Amendment, the Privileges or Immunities Clause. In the article, a legal analysis based on the clause is applied to medical school admissions. Depending on whether a fundamental rights reading or an antidiscrimination (equality) reading of the clause is applied, opposite conclusions are reached on the constitutionality of affirmative action in medical school admissions. This analysis demonstrates why affirmative action in admissions--in this case medical school admissions, which directly affect the composition of the Nation's physician workforce--is a complex and difficult constitutional question.

Taking Action to Promote Diversity and Inclusion: One Program's Efforts Through Pathway Opportunities, Holistic Admissions, and Curricular Changes.

AIMS: Concerted, effective, and sustainable change in healthcare education programs is a critical step towards creating more diverse, inclusive, and equitable professions. This commentary demonstrates how one entry-level physical therapist education program, through a process of reflection, prioritization, and action, is taking steps to increase diversity, equity, and inclusivity within their program. RATIONALE: This article highlights initiatives that are leveraging existing partnerships and creating new ones to reach and mentor students from diverse communities, steps taken towards a more holistic and equitable admissions process, implementation of curricular changes to intentionally discuss the social determinants of health, and engagement of faculty and students to foster personal and professional development on diversity, equity, and inclusion topics. Outcomes to track the effectiveness of the strategies being used by each initiative are shared. CONCLUSION: To create active agents of change, education programs must create a diverse and equitable space for students and guide them to become leaders who can transform society. Steps taken by an entry-level physical therapist education program to implement strategies to promote diversity, equity and inclusion can serve as a road map for other healthcare professional programs.

The association between applicant gender and racial or ethnic identity and success in the admissions process at a Canadian medical school: a prospective cohort study.

Background: Canadian data suggests that Black candidates may be less successful than other groups when applying to medical school. We sought to comprehensively describe the racial and/or ethnic identity, gender identity, sexual orientation, and ability of applicants to a single Canadian medical school. We also examined for an association between success at each application stage and applicant gender and racial identity. Methods: Class of 2024 applicants to a single Canadian medical school were invited to complete a demographics survey. The odds of achieving each application stage (offered an interview, offered a position, and matriculating) were determined for each demographic group. Results: There were 595 participants (32.4% response rate). The demographics of the applicant pool and matriculating class were similar. There was no difference in interview offers or matriculation between BIPOC and white candidates. Cisgender men were overrepresented in interviews compared to cisgender women (OR 0.64; 95%CI 0.43-0.95; p = 0.03) but not in matriculation. BIPOC cisgender women received more interview invitations compared to other groups (OR 2.74, 95%CI 1.20-6.25; p = 0.02). Conclusions: Differences in applicant success for differing demographic groups were most pronounced being offered an interview.

Contexte: Des données canadiennes portent à croire que les candidats noirs peuvent avoir moins de succès que d'autres groupes lorsqu'ils postulent à une faculté de médecine. Nous avons cherché à décrire de manière exhaustive l'identité raciale et/ou ethnique, l'identité de genre, l'orientation sexuelle et les capacités des candidats à une seule faculté de médecine canadienne. Nous avons également cherché à établir un lien entre la réussite à chaque étape de la candidature et le sexe et l'identité raciale du candidat ou de la candidate. Méthodes: Les candidats de la promotion 2024 à une faculté de médecine canadienne ont été invités à répondre à une enquête démographique. Les chances d'atteindre chaque étape de la candidature (entrevue offerte, place offerte et inscription) ont été déterminées pour chaque groupe démographique. Résultats: Il y a eu 595 participants (taux de réponse de 32,4 %). Les caractéristiques démographiques des candidats à l'admission et des étudiants admis étaient similaires. Il n'y avait pas de différence entre les candidats blancs et les candidats autochtones, noirs et de couleur (PANDC) en ce qui concerne les offres d'entrevue ou les admissions. Les hommes cisgenres étaient surreprésentés dans les entrevues par rapport aux femmes cisgenres (OR 0,64; 95%CI 0,43-0,95 ; p=0,03) mais pas dans les admissions. Les femmes cisgenres appartenant au groupe des PANDC ont reçu plus d'offres d'entrevue que les autres groupes (OR 2,74, 95%CI 1,20-6,25 ; p=0,02). Conclusions: Les différences les plus marquées dans la réussite des candidats à l'admission parmi les différents groupes démographiques étaient quant aux offres d'entrevue.

Scoping review of current challenges and circumstances impacting Indigenous applications to Canadian medical schools.

Introduction: Considering the relevant 2015 Truth and Reconciliation Commission recommendations, this paper reviews the current state of Canadian medical schools' Indigenous admissions processes and explores continued barriers faced by Indigenous applicants. Methods: A summary of literature illustrating disadvantages for Indigenous applicants of current admissions tools is presented. A grey literature search of current admissions requirements, interview processes, and other relevant data from each medical school was performed. Tables comparing differences in their approaches are included. A calculation of Indigenous access to medical school seats compared to the broader Canadian population was conducted. Gaps in execution are explored, culminating in a table of recommendations. Results: Despite formal commitments to reduce barriers, Indigenous applicants to medical school in Canada still face barriers that non-Indigenous applicants do not. Most programs use tools for admission known to disadvantage Indigenous applicants. Indigenous applicants do not have equitable access to medical school seats. Facilitated Indigenous stream processes first ensure Indigenous applicants meet all minimum requirements of Canadian students, and then require further work. Discussion: Seven years after the Truth and Reconciliation Commission called on Canadian universities and governments to train more Indigenous health care providers, there has been limited progress to reduce the structural disadvantages Indigenous students face when applying to medical school. Based on best practices observed in Canada and coupled with relevant Indigenous-focused literature, recommendations are made for multiple stakeholders. Conclusions: The study was limited by the data available on numbers of Indigenous applicants and matriculants. Where available, data are not encouraging as to equitable access to medical school for Indigenous populations in Canada. These findings were presented at the International Congress of Academic Medicine 2023 Conference, April 2023, Quebec City, Canada.

Introduction: Compte tenu des recommandations pertinentes de la Commission de vérité et réconciliation de 2015, cet article examine l'état actuel des processus d'admission des Autochtones dans les facultés de médecine canadiennes et explore les obstacles persistants auxquels sont confrontés les candidats autochtones. Méthodes: Un résumé de la littérature illustrant les désavantages des outils d'admission actuels pour les candidats autochtones est présenté. Une recherche de la littérature grise a été effectuée sur les exigences d'admission actuelles, les processus d'entrevue et d'autres données pertinentes de chaque faculté de médecine. Des tableaux comparant les différences entre leurs approches sont inclus. Un calcul de l'accès des Autochtones aux places dans les facultés de médecine par rapport à l'ensemble de la population canadienne a été effectué. Les lacunes dans l'exécution sont explorées, aboutissant à un tableau de recommandations. Résultats: Malgré les engagements officiels visant à réduire les obstacles, les candidats autochtones qui appliquent aux facultés de médecine canadiennes se heurtent encore à des obstacles auxquels les candidats non autochtones ne sont pas confrontés. La plupart des programmes utilisent des outils d'admission connus pour désavantager les candidats autochtones. Les candidats autochtones n'ont pas un accès équitable aux places dans les facultés de médecine. Des processus d'accès facilités pour les autochtones permettent d'abord de s'assurer que les candidats autochtones satisfont à toutes les exigences minimales des étudiants canadiens, puis nécessitent d'autres travaux. Discussion: Sept ans après que la Commission de vérité et réconciliation ait demandé aux universités et aux gouvernements canadiens de former davantage de prestataires de soins en santé autochtone, les progrès réalisés pour réduire les désavantages structurels auxquels les étudiants autochtones sont confrontés lorsqu'ils posent leur candidature à une faculté de médecine sont limités. Sur la base des meilleures pratiques observées au Canada et de la littérature autochtone pertinente, des recommandations sont formulées à l'intention de multiples parties prenantes. Conclusions: L'étude est limitée par les données disponibles sur le nombre de candidats et d'étudiants autochtones. Lorsqu'elles sont disponibles, les données ne sont pas encourageantes en ce qui concerne l'accès équitable aux études de médecine pour les populations autochtones au Canada. Ces conclusions ont été présentées lors de l'édition 2023 du Congrès international de médecine universitaire (CIMU) qui s'est déroulé en avril 2023 dans la ville de Québec, au Canada.

Predictors of Plastic Surgery Applicant Success: An Analysis of the Texas STAR Database.

INTRODUCTION: Integrated plastic surgery residency is one of the most competitive medical specialties. Although previous studies have surveyed integrated plastic surgery residency program directors regarding desired applicant characteristics, there is a paucity of literature assessing detailed application characteristics and reported match outcomes from applicants in recent application cycles. This study examines application characteristics associated with matching into integrated plastic surgery residency from 2017 to 2023. METHODS: The authors accessed the Texas Seeking Transparency in Application to Residency database, which contains survey information from graduating medical students nationwide regarding residency application characteristics and specialties/programs to which applications were submitted. Characteristics of matched versus unmatched applicants between 2017 and 2023 were compared using χ 2 and 2-sided, independent t tests. Univariate logistic regression models were used to assess predictors of a successful match. RESULTS: A total of 381 integrated plastic surgery residency applicants responded to the Texas Seeking Transparency in Application to Residency survey from 2017 to 2023. Mean United States Medical Licensing Exam Step 2 CK scores; the number of away rotations, interview offers, and honored clerkships; and Alpha Omega Alpha membership rate were significantly associated with and predictive of matching. Preference signaling of programs was associated with an increased interview offer rate. CONCLUSIONS: Higher board examination scores, increasing numbers of honored clerkships, away rotations, and Alpha Omega Alpha membership were identified as statistically significant predictors of matching into integrated plastic surgery residency. Prospective applicants should use this information to help guide their efforts in these areas that appear to be associated with a successful transition to residency.

The effects of language proficiency and awareness of time limit in animated vs. text-based situational judgment tests.

BACKGROUND: Situational Judgment Tests (SJTs) are commonly used in medical school admissions. However, it has been consistently found that native speakers tend to score higher on SJTs than non-native speakers, which can be particularly problematic in the admission context due to the potential risk of limited fairness. Besides type of SJT, awareness of time limit may play a role in subgroup differences in the context of cognitive load theory. This study examined the influence of SJT type and awareness of time limit against the background of language proficiency in a quasi high-stakes setting. METHODS: Participants (N = 875), applicants and students in healthcare-related study programs, completed an online study that involved two SJTs: one with a text-based stimulus and response format (HAM-SJT) and another with a video-animated stimulus and media-supported response format (Social Shapes Test, SST). They were randomly assigned to a test condition in which they were either informed about a time limit or not. In a multilevel model analysis, we examined the main effects and interactions of the predictors (test type, language proficiency and awareness of time limit) on test performance (overall, response percentage). RESULTS: There were significant main effects on overall test performance for language proficiency in favor of native speakers and for awareness of time limit in favor of being aware of the time limit. Furthermore, an interaction between language proficiency and test type was found, indicating that subgroup differences are smaller for the animated SJT than for the text-based SJT. No interaction effects on overall test performance were found that included awareness of time limit. CONCLUSION: A SJT with video-animated stimuli and a media-supported response format can reduce subgroup differences in overall test performance between native and non-native speakers in a quasi high-stakes setting. Awareness of time limit is equally important for high and low performance, regardless of language proficiency or test type.

Reform of the first year of medical studies and diversification of student profiles in France: an unmet need?

OBJECTIVES: To determine whether the reform of the first year of medical studies implemented in September 2020 in France met its objective of diversifying the profiles of students admitted to second year at the faculty of medicine at the University of Tours. METHODS: Single-centered, retrospective study, covering students who passed the first year of medical studies between 2018 and 2022. Student profiles originating from three different entry gateways (PACES, PASS and L.AS) to the second year of medical studies were compared. RESULTS: One thousand four hundred and seventy-nine students over five promotions were included (806 in PACES, 329 in PASS, 198 in L.AS). The ratio of students who had obtained a baccalaureate with high or highest honors was significantly higher in PACES (85%) and PASS (96%) compared to L.AS (66%; p < 0.001). These differences were related to increased student intake via a standard pass in L.AS (21% compared to 3.2% in PACES and 0.9% in PASS) (p < 0.001). In terms of geographical origin, the proportion of students residing in regions outside the University City area increased significantly in L.AS (11%) compared to PACES (1.7%) and PASS (3.3%) (p < 0.001). The mean number of parents from the white-collar and knowledge professional category was significantly higher in PACES (0.91) and PASS (1.06) compared to L.AS (0.80; p < 0.001). CONCLUSION: Students with a scientific background and who obtained highest honors in their high school diploma, remain the standard in PACES and PASS. Diversification of student profiles was achieved only within the L.AS gateway, which represented 42% of total second year admissions during the post-reform year. Student profile diversification was therefore a partially achieved objective and follow up studies of future promotions is needed to assess the medium and long-term impact of the reform. Particular attention should be paid to the future of these students who have different profiles between L.AS and PASS to determine whether these changes will have any impact in the quality of healthcare for the French population.

Pharmacy faculty experiences with student academic entitlement: a multinational study from the Arab world.

Academic Entitlement (AE) is the expectation by students to receive high grades or preferential treatment without significant effort. Exploring AE from faculty perspective has not been investigated in Arab colleges of pharmacy. The aim of this study was to explore experiences and perceptions towards student AE among pharmacy faculty in the Arab World. A cross-sectional, self-administered, anonymous, electronic survey was sent to pharmacy faculty across pharmacy colleges in Arab countries. The survey collected demographic data, an AE measure including 17 items reflecting seven AE components, and faculty perceptions and perceived reasons for AE. A total of 345 responses were collected. The AE level was moderate (46.05 ±7.29), and the highest scores among its components were for customer service expectation (62%) and responsibility avoidance (59%). In multiple linear regression, AE showed positive significant association with faculty in clinical pharmacy departments and those having fewer years of experience. Most common complaints heard by faculty from students were requests to turn in assignments late (90%), while the most common communication issues faculty faced with students were unprofessional verbal communication (58%) and unprofessional messages on social media (57%). Poor admission criteria (40%) and existence of multiple private colleges of pharmacy (37%) were the most common perceived reasons for AE by participating faculty. This study reveals moderate AE experienced by pharmacy faculty in the Arab World, as well as common complaints, communication issues, and AE reasons. In collaboration with other stakeholders, faculty play an important role in indicating expectations from students regarding AE, and research is warranted to check if such interventions reduce AE among pharmacy students.

A survey of medical school aspirant perceptions of an unexpected lottery-facilitated admissions adaptation.

INTRODUCTION: Due to the COVID-19 pandemic, the Undergraduate Medical Doctor (MD) Programme at McMaster University (Hamilton, Canada) was unable to run in-person medical school interviews in March 2020, prompting an alternate solution that maximised admission opportunities for Indigenous applicants, prioritised admission for those rated most highly in the interview determination process, and allocated subsequent offers via lottery. METHODS: A short survey was administered to applicants who had been offered an admissions interview and were subsequently impacted by the admissions adaptations. The survey elicited perceptions of the adaptation through Likert scale ratings and free-text responses. Survey data were analysed via a sequential (quantitative to qualitative) mixed-methods design. RESULTS: 196 of 552 potential participants completed the survey. Across quantitative and qualitative analyses, respondents reported that the adaptation had a negative impact on their professional development and personal life. Ratings of negative perception were greater for those who did not receive an offer than for those who accepted or declined an offer. Free text responses emphasised considerable criticism for the lottery portion of the adaptation and displeasure that efforts made in constructing applications were less relevant than anticipated. DISCUSSION: The negative responses to this unexpected change highlight the profound upstream impact admission policies have on the preapplication behaviours of aspiring medical students. The outcomes support a refined understanding of the value candidates place on the interview in appraising their own suitability for a career as a physician.

Investigating the internal structure of multiple mini interviews-A perspective from Pakistan.

BACKGROUND: Healthcare professionals require many personal attributes in addition to cognitive abilities and psychomotor skills for competent practice. Multiple Mini- Interviews are being employed globally to assess personality attributes of candidates for selection in health professions education at all level of entry; these attributes are namely, communication skills, critical thinking, honesty, responsibility, health advocacy, empathy and sanctity of life. Considering the high stakes involved for students, faculty, institutions and the society, rigorous quality assurance mechanisms similar to those used for student assessment must be employed for student selection, throughout the continuum of medical education. It is a difficult undertaking as these psychological constructs are difficult to define and measure. Though considered to yield reliable and valid scores, studies providing multiple evidences of internal structure especially dimensionality of Multiple Mini-Interviews are sparse giving rise to questions if they are measuring a single or multiple constructs and even if they are measuring what they are purported to be measuring. OBJECTIVE: The main objective is to provide statistical support of the multi-dimensional nature of our Multiple Mini Interviews, hypothesized a-priori, through CFA. Another objective is to provide multiple evidences for the internal structure. Our study highlights the link between content and internal structure evidences of the constructs, thus establishing that our Multiple Mini Interviews measure what they were intended to measure. METHOD: After securing permission from the Institutional review board, an a-priori seven factor-model was hypothesized based on the attributes considered most essential for the graduating student of the institution. After operationally defining the attributes through extensive literature search, scenarios were constructed to assess them. A 5-point rating scale was used to rate each item on the station. A total 259 students participated in the multiple mini interviews over a period of three days. A training workshop had been arranged for the participating faculty. RESULTS: The reliability coefficient using Cronbach's alpha were calculated (range from 0.73 to 0.94), Standard Error of Measurement (ranged from 0.80 to1.64), and item to station-total correlation ranged from 0.43-0.50 to 0.75-0.83. Inter-station correlation was also determined. Confirmatory factor analysis endorsed the results of Exploratory factor analysis in the study revealing a seven model fit with multiple indices of Goodness-of-fit statistics such as Root mean square error of approximation (RMSEA) value 0.05, Standardized root mean square residual (SRMR) value with less than 0.08. All these indices showed that model fit is good. The Confirmatory factor analysis confirmed the multi-dimensional nature of our MMIs and also confirmed that our stations measured the attributes that they were supposed to measure. CONCLUSION: This study adds to the validity evidence of Multiple Mini-Interviews, in selection of candidates, with required personality traits for healthcare profession. It provides the evidence for the multi-dimensional structure of Multiple Mini interviews administered with multiple evidences for its internal structure and demonstrates the independence of different constructs being measured.

Urology Residency Applicant Selection: Program Directors' New Criteria.

OBJECTIVE: To evaluate the impact of the recent changes to the urology residency application process on the criteria utilized by residency program directors (PDs) for interview invitations and their perspectives concerning these changes. METHODS: One hundred thirty-seven urology residency PDs were invited to participate in an anonymous survey to explore interview selection criteria and the impact of the increase in preference signals (PS) per applicant. RESULTS: Fifty-eight PDs (42.8%) completed the survey. The highest-ranked criteria were letters of recommendation (LoR) and successful sub-internship (sub-I) at the PD's institution, without statistically significant differences between these 2. Gender, ethnicity, and medical school prestige were the lowest rated criteria, without significant differences between these 3. Compared to before the increase in the number of PS per applicant, 80.7% of PDs reported that not receiving a PS from an applicant this cycle would more negatively impact the chances of offering an interview to that applicant. Moreover, 12.2% stated they would not interview any applicants who did not send a PS. Finally, 62.1% of PDs believed recent changes worsened the process. CONCLUSION: Recent changes impacted PDs applicant evaluation, with the highest ranked criteria being LoRs and sub-I. Paradoxically, the increase in the number of PS per applicant has increased their importance as applicants are much less likely to receive interview offers from programs they have not signaled. Lastly, most PDs believe changes have worsened the evaluation process.