Current State and Proposal of Best Practices of Standardized Patients for Spanish-Language Objective Structured Clinical Encounters in US Medical Schools.

INTRODUCTION: Medical Spanish programs commonly engage Spanish-speaking standardized patients (SPs) for communication skills assessment, yet no studies address SP recruitment, selection, or training. METHODS: We sent questionnaires to medical Spanish faculty at 20 US medical schools to gauge their practices in recruiting and selecting Spanish-language SPs. We invited faculty to distribute a separate questionnaire to Spanish-language SPs to gather SP language abilities, training, and experience. We analyzed data using descriptive statistics and qualitative content analysis. When available, we reviewed SP video encounters to formally assess participating SPs' linguistic performance using the SP Oral Language Observation Matrix, a rating tool adapted from the Physician Oral Language Observation Matrix to assess oral medical Spanish proficiency. RESULTS: Eighty percent of faculty (16/20) responded. Standardized patient recruitment sources included institutional English-language SPs, Hispanic student groups and professional organizations, communities, and language professionals. Faculty-reported strategies to determine language readiness included interviewing SP candidates in Spanish and asking them to self-rate language skills using a validated scale. Fifteen SPs (54%, 15/28) from 5 schools responded to the SP questionnaire, and one third (5/15) reported that their Spanish was not assessed before being selected as an SP. In addition, one third (5/15) did not receive any initial training before performing a medical Spanish case. Raters assessed 11 different SPs using the SP Oral Language Observation Matrix, and 6 were rated as linguistically "ready" for the SP role. CONCLUSIONS: Current approaches to recruitment, training, and language assessment of SPs vary. We propose strategies to ensure that medical Spanish encounters authentically reflect Spanish-speaking patients.

Validity Evidence for the Physician Oral Language Observation Matrix as a Measure of Medical Spanish Proficiency.

BACKGROUND: The lack of a standardized language assessment process for medical students and physicians communicating in a non-English language threatens healthcare quality and safety. OBJECTIVE: To evaluate the validity of a new rating tool, the Physician Oral Language Observation Matrix (POLOM)™, in assessing medical students' oral communication with Spanish-speaking standardized patients (SPs). DESIGN: POLOM scores were compared to measures of student medical Spanish proficiency to examine convergent validity and to measures of clinical performance to examine concurrent/criterion validity. PARTICIPANTS: Forty-two students at two schools completed SP encounters between January 2021 and April 2022, and POLOM raters scored the videorecorded performances between January and June 2022. MAIN MEASURES: Two approaches to generating POLOM total scores were investigated: rater average and strict consensus. Convergent validity was examined via the POLOM's correlations with (1) the phone-based Clinician Cultural and Linguistic Assessment (CCLA) and (2) the self-rated Interagency Language Roundtable scale for healthcare (ILR-H). Concurrent/criterion validity was examined via correlations with (1) the Comunicación y Habilidades Interpersonales (CAI) scale, (2) a checklist completed by the SP, and (3) a faculty rating of the student's post-encounter clinical note. Pearson's correlations of r ≥ 0.5 and r ≥ 0.2 were considered evidence of convergent validity and concurrent/criterion validity, respectively. KEY RESULTS: Both rater average and strict consensus POLOM scores were strongly correlated with ILR-H (r = 0.72) and CCLA (r ≥ 0.60), providing evidence of convergent validity. The POLOM was substantially correlated with the CAI (r ≥ 0.29), the SP Checklist (r = 0.32), and the faculty scoring of the student's clinical note (r ≥ 0.24), providing concurrent/criterion validity evidence. CONCLUSIONS: The POLOM has demonstrated evidence of convergent and concurrent/criterion validity as a measure of medical students' Spanish proficiency during SP encounters. Additional research is needed to evaluate how the POLOM can be implemented with resident and practicing physicians, applied to other health professions, and adapted to other languages.

Development of a Tool to Assess Medical Oral Language Proficiency.

PURPOSE: To communicate with linguistically diverse patients, medical students and physicians often use their non-English-language skills. However, there is no standard protocol to determine whether those skills are adequate before patient care. This causes many physicians, institutions, educators, and learners to forgo non-English-language proficiency assessment altogether. The purpose of this study is to report on the development, refinement, and interrater reliability of the Physician Oral Language Observation Matrix (POLOM), a rater-based tool assessing 6 language skill categories observed during clinical interactions: comprehension, fluency/fluidity, vocabulary, pronunciation, grammar, and communication. This study focused on the use of the POLOM in Spanish interactions. METHOD: The authors adapted an existing language observation tool for use in clinical settings, creating the preliminary POLOM. Next, they iteratively refined the tool from April to July 2021 using videorecorded medical student-standardized patient encounters from a U.S.-based medical Spanish program. In each refinement iteration, 4 bilingual raters (2 physicians and 2 linguists) independently rated 3 to 6 encounters and convened to discuss ratings with the goals of improving instrument instructions, descriptors, and subsequent rater agreement. Using the final POLOM, raters independently rated 50 videos in rotating interdisciplinary pairs. Generalizability theory was applied to estimate reliability via interrater agreement (dependability) coefficients (range 0-1) for each POLOM category and the total score. RESULTS: POLOM total score dependability equaled 0.927 (single rater) and 0.962 (averaged across 2 raters). The highest mean score was observed for the comprehension category (4.15; range 1-5) while the lowest was for communication (3.01; range 1-5). CONCLUSIONS: Raters achieved a high level of agreement on POLOM assessments of students' medical oral Spanish proficiency. The POLOM is the first assessment tool that provides examinees and instructors with both a holistic and detailed review of clinician non-English oral language skills as contextualized for patient care.

An Overview of Online Resources for Medical Spanish Education for Effective Communication with Spanish-Speaking Patients.

PHENOMENON: Despite the rapid development of virtual medical Spanish educational materials, online resources lack transparency and a peer-review process. The purpose of this interdisciplinary study was to provide a critical inventory of virtual resources for medical Spanish education, thereby providing a panorama of the current state of online medical Spanish. APPROACH: Research team members conducted iterative searches to identify medical Spanish online resources, which were then screened for predetermined inclusion/exclusion criteria. Between June and August 2020, a panel of medical and language experts then adapted and applied a previously published evaluation tool to determine whether resources that met study criteria would help learners achieve medical Spanish core competencies and to what extent each resource incorporated communicative language activities. Consensus meetings were conducted to resolve disagreements and identify gaps in online education. FINDINGS: Out of 465 resources, 127 were further screened, and eight were selected for evaluation. Medical and language specialists independently scored each resource and, following discussions, achieved consensus. Overall, no resource met suitability criteria for all five medical Spanish learner competencies or cultural elements, and only one was suitable for achieving the self-assessment competency. INSIGHTS: Interdisciplinary consensus meetings provide an important avenue for resolving differences of opinion and for integrating both language and medical perspectives into the evaluation process. Existing online resources should be used in conjunction with other materials to ensure that all core competencies for medical Spanish education are addressed. This study revealed important gaps in online resources, including a need to target advanced Spanish learners, apply authentic communicative activities, include assessment opportunities, and integrate culture in the learning program. Based on the current state of online medical Spanish, we offer recommendations for future resources.

Validity Evidence Supporting the Comunicación y Habilidades Interpersonales (CAI) Scale for Medical Spanish Communication and Interpersonal Skills Assessment.

PURPOSE: While many schools teach medical Spanish to improve communication with the growing Spanish-speaking population, a standardized assessment methodology is lacking. This study evaluates validity evidence for a newly developed Spanish Communication and Interpersonal Skills (CIS) scale. METHOD: The authors developed the Comunicación y Habilidades Interpersonales (CAI, pronounced /ki/) scale by culturally and linguistically adapting a 14-item validated English CIS scale. Sources of validity evidence were gathered targeting content, response process, internal structure, relations to other variables, and consequences. The CAI was administered as part of an objective structured clinical examination at 2 medical schools from April to June 2020. RESULTS: The authors mapped CAI items onto USMLE Step 2 CIS behavioral functions and medical Spanish learner competencies to examine content validity. The mean item difficulty and item discrimination estimates are 2.86 (SD = 0.34) and 0.81 (SD = 0.08), respectively, demonstrating good psychometric properties at the item level. Internal-consistency reliability for a single case was 0.96. Learner variance accounted for 12% of total variance, demonstrating ability to differentiate higher and lower performing learners; the person-case interaction accounted for 44% of total variance, indicating strong case specificity. CONCLUSIONS: The CAI has higher case specificity than previously reported in English, suggesting that the nuanced contextual elements of individual cases may matter even more when learners are using limited language skills. Implementing the CAI more widely may promote inclusion of patient-centered communication skills as part of curricular content and activities. This study's validity evidence may inform the development of a future medical Spanish certification examination.