Impact of a Regional Grant Program Through the Lens of Social Cognitive Career Theory: A Mixed-Method Evaluation.

PURPOSE: Evaluations of educational grant programs have focused on research productivity, with few examining impacts on grantees or effective program characteristics. This evaluation examined the regional grant program sponsored by Group on Educational Affairs to examine if and how grantees' careers were affected by funding, and if these experiences aligned with program goals. METHOD: In this concurrent, mixed-methods theory-driven evaluation, quantitative and qualitative data were analyzed independently and then integrated to examine complementarity. Quantitative data examined differences among 4 geographic regions and included proposal and grantee characteristics abstracted from administrative records of 52 funded proposals from 2010-2015 grant cycles. Qualitative data from 23 interviews conducted from 2018 to 2019 explored the impact on grantees, with Social Cognitive Career Theory (SCCT) serving as a framework for deductive thematic analysis. To facilitate integration of findings, quantitative data were layered onto each interview to permit exploration of associations between the 2 data types. RESULTS: Although significant regional differences existed in project length and amount of funding, there were few regional differences in grantee experiences. Despite small funding amounts, grants were perceived as career launching pads. The SCCT framework accounted for grantee experiences, including researcher identity formation and subsequent research, but did not capture collaboration phenomena. Integration of the 2 data types identified experience patterns unique to different groups of grantees (e.g., more or less research experience). The diversity among grantees suggests that clarification of program goals and stronger alignment with criteria for funding may be warranted. CONCLUSIONS: This evaluation illuminates why small educational grant programs may or may not impact interest and productivity in research. Implications exist for funders, including clarifying program goals and providing support for less experienced grantees. Future research should explore grantee subsets (e.g., underrepresented in medicine) to further identify what fosters or inhibits careers of medical education scholars.

A Community-engaged Curriculum at Alpert Medical School: Centering Patient Communities in Medical Education.

BACKGROUND: Incorporating opportunities for community engagement into undergraduate medical education (UME) can help learners to identify and address social determinants of health (SDoH). Multiple challenges exist in operationalizing these experiences. METHODS: Using the Assessing Community Engagement (ACE) model, course directors at the Warren Alpert Medical School of Brown University (AMS) mapped community engagement initiatives to the four-year curriculum. FINDINGS: Service-learning, community engagement projects, and clinical rotations at health centers and free clinics aim to equip learners at AMS with the necessary skills to address SDoH. Careful consideration should be given to the time and resources required to facilitate relationships with community-based agencies, learner reflection, program evaluation, and community-level outcomes. CONCLUSIONS: Community engagement activities should be aligned with learning objectives during the pre-clerkship and clerkship stages of the existing UME curricula. Embarking on a curriculum redesign can create opportunities to expand partnerships with local agencies and deepen student engagement.

Applying Conceptual and Theoretical Frameworks to Health Professions Education Research: An Introductory Workshop.

Introduction: Literature suggests that the quality and rigor of health professions education (HPE) research can be elevated if the research is anchored in existing theories and frameworks. This critical skill is difficult for novice researchers to master. We created a workshop to introduce the practical application of theories and frameworks to HPE research. Methods: We conducted two 60- to 75-minute workshops, one in 2019 at an in-person national conference and another in 2021 during an online national education conference. After a brief role-play introduction, participants applied a relevant theory to a case scenario in small groups, led by facilitators with expertise in HPE research. The workshop concluded with a presentation on applying the lessons learned when preparing a scholarly manuscript. We conducted a postworkshop survey to measure self-reported achievement of objectives. Results: Fifty-five individuals participated in the in-person workshop, and approximately 150 people completed the online workshop. Sixty participants (30%) completed the postworkshop survey across both workshops. As a result of participating in the workshop, 80% of participants (32) indicated they could distinguish between frameworks and theories, and 86% (32) could apply a conceptual or theoretical framework to a research question. Strengths of the workshop included the small-group activity, access to expert facilitators, and the materials provided. Discussion: The workshop has been well received by participants and fills a gap in the existing resources available to HPE researchers and mentors. It can be replicated in multiple settings to model the application of conceptual and theoretical frameworks to HPE research.

Research Methodologies in Health Professions Education Publications: Breadth and Rigor.

PURPOSE: Research methodologies represent assumptions about knowledge and ways of knowing. Diverse research methodologies and methodological standards for rigor are essential in shaping the collective set of knowledge in health professions education (HPE). Given this relationship between methodologies and knowledge, it is important to understand the breadth of research methodologies and their rigor in HPE research publications. However, there are limited studies examining these questions. This study synthesized current trends in methodologies and rigor in HPE papers to inform how evidence is gathered and collectively shapes knowledge in HPE. METHOD: This descriptive quantitative study used stepwise stratified cluster random sampling to analyze 90 papers from 15 HPE journals published in 2018 and 2019. Using a research design codebook, the authors conducted group coding processes for fidelity, response process validity, and rater agreement; an index quantifying methodological rigor was developed and applied for each paper. RESULTS: Over half of research methodologies were quantitative (51%), followed by qualitative (28%), and mixed methods (20%). No quantitative and mixed methods papers reported an epistemological approach. All qualitative papers that reported an epistemological approach (48%) used social constructivism. Most papers included participants from North America (49%) and Europe (20%). The majority of papers did not specify participant sampling strategies (56%) or a rationale for sample size (80%). Among those reported, most studies (81%) collected data within 1 year.The average rigor score of the papers was 56% (SD = 17). Rigor scores varied by journal categories and research methodologies. Rigor scores differed between general HPE journals and discipline-specific journals. Qualitative papers had significantly higher rigor scores than quantitative and mixed methods papers. CONCLUSIONS: This review of methodological breadth and rigor in HPE papers raises awareness in addressing methodological gaps and calls for future research on how the authors shape the nature of knowledge in HPE.

My Life, My Story: Integrating a Life Story Narrative Component Into Medical Student Curricula.

INTRODUCTION: Medical students experience burnout, depersonalization, and decreases in empathy throughout medical training. My Life, My Story (MLMS) is a narrative medicine project that aims to combat these adverse outcomes by teaching students to interview patients about their life story, with the goal of improving patient-centered care competencies, such as empathy. METHODS: The MLMS project was started in the Veterans Affairs (VA) system and has since spread to dozens of VA sites. We adapted and integrated this project into the Warren Alpert Medical School of Brown University curriculum. As part of the required curriculum, first- and third-year medical students participated in a life story interview with a community-based volunteer or a patient in the inpatient hospital setting, transcribed the story, and reviewed the written story with the patient. We assessed student perceptions of the project, changes in empathy, and changes in burnout symptoms. RESULTS: A total of 240 students participated in this project. Students spent an average of 70.7 minutes interviewing patients. A majority of the students believed MLMS was a good use of time (77%), fostered connection with patients (79%), and was effective in recognizing patients' thoughts and feelings (69%). DISCUSSION: To our knowledge, this is one of the first life story interview interventions to be implemented into a required medical school curriculum and outside the VA setting. MLMS may assist students in improving clinical empathy skills and create a structure for medical trainees to better understand their patients.

Novel use of an OSCE to assess medical students' responses to a request for a low value diagnostic imaging test: A mixed methods analysis.

OBJECTIVE: Evaluate medical students' communication skills with a standardized patient (SP) requesting a low value test and describe challenges students identify in addressing the request. METHODS: In this mixed-methods study, third-year students from two medical schools obtained a history, performed a physical examination, and counseled an SP presenting with uncomplicated low back pain who requests an MRI which is not indicated. SP raters evaluated student communication skills using a 14-item checklist. Post-encounter, students reported whether they ordered an MRI and challenges faced. RESULTS: Students who discussed practice guidelines and risks of unnecessary testing with the SP were less likely to order an MRI. Students cited several challenges in responding to the SP request including patient characteristics and circumstances, lack of knowledge about MRI indications and alternatives, and lack of communication skills to address the patient request. CONCLUSIONS: Most students did not order an MRI for uncomplicated LBP, but only a small number of students educated the patient about the evidence to avoid unnecessary imaging or the harm of unnecessary testing. PRACTICE IMPLICATIONS: Knowledge about unnecessary imaging in uncomplicated LBP may be insufficient to adhere to best practices and longitudinal training in challenging conversations is needed.

Development of a pilot procedural skills training course for preclerkship medical students.

OBJECTIVE: Despite procedural skills being recognized as an important component of medical school education, medical students are not confident in their ability to carry out a range of medical procedures. We conducted an institutional needs assessment and used the results to inform the creation of a procedure-based preclinical elective for first- and second-year students. METHODS: We surveyed second-, third-, and fourth-year medical students at Alpert Medical School as well as select program directors to guide selection of a list of procedures to be taught in the elective. We then created an extracurricular 10-week procedural skills course for preclerkship medical students utilizing a hands-on, flipped classroom practice model.  Volunteer preceptors were recruited from the Department of Emergency Medicine to participate with a student-to-faculty ratio not exceeding 5:1. Knowledge and skill acquisition were assessed using a multiple-choice knowledge exam and 4-station practical exam, respectively. Pre- and post-course online surveys were used to assess self-perceived confidence for all procedures. RESULTS: We implemented our procedural skills training course for first- and second-year medical students in the fall of 2015. Forty-four students applied for the first iteration of the course and 15 students were selected to participate. Fourteen students ultimately completed the elective as well as the subsequent course surveys, multiple-choice exam, and practical exam. Students who participated in the elective had increased levels of self-reported confidence at the conclusion of the elective and performed better on a practical exam and multiple-choice exam compared to students who participated in only the standard curriculum. CONCLUSION: A longitudinal preclerkship procedural course early during medical school is a feasible method of teaching procedural skills to a cohort of learners. A number of adjustments could be made to the course in order to scale up and include a larger cohort of students at our own or another institution.

Outcomes of a Longitudinal Quality Improvement and Patient Safety Preclerkship Curriculum.

PURPOSE: To evaluate whether quality improvement (QI) and patient safety (PS) training in preclerkship medical education resulted in students' development and retention of knowledge, application-based and perceived skills, and attitudes throughout clerkships. METHOD: A longitudinal QI/PS curriculum with multimodal curricular components was implemented in the preclerkship curriculum between 2015 and 2017 at the Warren Alpert Medical School of Brown University, Rhode Island. Assessments were administered at baseline (T1), end of year 1 (T2), Clinical Skills Clerkship (T3), and end of clerkships (T4) in the intervention cohort (n = 97). In 2018, T4 data for this cohort were compared with a prior control cohort (n = 97) at T4. RESULTS: Results of knowledge-based multiple-choice questions (MCQs) (t[134] = -1.57, P < .001) and application-based skills (t[132] = -8.91, P < .001) demonstrated significant improvement from T1 to T2 (intervention cohort). Assessments of perceived skills showed significant growth from T1 to T2 (t[137] = -23.38, P < .001). Performance on application-based skills significantly improved from T2 to T3 (t[123] = -4.11, P < .001). Compared with the control cohort, the intervention cohort had significantly higher scores on MCQs (t[187.88] = 3.98, P < .001), application-based skills (t[72.69] = 6.40, P < .001), perceived skills (t[106.99] = 5.24, P < .001), and attitudes (t[152] = 5.86, P < .001). CONCLUSIONS: Incorporation of preclerkship QI/PS training resulted in improvements in knowledge, application-based and perceived skills, and attitudes that were retained throughout clerkships.

Substance use screening, brief intervention, and referral to treatment training for emergency medicine trainees.

INTRODUCTION: Emergency medicine (EM) providers are in an opportunistic position to identify and intervene with patients at risk for alcohol misuse and related problems. However, alcohol screening, brief intervention, and referral to treatment (SBIRT) services are underutilized within the emergency department. Providing SBIRT training to trainees may help to increase utilization of these valuable services in the future. An SBIRT training program for EM faculty and trainees was developed and delivered to increase trainees' skills and practice of SBIRT services. METHODS: The SBIRT training program included unique tracks for medical students, physician assistant (PA) students, EM residents and faculty preceptors. Faculty and trainees completed performance measures at the end of each training session, 30 days post-training and 12 months later. RESULTS: SBIRT training was provided to 238 trainees and 65 faculty members. At all follow-up time points, satisfaction of training and usefulness varied by trainee type with PA students rating constructs higher than both medical students and EM residents. At the 12-month follow-up survey, there was no significant difference in ratings of sharing the information (χ2 (2)=0.38, P=0.33) between these trainees, with the majority of all trainees (96% of PA students, 83% of residents and 68% of medical students) responding that they had applied what they learned in the training to their work. CONCLUSION: An SBIRT training curriculum for EM trainees was delivered successfully and utilized 12 months after implementation.

A Content Analysis of the ACGME Specialty Milestones to Identify Performance Indicators Pertaining to the Development of Residents as Educators.

Construct: For curriculum development purposes, this study examined how the development of residents as educators is reflected in the Accreditation Council for Graduate Medical Education (ACGME) Milestones. Background: Residents teach patients, families, medical students, physicians, and other health professionals during and beyond their training. Despite this expectation, it is unclear how the development of residents as educators is reflected in the specialty-specific Milestones. Approach: We performed a textual content analysis of 25 specialty Milestone documents available as downloads from the ACGME website in December 2015. Syntactical units of interest included developmental progressions that describe the development of educators over the course of residency training and 16 key terms identified during the analysis. We then categorized the terms by associated Milestone level, ACGME core competency, and targeted learner(s). Results: We identified 10 developmental progressions and 546 instances of the 16 key terms that describe the development of physician educators. The frequency of terms among specialties was quite variable (5-46 terms per specialty, Mdn = 21). The majority of education-related terms appeared at advanced Milestone levels; there were 139 (26%) such instances in Level 4 and 296 (54%) in Level 5. Education-related terms were identified in all six ACGME core competencies, with greatest frequency in Patient Care (157, 29%). Other residents were the learners most frequently targeted by education-related Milestones (211, 40%). Conclusions: The current ACGME Milestones largely imply that resident teaching is a high-level or aspirational goal, achieved without a clear or consistently assessed developmental progression. These findings run counter to the theoretical basis that underlies the development of the Milestones. Wide variation among specialties indicates lack of consensus around the ideal skill set of the resident educator and limits the utility of these documents for curriculum development in this domain.

Reliability of the Clinical SBIRT Proficiency Checklist for Medical Students.

INTRODUCTION: The Substance Abuse and Mental Health Services Agency (SAMHSA) recognizes Screening, Brief Intervention, and Referral to Treatment (SBIRT) as a comprehensive approach to identify individuals with or at risk of developing substance use disorders. Few studies have explored tools for assessing medical student SBIRT competencies. METHODS: First-year medical students trained in SBIRT and Motivational Interviewing completed an Objective Structured Clinical Encounter (OSCE) with a standardized patient who presented with substance use. Six trained members of our research team reviewed 118 OSCE videos utilizing the Clinical SBIRT Proficiency Checklist (CSPC); additionally 30% (n = 37) were randomly selected for pair-review to examine interrater reliability. RESULTS: Interrater reliability was Cohen's kappa of 0.89 for the presence of SBIRT skills and 0.39 agreement for the absence of skills. Across the videos, the most commonly observed skill was screening for alcohol use (75.4%, 95%CI 66.5, 84.3), while organizing referral for treatment was infrequently observed (36.4%, 95% CI 22.0, 50.8). CONCLUSIONS: The CSPC is a reliable tool for assessing medical student SBIRT skills on an OSCE. These findings provide insights on medical student SBIRT knowledge and provides a practical tool for providing early clinically relevant feedback on these skills.

Addressing racial bias in wards.

Health disparities fall along racial lines, in part, due to structural inequalities limiting health care access. The concept of race is often taught in health professions education with a clear biologic underpinning despite the significant debate in the literature as to whether race is a social or biologic construct. The teaching of race as a biologic construct, however, allows for the simplification of race as a risk factor for disease. As health care providers, it is part of our professional responsibility and duty to patients to think and talk about race in a way that is cognizant of broader historical, political, and cultural literature and context. Openly discussing the topic of race in medicine is not only uncomfortable but also difficult given its controversies and complicated context. In response, we provide several evidence-based steps to guide discussions around race in clinical settings, while also hopefully limiting the use of bias and racism in the practice of medicine.

Intimate Partner Violence Screening and Counseling: An Introductory Session for Health Care Professionals.

Introduction: Intimate partner violence is a serious public health concern in the United States. Despite recommendations that physicians should routinely screen their patients, research has shown that lack of specific training has resulted in many health care professionals feeling unable to adequately perform this difficult but vital task. Though many educational resources exist to teach intimate partner violence screening, they often lack specific guidance on how to navigate this difficult conversation. In addition, they often lack formal teaching on how to counsel and refer patients who are victims of intimate partner violence. Methods: This unique module, intended for a small-group setting of four to eight students, contains an intimate partner violence checklist with sample language that covers both screening and counseling using a motivational interviewing framework. Additional materials include a checklist companion for tips on how to navigate the conversation, two cases for role-play, a facilitator guide, and an objective structured clinical encounter case and assessment rubric. Results: This module was given to 260 second-year medical students at the Warren Alpert Medical School between 2015 and 2017 and was rated highly by almost 90% of students. Discussion: After completing this module, learners will be able to appropriately screen for intimate partner violence as well as counsel and refer patients who have screened positive. By implementing this module, educators can increase the number of health care professionals able to broach this difficult conversation with patients who may be in need of help and may otherwise go unaided.

Creating a Cadre of Fellowship-Trained Medical Educators, Part II: A Formal Needs Assessment to Structure Postgraduate Fellowships in Medical Education Scholarship and Leadership.

PURPOSE: Education leaders at the 2012 Academic Emergency Medicine Consensus Conference on education research proposed that dedicated postgraduate education scholarship fellowships (ESFs) might provide an effective model for developing future faculty as scholars. A formal needs assessment was performed to understand the training gap and inform the development of ESFs. METHOD: A mixed-methods needs assessment was conducted of four emergency medicine national stakeholder groups in 2013: department chairs; faculty education/research leaders; existing education fellowship directors; and current education fellows/graduates. Descriptive statistics were reported for quantitative data. Qualitative data from semistructured interviews and free-text responses were analyzed using a thematic approach. RESULTS: Participants were 11/15 (73%) education fellowship directors, 13/20 (65%) fellows/graduates, 106/239 (44%) faculty education/research leaders, and a convenience sample of 26 department chairs. Department chairs expected new education faculty to design didactics (85%) and teach clinically (96%). Faculty education/research leaders thought new faculty were inadequately prepared for job tasks (83.7%) and that ESFs would improve the overall quality of education research (91.1%). Fellowship directors noted that ESFs provide skills, mentorship, and protected time for graduates to become productive academicians. Current fellows/graduates reported pursing an ESF to develop skills in teaching and research methodology. CONCLUSIONS: Stakeholder groups uniformly perceived a need for training in education theory, clinical teaching, and education research. These findings support dedicated, deliberate training in these areas. Establishment of a structure for scholarly pursuits prior to assuming a full-time position will effectively prepare new faculty. These findings may inform the development, implementation, and curricula of ESFs.

Creating a Cadre of Fellowship-Trained Medical Educators: A Qualitative Study of Faculty Development Program Leaders' Perspectives and Advice.

PURPOSE: Well-trained educators fill essential roles across the medical education continuum. Some medical schools offer programs for existing faculty to enhance teaching and scholarship. No standard postgraduate training model exists for residency graduates to attain competency as faculty members before their first academic appointment. The objective of this study is to inform the development of postgraduate medical education fellowships by exploring perceptions of educational leaders who direct well-established faculty development programs. METHOD: The authors undertook a qualitative study, using purposeful sampling to recruit participants and a constant comparative approach to identify themes. They conducted semistructured telephone interviews with directors of faculty development fellowships using an interpretivist/constructivist paradigm (November 2013). Questions addressed curricular and fiscal structure, perceived benefits and challenges, and advice for starting a postgraduate fellowship. RESULTS: Directors reported institutional and participant benefits, notably the creation of a community of educators and pool of potential leaders. Curricular offerings focused on learning theory, teaching, assessment, leadership, and scholarship. Funding and protected time were challenges. Advice for new program directors included evaluating best practices, defining locally relevant goals; garnering sufficient, stable financial support; and rallying leaders' endorsement. CONCLUSIONS: Medical education fellowships cultivate leaders and communities of trained educators but require participants to balance faculty responsibilities with professional development. Advice of current directors can inform the development of postgraduate programs modeled after accredited clinical specialty fellowships. Programs with the support of strategic partners, financial stability, and well-defined goals may allow new faculty to begin their careers with existing competency in medical education skills.

Twelve tips for developing, implementing, and sustaining medical education fellowship programs: Building on new trends and solid foundations.

Medical education fellowship programs (MEFPs) are a form of faculty development contributing to an organization's educational mission and participants' career development. Building an MEFP requires a systematic design, implementation, and evaluation approach which aligns institutional and individual faculty goals. Implementing an MEFP requires a team of committed individuals who provide expertise, guidance, and mentoring. Qualified MEFP directors should utilize instructional methods that promote individual and institutional short and long term growth. Directors must balance the use of traditional design, implementation, and evaluation methodologies with advancing trends that may support or threaten the acceptability and sustainability of the program. Drawing on the expertise of 28 MEFP directors, we provide twelve tips as a guide to those implementing, sustaining, and/or growing a successful MEFP whose value is demonstrated by its impacts on participants, learners, patients, teaching faculty, institutions, the greater medical education community, and the population's health.