Better together: A multistakeholder approach to developing specialty-wide entrustable professional activities in emergency medicine.

Purpose: Entrustable professional activities (EPAs) are a widely used framework for curriculum and assessment, yet the variability in emergency medicine (EM) training programs mandates the development of EPAs that meet the needs of the specialty as a whole. This requires eliciting and incorporating the perspectives of multiple stakeholders (i.e., faculty, residents, and patients) in the development of EPAs. Without a shared understanding of what a resident must be able to do upon graduation, we run the risk of advancing ill-prepared residents that may provide inconsistent care. Methods: In an effort to address these challenges, beginning in February 2020, the authors assembled an advisory board of 25 EM faculty to draft and reach consensus on a final list of EPAs that can be used across all training programs within the specialty of EM. Using modified Delphi methodology, the authors came to consensus on an initial list of 22 EPAs. The authors presented these EPAs to faculty supervisors, residents, and patients for refinement. The authors collated and analyzed feedback from focus groups of residents and patients using thematic analysis. The EPAs were subsequently refined based on this feedback. Results: Stakeholders in EM residency training endorsed a final revised list of 22 EPAs. Stakeholder focus groups highlighted two main thematic considerations that helped shape the finalized list of EM EPAs: attention to the meaningful nuances of EPA language and contextualizing the EPAs and viewing them developmentally. Conclusions: To foreground all key stakeholders within the EPA process for EM, the authors chose within the development process to draft; come to consensus; and refine EPAs for EM in collaboration with relevant faculty, patient, and resident stakeholders. Each stakeholder group contributed meaningfully to the content and intended implementation of the EPAs. This process may serve as a model for others in developing stakeholder-responsive EPAs.

Help Me Help You: How to Request a Letter of Recommendation.

Letters of recommendation (LORs) are an essential component of a career in medicine. The process for obtaining certain letters, particularly those associated with scheduled periods of professional transition, often is governed by established institutional or specialty norms. The process of requesting LORs in more common scenarios-local or national awards, committee assignments, and leadership positions-many times is less clearly defined, however. Despite the important role that LORs play in professional development, the published literature on how to solicit a recommendation is limited, creating challenges for both those requesting LORs ("applicants") and the letter writers. This perspective piece offers insight on how to best identify and communicate with a potential writer. These suggestions are derived from the limited relevant literature and from the authors' experience both with requesting letters themselves and writing letters as leaders in undergraduate and graduate medical education. The goal is to reduce ambiguity for applicants and ensure that writers receive the information necessary to provide an informed and effective recommendation.

Clinicians' Insights on Emergency Department Boarding: An Explanatory Mixed Methods Study Evaluating Patient Care and Clinician Well-Being.

BACKGROUND: The aim of this study was to describe clinicians' insights into the quality and safety of patient care delivered to emergency department (ED) boarding patients, as well as clinician safety and satisfaction related to ED boarding. METHODS: This was a single-site, mixed methods sequential explanatory study. Quantitative data were obtained from a cross-sectional survey sent to ED attending physicians, resident physicians, advanced practice providers, and nurses. Semistructured focus group interviews with a subsample of participants sought to add depth to the interpretation of survey data and identify areas of improvement in boarding care. Chi-square and Wilcoxon rank sum tests were used to evaluate for response differences between groups. Qualitative data were thematically coded and analyzed. RESULTS: A total of 94 questionnaires were obtained for a response rate of 34.1%. Clinicians reported that boarding highly contributed to the perception of burnout. All groups reported high rates of perceived verbal and/or physical abuse from boarding patients (86.8% of nurses, 41.1% of providers, p = 0.0002). A total of 39 clinicians participated in focus groups regarding boarding care, and six themes were identified, including patient safety concerns, lack of knowledge/resources/training, and poor communication. Key themes identified as possible solutions to improve care included standardization of care, proactive planning, and culture change. CONCLUSION: Clinicians identified many concerns regarding patient safety and the quality of care delivered to boarding patients and identified several areas for improvement. Clinicians also felt that boarding negatively affected their satisfaction and safety.

Curated collections for educators: Nine key articles and article series for teaching qualitative research methods.

Background: Qualitative research explains observations, focusing on how and why phenomena and experiences occur. Qualitative methods go beyond quantitative data and provide critical information inaccessible through quantitative methods. However, at all levels of medical education, there is insufficient exposure to qualitative research. As a result, residents and fellows complete training ill-equipped to appraise and conduct qualitative studies. As a first step to increasing education in qualitative methods, we sought to create a curated collection of papers for faculty to use in teaching qualitative research at the graduate medical education (GME) level. Methods: We conducted literature searches on the topic of teaching qualitative research to residents and fellows and queried virtual medical education and qualitative research communities for relevant articles. We searched the reference lists of all articles found through the literature searches and online queries for additional articles. We then conducted a three-round modified Delphi process to select papers most relevant to faculty teaching qualitative research. Results: We found no articles describing qualitative research curricula at the GME level. We identified 74 articles on the topic of qualitative research methods. The modified Delphi process identified the top nine articles or article series most relevant for faculty teaching qualitative research. Several articles explain qualitative methods in the context of medical education, clinical care, or emergency care research. Two articles describe standards of high-quality qualitative studies, and one article discusses how to conduct the individual qualitative interview to collect data for a qualitative study. Conclusions: While we identified no articles reporting already existing qualitative research curricula for residents and fellows, we were able to create a collection of papers on qualitative research relevant to faculty seeking to teach qualitative methods. These papers describe key qualitative research concepts important in instructing trainees as they appraise and begin to develop their own qualitative studies.

Education Theory Made Practical: Creating open educational resources via an apprenticeship model.

Introduction: Clinical faculty may have limited knowledge of education theories and best practices in health professions education. Many faculty development programs focus on passive learning with limited application to practice. There is a need for more active engagement for early career educators. Methods: We created an apprenticeship-based electronic book series focused on translating education theories into practical applications for clinician educators. Chapters were authored by teams of two to four geographically separated early career educators, who were tasked with explaining an education theory and relating it to their educational practice. The chapters underwent internal peer review, followed by open peer review as a blog post and eventual publication. Usage data were collected, and surveys were sent to authors and end-users. Results: Six volumes (60 total chapters) have been created to date by 180 unique authors and 17 editors over a 6-year period. There have been 65,571 total blog page views and 17,180 total book downloads across the five published volumes. Authors reported an increase in their perceived knowledge (pre 2.6 ± 1.7 vs. post 7.2 ± 1.1, mean difference 4.5/9.0, 95% confidence interval [CI] 4.0-5.0, p < 0.001) after writing their chapter. Authors also reported career benefits including authorship for academic advancement/promotion and developing an area of education theory expertise. End-users also reported a mean increase in their perceived knowledge (pre 4.4 ± 2.5 vs. post 7.3 ± 1.4, mean difference 2.9/9.0, 95% CI 2.1-3.8, p < 0.001) after reading a chapter. Conclusion: The Education Theory Made Practical electronic book series represents a proof of concept for an apprenticeship-based model to teach education theory, while also creating scholarship and open access resources for the broader community.

Teaching Quality Improvement: The Use of Education Theories Across the Medical Education Spectrum.

It is well recognized that the principles and practices of patient safety and quality improvement (QI) need to be included in medical education. The implementation of patient safety and QI learning experiences at the undergraduate medical education (UME) and graduate medical education (GME) levels has been variable. Consistent teaching of QI across the UME-GME-continuing medical education (CME) spectrum may result in a systemic change of improved patient care and patient safety in clinical practice. We propose using education theories to frame the development of QI curricula for a longitudinal integration in medical education and clinical practice. The basic principles of four education theories, namely, reflective practice, deliberate practice, social constructivism, and organizational learning, are briefly described, and examples of their applications to QI teaching are discussed. The incorporation of education theory into the design and implementation of a longitudinal QI curriculum threaded across the UME-GME-CME spectrum may empower learners with a comprehensive and lasting understanding of QI principles and training in patient safety practice, which are essential prerequisites for the formation of a physician workforce capable of creating sustainable change in patient care.

Implementation of a standardized pregnancy screening process to address gender disparities in radiology turn-around-time and ED length of stay.

OBJECTIVE: The objective of this initiative was to quantify and intervene upon suspected gender disparities in CT turn-around-time and emergency department (ED) length of stay. METHODS: This was a single-site before-after quality improvement initiative including patients aged 12-50 who underwent CT chest and/or abdomen/pelvis. The intervention included protocolization of the pregnancy screening process in triage. Primary outcomes included the difference between women of childbearing age and similarly aged men in regards to CT turn-around-time and ED length of stay. Pre- and post-intervention data were analyzed, including an "intensive intervention period" subanalysis. RESULTS: CT turn-around-time for women of childbearing age was 19 min longer than for similarly aged men at baseline and did not change significantly post-intervention. ED length of stay was 27 min longer for women of childbearing age compared to similarly aged men at baseline and 7 min longer post-intervention, although this was still a significant difference. During the intensive intervention period, CT turn-around-time for women of childbearing age was 15 min longer than similarly aged men but the difference in ED length of stay of 10 min was no longer significant. CONCLUSIONS: There is gender disparity in CT turn-around-time and ED length of stay in our ED, highlighting an important area for improvement to promote equitable care. A quality improvement initiative that aimed to protocolize pregnancy testing in triage did not show sustainable improvement in these outcomes but did result in increased pregnancy testing.

RéSUMé: OBJECTIF: L'objectif de cette initiative était de quantifier et d'intervenir sur les disparités présumées entre les sexes dans le délai d'exécution du scanner et la durée du séjour au service des urgences. MéTHODES: Il s'agissait d'une initiative d'amélioration de la qualité à un seul site avant-après, incluant les patients âgés de 12 à 50 ans qui ont subi une TDM thoracique et/ou abdominale/bassin. L'intervention comprenait la protocolisation du processus de dépistage de la grossesse lors du triage. Les résultats primaires comprenaient la différence entre les femmes en âge de procréer et les hommes du même âge en ce qui concerne le délai d'exécution du scanner et la durée du séjour aux urgences. Les données avant et après l'intervention ont été analysées, y compris une sous-analyse de la "période d'intervention intensive". RéSULTATS: Le temps d'exécution de la TDM chez les femmes en âge de procréer était 19 minutes de plus que chez les hommes de même âge au départ et n'a pas changé de façon significative après l'intervention. La durée de séjour aux urgences était de 27 minutes de plus pour les femmes en âge de procréer par rapport aux hommes du même âge au départ et de 7 minutes de plus après l'intervention, bien que cette différence reste significative. Au cours de la période d'intervention intensive, le temps de passage au scanner des femmes en âge de procréer était de 15 minutes plus long que celui des hommes du même âge, mais la différence de 10 minutes dans la durée de séjour aux urgences n'était plus significative. CONCLUSIONS: Il existe une disparité entre les sexes en ce qui concerne le délai d'exécution du scanner et la durée de séjour dans nos urgences, ce qui met en évidence un domaine important à améliorer pour promouvoir des soins équitables. Une initiative d'amélioration de la qualité visant à protocoliser le test de grossesse au triage n'a pas montré d'amélioration durable de ces résultats mais a entraîné une augmentation des tests de grossesse.

Implementation and Effects of a Team-Based Quality Improvement Training Program Across a Health System: The Northwestern Medicine Academy for Quality and Safety Improvement.

BACKGROUND: Professional organizations emphasize the need to train health care professionals in quality improvement (QI). Many reports of QI education programs involve small numbers of participants. Little is known about QI education programs on a larger scale and whether participants subsequently engage in QI activities. METHODS: The Northwestern Medicine Academy for Quality and Safety Improvement (NM AQSI) was developed to prepare individuals across the Northwestern health system to lead QI. The 7-month program consists of classwork and team-based project work. Participant knowledge was assessed using a multiple-choice test and adapted Quality Improvement Knowledge Application Tool (QIKAT). The study team surveyed participants 18 months after AQSI completion to assess their activity in QI. Project status was assessed at AQSI completion and at 18 months. RESULTS: Over 8 years, 80 teams consisting of 441 individuals participated, representing a range of specialties, settings, and professions. Participants had higher multiple-choice test (70.7 ± 14.0 vs. 78.1 ± 13.0; p < 0.001) and adapted QIKAT scores (56.1 ± 15.9 vs. 60.8 ± 15.8; p < 0.001) after AQSI. The majority of participants at 18 months (180/243; 74.1%) had engaged in subsequent QI efforts; many (105/243; 43.2%) had led other QI projects, and (103/243; 42.4%) provided QI mentorship to others. The majority of teams (53/80; 66.3%) improved project measure performance. CONCLUSION: NM AQSI is a team-based QI training program that shows measurable improvements in care and a high degree of participants' subsequent involvement in QI. Other health systems may use a similar approach to successfully train health care professionals to lead QI.

Creation and Implementation of a Mastery Learning Curriculum for Emergency Department Thoracotomy.

INTRODUCTION: Emergency department thoracotomy (EDT) is a lifesaving procedure within the scope of practice of emergency physicians. Because EDT is infrequently performed, emergency medicine (EM) residents lack opportunities to develop procedural competency. There is no current mastery learning curriculum for residents to learn EDT. The purpose of this study was to develop and implement a simulation-based mastery learning curriculum to teach and assess EM residents' performance of the EDT. METHODS: We developed an EDT curriculum using a mastery learning framework. The minimum passing standard (MPS) for a previously developed 22-item checklist was determined using the Mastery Angoff approach. EM residents at a four-year academic EM residency program underwent baseline testing in performing an EDT on a simulation trainer. Performance was scored by two raters using the checklist. Learners then participated in a novel mastery learning EDT curriculum that included an educational video, hands-on instruction, and deliberate practice. After a three-month period, residents then completed initial post testing. Residents who did not meet the minimum passing standard after post testing participated in additional deliberate practice until mastery was obtained. Baseline and post-test scores, and time to completion of the procedure were compared with paired t-tests. RESULTS: Of 56 eligible EM residents, 54 completed baseline testing. Fifty-two residents completed post-testing until mastery was reached. The minimum passing standard was 91.1%, (21/22 items correct on the checklist). No participants met the MPS at the baseline assessment. After completion of the curriculum, all residents subsequently reached the MPS, with deliberate practice sessions not exceeding 40 minutes. Scores from baseline testing to post-testing significantly improved across all postgraduate years from a mean score of 10.2/22 to 21.4/22 (p <0.001). Mean time to complete the procedure improved from baseline testing (6 minutes [min] and 21 seconds [sec], interquartile range [IQR] = 4 min 54 sec - 7 min 51 sec) to post-testing (5 min 19 seconds, interquartile range 4 min 17sec - 6 min 15 sec; p = 0.001). CONCLUSION: This simulation-based mastery learning curriculum resulted in all residents performing an EDT at a level that met or exceeded the MPS with an overall decrease in time needed to perform the procedure.

Emergency Department Thoracotomy: Development of a Reliable, Validated Checklist for Procedural Training.

OBJECTIVES: Emergency department thoracotomy (EDT) is a rare and challenging procedure. Emergency medicine (EM) residents have limited opportunities to perform the procedure in clinical or educational settings. Standardized, reliable, validated checklists do not exist to evaluate procedural competency. The objectives of this project were twofold: 1) to develop a checklist containing the critical actions for performing an EDT that can be used for future procedural skills training and 2) to evaluate the reliability and validity of the checklist for performing EDT. METHODS: After a literature review, a preliminary 22-item checklist was developed and disseminated to experts in EM and trauma surgery. A modified Delphi method was used to revise the checklist. To assess usability of the checklist, EM and trauma surgery faculty and residents were evaluated performing an EDT while inter-rater reliability was calculated with Cohen's kappa. A Student's t-test was used to compare the performance of participants who had or had not performed a thoracotomy in clinical practice. Item-total correlation was calculated for each checklist item to determine discriminatory ability. RESULTS: A final 22-item checklist was developed for EDT. The overall inter-rater reliability was strong (κ = 0.84) with individual item agreement ranging from moderate to strong (κ = 0.61 to 1.00). Experts (attending physicians and senior residents) performed well on the checklist, achieving an average score of 80% on the checklist. Participants who had performed EDT in clinical practice performed significantly better than those that had not, achieving an average of 80.7% items completed versus 52.3% (p < 0.05). Seventeen of 22 items had an item-total correlation greater than 0.2. CONCLUSIONS: A final 22-item consensus-based checklist was developed for the EDT. Overall inter-rater reliability was strong. This checklist can be used in future studies to serve as a foundation for curriculum development around this important procedure.

The Standardized Letter of Evaluation Narrative: Differences in Language Use by Gender.

INTRODUCTION: Prior research demonstrates gender differences in language used in letters of recommendation. The emergency medicine (EM) Standardized Letter of Evaluation (SLOE) format limits word count and provides detailed instructions for writers. The objective of this study is to examine differences in language used to describe men and women applicants within the SLOE narrative. METHODS: All applicants to a four-year academic EM residency program within a single application year with a first rotation SLOE available were included in the sample. We used the Linguistic Inquiry and Word Count (LIWC) program to analyze word frequency within 16 categories. Descriptive statistics, chi-squared, and t-tests were used to describe the sample; gender differences in word frequency were tested for using Mann-Whitney U tests. RESULTS: Of 1117 applicants to the residency program, 822 (82%) first-rotation SLOEs were available; 64% were men, and 36% were women. We did not find a difference in baseline characteristics including age (mean 27 years), top 25 schools (22.5%), Alpha Omega Alpha Honor Medical Society rates (13%), and having earned advanced degrees (10%). The median word count per SLOE narrative for men was 171 and for women was 180 (p = 0.15). After adjusting for letter length, word frequency differences between genders were only present in two categories: social words (women: 23 words/letter; men: 21 words/letter, p = 0.02) and ability words (women: 2 words/letter; men: 1 word/letter, p = 0.04). We were unable to detect a statistical difference between men and women applicants in the remaining categories, including words representing communal traits, agentic traits, standout adjectives, grindstone traits, teaching words, and research words. CONCLUSION: The small wording differences between genders noted in two categories were statistically significant, but of unclear real-world significance. Future work is planned to evaluate how the SLOE format may contribute to this relative lack of bias compared to other fields and formats.

Immediate and long-term effects of a team-based quality improvement training programme.

BACKGROUND: Although many studies of quality improvement (QI) education programmes report improvement in learners' knowledge and confidence, the impact on learners' future engagement in QI activities is largely unknown and few studies report project measures beyond completion of the programme. METHOD: We developed the Academy for Quality and Safety Improvement (AQSI) to prepare individuals, across multiple departments and professions, to lead QI. The 7-month programme consisted of class work and team-based project work. We assessed participants' knowledge using a multiple choice test and an adapted Quality Improvement Knowledge Assessment Test (QIKAT) before and after the programme. We evaluated participants' postprogramme QI activity and project status using surveys at 6 and 18 months. RESULTS: Over 5 years, 172 individuals and 32 teams participated. Participants had higher multiple choice test (71.9±12.7 vs 79.4±13.2; p<0.001) and adapted QIKAT scores (55.7±16.3 vs 61.8±14.7; p<0.001) after the programme. The majority of participants at 6 months indicated that they had applied knowledge and skills learnt to improve quality in their clinical area (129/148; 87.2%) and to implement QI interventions (92/148; 62.2%). At 18 months, nearly half (48/101; 47.5%) had led other QI projects and many (41/101; 40.6%) had provided QI mentorship to others. Overall, 14 (43.8%) teams had positive postintervention results at AQSI completion and 20 (62.5%) had positive results at some point (ie, completion, 6 months or 18 months after AQSI). CONCLUSIONS: A team-based QI training programme resulted in a high degree of participants' involvement in QI activities beyond completion of the programme. A majority of team projects showed improvement in project measures, often occurring after completion of the programme.

Women's Night in Emergency Medicine Mentorship Program: A SWOT Analysis.

INTRODUCTION: Women in emergency medicine (EM) at all career stages report gender-specific obstacles to satisfaction and advancement. Programs that facilitate longitudinal mentoring, professional development, and networking may ameliorate these barriers. METHODS: We designed and implemented a program for female residents, faculty, and alumnae from our EM training program to enhance social support, leadership training and professional mentorship opportunities. An anonymous, online survey was sent to participants at the end of the academic year, using a SWOT (strengths, weaknesses, opportunities, and threats) format. The survey collected free-text responses designed to evaluate the program. RESULTS: Of 43 invited participants, 32 responded (74.4%). Eight themes emerged from the free-text responses and were grouped by SWOT domain. We identified four themes relating to the "strength" domain: 1) creating a dedicated space; 2) networking community; 3) building solidarity; and 4) providing forward guidance. Responses to the "weaknesses" and "threats" questions were combined due to overlapping codes and resulted in three themes: 5) barriers to participation; 6) the threat of poorly structured events lapsing into negativity; and 7) concerns about external optics. A final theme: 8) expansion of program scope was noted in the "opportunity" domain. CONCLUSION: This program evaluation of the Women's Night curriculum demonstrates it was a positive addition to the formal curriculum, providing longitudinal professional development opportunities. Sharing the strengths of the program, along with identified weaknesses, threats, and opportunities for advancement allows other departments to learn from this experience and implement similar models that use existing intellectual and social capital.

A User's Guide to the ALiEM Emergency Medicine Match Advice Web Series.

ALiEM EM Match Advice is a web series hosted on the Academic Life in Emergency Medicine website. The intended audience includes senior medical students seeking a residency in emergency medicine (EM) and the faculty members who advise them. Each episode features a panel of three EM program directors who discuss a critical step in the residency application process. This article serves as a user's guide to the series, including a timeline for viewing each episode, brief summaries of the panel discussions, and reflection questions for discussion between students and their faculty advisors.