Hawks and Doves in Standardized Letters of Evaluation: 6 Years of Rating Distributions and Trends in Emergency Medicine.

Background Standardized Letters of Evaluation (SLOEs) are an important part of resident selection in many specialties. Often written by a group, such letters may ask writers to rate applicants in different domains. Prior studies have noted inflated ratings; however, the degree to which individual institutions are "doves" (higher rating) or "hawks" (lower rating) is unclear. Objective To characterize institutional SLOE rating distributions to inform readers and developers regarding potential threats to validity from disparate rating practices. Methods Data from emergency medicine (EM) SLOEs between 2016 and 2021 were obtained from a national database. SLOEs from institutions with at least 10 letters per year in all years were included. Ratings on one element of the SLOE-the "global assessment of performance" item (Top 10%, Top Third, Middle Third, and Lower Third)-were analyzed numerically and stratified by predefined criteria for grading patterns (Extreme Dove, Dove, Neutral, Hawk, Extreme Hawk) and adherence to established guidelines (Very High, High, Neutral, Low, Very Low). Results Of 40 286 SLOEs, 20 407 met inclusion criteria. Thirty-five to 50% of institutions displayed Neutral grading patterns across study years, with most other institutional patterns rated as Dove or Extreme Dove. Adherence to guidelines was mixed and fewer than half of institutions had Very High or High adherence each year. Most institutions underutilize the Lower Third rating. Conclusions Despite explicit guidelines for the distribution of global assessment ratings in the EM SLOE, there is high variability in institutional rating practices.

The Emergency Medicine Group Standardized Letter of Evaluation as a Workplace-based Assessment: The Validity Is in the Detail.

INTRODUCTION: Interest is growing in specialty-specific assessments of student candidates based on clinical clerkship performance to assist in the selection process for postgraduate training. The most established and extensively used is the emergency medicine (EM) Standardized Letter of Evaluation (SLOE), serving as a substitute for the letter of recommendation. Typically developed by a program's leadership, the group SLOE strives to provide a unified institutional perspective on performance. The group SLOE lacks guidelines to direct its development raising questions regarding the assessments, processes, and standardization programs employ. This study surveys EM programs to gather validity evidence regarding the inputs and processes involved in developing group SLOEs. METHODS: A structured telephone interview was administered to assess the input data and processes employed by United States EM programs when generating group SLOEs. RESULTS: With 156/178 (87.6%) of Accreditation Council of Graduate Medical Education-approved programs responding, 146 (93.6%) reported developing group SLOEs. Issues identified in development include the following: (1) 84.9% (124/146) of programs limit the consensus process by not employing rigorous methodology; (2) several stakeholder groups (nurses, patients) do not participate in candidate assessment placing final decisions at risk for construct under-representation; and (3) clinical shift assessments don't reflect the task-specific expertise of each stakeholder group nor has the validity of each been assessed. CONCLUSION: Success of the group SLOE in its role as a summative workplace-based assessment is dependent upon valid input data and appropriate processes. This study of current program practices provides specific recommendations that would strengthen the validity arguments for the group SLOE.

Emergency Medicine Standardized Letter of Evaluation (SLOE): Findings From the New Electronic SLOE Format.

BACKGROUND: Emergency medicine (EM) uses a standardized template for residency application letters of recommendations. In 1997, the Standardized Letter of Recommendation was developed with categories for applicant comparison. Now named the Standardized Letter of Evaluation (SLOE), it is a universal requirement in the EM residency application process. In 2016, a website called "eSLOE" was launched for credentialed academic EM faculty to enter applicant data, which generates a SLOE. OBJECTIVE: This article outlines website creation for the eSLOE and its successful national implementation in the 2016-2017 EM residency application cycle. We analyzed current trends in applicant assessments from the eSLOE data and compared them to prior data. METHODS: Data from 2016-2017 were sorted and analyzed for each question on the eSLOE. An analysis of Global Assessment and Qualifications for EM rankings, clerkship grade, and comparison with prior SLOE data was performed. RESULTS: Analysis of 6715 eSLOEs for 3138 unique applicants revealed the following Global Assessment rankings: top 10%, top one-third, middle one-third, and lower one-third. There was less spread with the distribution for clerkship grade and Qualifications for EM. The 2011-2012 standard letter of recommendation global assessment data, with top 10%, top one-third, middle one-third, and lower one-third, also revealed top-clustered results with less spread compared with the ranking usage in 2016-2017. CONCLUSIONS: Results indicate an improved spread of all rank categories for Global Assessment, enhancing the eSLOE's applicant discrimination. There has been an overall improvement in rank designation when compared with previously published data.

When the Learner Is the Expert: A Simulation-Based Curriculum for Emergency Medicine Faculty.

Emergency physicians supervise residents performing rare clinical procedures, but they infrequently perform those procedures independently. Simulation offers a forum to practice procedural skills, but simulation labs often target resident learners, and barriers exist to faculty as learners in simulation-based training. Simulation-based curricula focused on improving emergency medicine (EM) faculty's rare procedure skills were not discovered on review of published literature. Our objective was to create a sustainable, simulation-based faculty education curriculum for rare procedural skills in EM. Between 2012 and 2019, most EM teaching faculty at a single, urban, Level 1 trauma center completed an annual two-hour simulation-based rare procedure lab with small-group learning and guided hands-on instruction, covering 30 different procedural education sessions for faculty learners. A questionnaire administered before and after each session assessed EM faculty physicians' self-perceived ability to perform these rare procedures. Participants' self-reported confidence in their performance improved for all procedures, regardless of prior procedural experience. Faculty participation was initially mandatory, but is now voluntary. Diverse strategies were used to address barriers in this learner group including eliciting learner feedback, offering continuing medical education credits, gradual roll-out of checklist assessments, and welcoming expertise of faculty leaders from EM and other specialties and professions. Participants perceived training to be most helpful for the most rarely-encountered clinical procedures. Similar curricula could be implemented with minimal risk at other institutions.

Assessing the Informed Consent Skills of Emergency Medicine Resident Physicians.

BACKGROUND: Informed consent (IC) is an essential component of shared medical decision making between patients and providers in emergency medicine (EM). The basic components required for adequate consent are well described, yet little is published investigating whether EM residents demonstrate adequate IC skills. OBJECTIVE: The objectives were to assess the ability of EM residents to obtain IC for an invasive emergency procedure using a novel assessment tool and to assess reliability and validity of the tool. METHODS: This was an observational study in which participants were initially blinded to the primary objectives of the study. Each participant conducted a video-recorded history and physical examination with a standardized patient, requiring tube thoracostomy due to spontaneous pneumothorax. Two faculty EM physicians independently reviewed the videos and evaluated the participants' IC skills. First, they gave an overall impression of whether IC was obtained; they then evaluated the participants using a 30-point scoring tool based on the five elements of IC (decision-making capacity, disclosure, voluntariness, understanding, and physician recommendation). Upon all participants' case completion, we revealed the primary objectives and gave participants the option to withdraw from the study. Descriptive statistics and kappa coefficient were generated from the data collected. RESULTS: Twenty-two residents completed the study. None withdrew from the study after the primary objectives were revealed. Twenty residents (91%) obtained adequate IC based on both reviewers' overall impression. One disagreement occurred between reviewers (κ = 0.64). The mean IC score on a 30-point scale was 18.5 ± 0.5. CONCLUSIONS: In a simulated setting, most EM residents at this training program possess the knowledge and skills necessary to obtain IC prior to an invasive procedure. The assessment tool appears reliable and demonstrates construct validity.

Do Emergency Medicine Residency Graduates Feel Prepared to Manage Closed Fractures After Training?

OBJECTIVES: Fractures comprise 3% of all emergency department (ED) visits. Although emergency physicians are often responsible for managing most of the initial care of these patients, many report a lack of proficiency and comfort with these skills. The primary objective was to assess how prepared recent emergency medicine (EM) residency graduates felt managing closed fractures upon completion of residency. Secondary objectives included whether residency training or independent practice contributed most to the current level of comfort with these procedures and which fractures were most commonly reduced without orthopedic consultation. METHODS: An anonymous online survey was sent to graduates from seven EM residency programs over a 3-month period to evaluate closed fracture reduction training, practice, and comfort level. Each site primary investigator invited graduates from 2010 to 2014 to participate and followed a set schedule of reminders. RESULTS: The response rate was 287/384 (74.7%) and included 3-year (198/287, 69%) and 4-year (89/287, 31%) programs. Practice in community, academic, and hybrid ED settings was reported by 150/287 (52.3%), 64/287 (22.3%), and 73/287 (25.4%), respectively. It was indicated by 137/287 (47.7%) that they reduce closed fractures without a bedside orthopedic consultation greater than 75% of the time. The majority of graduates felt not at all prepared (35/287, 12.2%) or somewhat prepared (126/287, 43.9%) upon residency graduation. Postresidency independent practice contributed most to the current level of comfort for 156/287 (54.4%). The most common fractures requiring reduction were wrist/distal radius and/or ulna, next finger/hand, and finally, ankle/distal tibia and/or fibula. CONCLUSIONS: Although most recent graduates feel at least "somewhat" prepared to manage closed fractures in the ED, most felt that independent practice was a greater contributor to their current level of comfort than residency training. Recent graduates indicate that fracture reduction without orthopedic consultation is common in today's clinical practice. This survey identifies common fractures requiring reduction which EM residencies may wish to consider prioritizing in their emergency orthopedic curricula to better prepare their residents for independent clinical practice.

The Standardized Letter of Evaluation for Postgraduate Training: A Concept Whose Time Has Come?

A medical student's letter of recommendation for postgraduate training applications should provide a fair and accurate assessment of academic and clinical performance, as well as define character attributes pertinent to the practice of medicine. Since its inception in 1997, the emergency medicine (EM) standardized letter of evaluation (SLOE) has evolved into an instrument that provides just such an assessment. Concise, standardized, and discriminating in its assessment of performance relevant to the practice of EM, the SLOE is judged by program directors in EM as the most valuable component of a potential resident's application. Other specialties would benefit from such a specialty-specific perspective, which is currently lacking in most Electronic Residency Application Service application materials. Creation of specialty-specific SLOEs which define performance metrics or competencies and noncognitive personality traits critical to each unique specialty would add substantially to the holistic review of our graduating students. As a result, specialty-specific SLOEs would increase the likelihood that programs could effectively identify applicants who would not only be a "good fit" for their programs but also graduate to become successful physicians.

Literature Evidence on Live Animal Versus Synthetic Models for Training and Assessing Trauma Resuscitation Procedures.

There are many models currently used for teaching and assessing performance of trauma-related airway, breathing, and hemorrhage procedures. Although many programs use live animal (live tissue [LT]) models, there is a congressional effort to transition to the use of nonanimal- based methods (i.e., simulators, cadavers) for military trainees. We examined the existing literature and compared the efficacy, acceptability, and validity of available models with a focus on comparing LT models with synthetic systems. Literature and Internet searches were conducted to examine current models for seven core trauma procedures. We identified 185 simulator systems. Evidence on acceptability and validity of models was sparse. We found only one underpowered study comparing the performance of learners after training on LT versus simulator models for tube thoracostomy and cricothyrotomy. There is insufficient data-driven evidence to distinguish superior validity of LT or any other model for training or assessment of critical trauma procedures.

Council of emergency medicine residency directors standardized letter of recommendation writers' questionnaire.

BACKGROUND: The Council of Emergency Medicine Residency Directors (CORD) Standardized Letter of Recommendation (SLOR) has become the primary tool used by emergency medicine (EM) faculty to evaluate residency candidates. A survey was created to describe the training, beliefs, and usage patterns of SLOR writers. METHODS: The SLOR Task Force created the survey, which was circulated to the CORD listserv in 2012. RESULTS: Forty-six percent of CORD members (320 of 695) completed the survey. Of the respondents, 39% (125 of 320) had fewer than 5 years of experience writing SLOR letters. Most were aware of published guidelines, and most reported they learned how to write a SLOR on their own (67.4%, 182 of 270). Sixty-eight percent (176 of 258) admitted to not following the instructions for certain questions. Self-reported grade inflation occurred "rarely" 36% (97 of 269) of the time and not at all 40% (107 of 269) of the time. CONCLUSIONS: The CORD SLOR has become the primary tool used by EM faculty to evaluate candidates applying for residency in EM. The SLOR has been in use in the EM community for 16 years. However, our study has identified some problems with its use. Those issues may be overcome with a revised format for the SLOR and with faculty training in the writing and use of this document.

2-(4-Iodo-2,5-dimethoxyphenyl)-N-[(2-methoxyphenyl)methyl]ethanamine (25I-NBOMe): clinical case with unique confirmatory testing.

INTRODUCTION: 2C designer drugs have been in use since the 1970s, but new drugs continue to develop from substitutions to the base phenethylamine structure. This creates new clinical profiles and difficulty with laboratory confirmation. 2-(4-Iodo-2,5-dimethoxyphenyl)-N-[(2-methoxyphenyl)methyl]ethanamine (25I-NBOMe) is a relatively new 2C drug that is more potent than structural 2C analogs; exposure reports are rare. Testing for 2C drugs is developing; specific testing for new analogs such as 25I-NBOMe is a challenge. These drugs do not reliably trigger a positive result on rapid drug immunoassays. Additionally, most facilities with confirmatory testing capabilities will not identify 25I-NBOMe; methods for detecting 25I-NBOMe in biological samples have not been clearly described nor have optimal metabolic targets for detecting 25I-NBOMe ingestion. CASE REPORT: An 18-year-old female presented following use of 25I-NBOMe. She had an isolated brief seizure, tachycardia, hypertension, agitation, and confusion. She improved with intravenously administered fluids and benzodiazepines and was discharged 7 h postingestion. Urine was analyzed using quantitative LC-MS/MS methodology for 25I-NBOMe, 2-(4-chloro-2,5-dimethoxyphenyl)-N-[(2-methoxyphenyl)-methyl]ethanamine (25C-NBOMe), and 2-(2,5-dimethoxyphenyl)-N-(2-methoxybenzyl)ethanamine (25H-NBOMe). 25I-NBOMe was found at a concentration of 7.5 ng/mL, and 25H-NBOMe was detected as well. Additional testing was pursued to characterize the metabolism of 25I-NBOMe; the sample was reanalyzed with UPLC-time-of-flight mass spectrometry to identify excreted metabolites. The sample was additionally analyzed for the presence of 2,5-dimethoxy-4-iodophenethylamine (2C-I), 4-bromo-2,5-dimethoxyphenethylamine (2C-B), and 1-(2,5-dimethoxy-4-ethylphenyl)-2-aminoethane (2C-E). DISCUSSION: This is a report of a patient presenting following exposure to 25I-NBOMe, a dangerous member of the evolving 2C drug class. The exposure was confirmed in a unique manner that could prove helpful in guiding further patient analysis and laboratory studies.

Characterization of the Council of Emergency Medicine Residency Directors' standardized letter of recommendation in 2011-2012.

OBJECTIVES: The Council of Emergency Medicine Residency Directors (CORD) introduced the standardized letter of recommendation (SLOR) in 1997, and it has become a critical tool for assessing candidates for emergency medicine (EM) training. It has not itself been evaluated since the initial studies associated with its introduction. This study characterizes current SLOR use to evaluate whether it serves its intended purpose of being standardized, concise, and discriminating. METHODS: This retrospective, multi-institutional study evaluated letters of recommendation from U.S. allopathic applicants to three EM training programs during the 2011-2012 Electronic Residency Application Service (ERAS) application cycle. Distributions of responses to each question on the SLOR were calculated, and the free-text responses were analyzed. Two pilots, performed on five applicants each, assisted in developing a strategy for limiting interrater reliability. RESULTS: Each of the three geographically diverse programs provided a complete list of U.S. allopathic applicants to their program. Upon randomization, each program received a list of coded applicants unique to their program randomly selected for data collection. The number of applicants was selected to reach a goal of approximately 200 SLORs per site (n = 602). Among this group, comprising 278 of 1,498 applicants (18.6%) from U.S. allopathic schools, a total of 1,037 letters of recommendation were written, with 724 (69.8%) written by emergency physicians. SLORs represented 57.9% (602/1037) of all LORs (by any kind of author) and 83.1% (602/724) of letters written by emergency physicians. Three hundred ninety-two of 602 SLORs had a single author (65.1%). For the question on "global assessment," students were scored in the top 10% in 234 of 583 of applications (40.1%; question not answered by some), and 485 of 583 (83.2%) of the applicants were ranked above the level of their peers. Similarly, >95% of all applicants were ranked in the top third compared to peers, for all but one section under "qualifications for emergency medicine." For 405 of 602 of all SLORs (67.2%), one or more questions were left unanswered, while 76 of all SLORs (12.6%) were "customized" or changed from the standard template. Finally, in 291 of 599 of SLORs (48.6%), the word count was greater than the recommended maximum of 200 words. CONCLUSIONS: Grade inflation is marked throughout the SLOR, limiting its ability to be discriminating. Furthermore, template customization and skipped questions work against the intention to standardize the SLOR. Finally, it is not uncommon for comments to be longer than guideline recommendations. As an assessment tool, the SLOR could be more discerning, concise, and standardized to serve its intended purpose.

Factors that influence medical student selection of an emergency medicine residency program: implications for training programs.

OBJECTIVES: An understanding of student decision-making when selecting an emergency medicine (EM) training program is essential for program directors as they enter interview season. To build upon preexisting knowledge, a survey was created to identify and prioritize the factors influencing candidate decision-making of U.S. medical graduates. METHODS: This was a cross-sectional, multi-institutional study that anonymously surveyed U.S. allopathic applicants to EM training programs. It took place in the 3-week period between the 2011 National Residency Matching Program (NRMP) rank list submission deadline and the announcement of match results. RESULTS: Of 1,525 invitations to participate, 870 candidates (57%) completed the survey. Overall, 96% of respondents stated that both geographic location and individual program characteristics were important to decision-making, with approximately equal numbers favoring location when compared to those who favored program characteristics. The most important factors in this regard were preference for a particular geographic location (74.9%, 95% confidence interval [CI] = 72% to 78%) and to be close to spouse, significant other, or family (59.7%, 95% CI = 56% to 63%). Factors pertaining to geographic location tend to be out of the control of the program leadership. The most important program factors include the interview experience (48.9%, 95% CI = 46% to 52%), personal experience with the residents (48.5%, 95% CI = 45% to 52%), and academic reputation (44.9%, 95% CI = 42% to 48%). Unlike location, individual program factors are often either directly or somewhat under the control of the program leadership. Several other factors were ranked as the most important factor a disproportionate number of times, including a rotation in that emergency department (ED), orientation (academic vs. community), and duration of training (3-year vs. 4-year programs). For a subset of applicants, these factors had particular importance in overall decision-making. CONCLUSIONS: The vast majority of applicants to EM residency programs employed a balance of geographic location factors with individual program factors in selecting a residency program. Specific program characteristics represent the greatest opportunity to maximize the success of the immediate interview experience/season, while others provide potential for strategic planning over time. A working knowledge of these results empowers program directors to make informed decisions while providing an appreciation for the limitations in attracting applicants.

Dealing with the difficult student in emergency medicine.

Dealing with a student who is perceived as difficult to work with or teach is inevitable in any academic physician's career. This paper will outline the basic categories of these difficulties pertinent to Emergency Medicine rotations in order to facilitate appropriate identification of problems. Strategies for evaluation and reporting of the difficult student are presented. Remediation, based on the type of difficulty, is addressed. Timeliness of reporting, evaluation, and feedback are invaluable to allow for appropriate assessment of the outcome of the remediation plan.

Developing expert medical teams: toward an evidence-based approach.

Current health care literature cites communication breakdown and teamwork failures as primary threats to patient safety. The unique, dynamic environment of the emergency department (ED) and the complexity of patient care necessitate the development of strong interdisciplinary team skills among emergency personnel. As part of the 2008 Academic Emergency Medicine Consensus Conference on "The Science of Simulation in Healthcare," our workshop group identified key theory and evidence-based recommendations for the design and implementation of team training programs. The authors then conducted an extensive review of the team training literature within the domains of organizational psychology, aviation, military, management, and health care. This review, in combination with the workshop session, formed the basis for recommendations and need for further research in six key areas: 1) developing and refining core competencies for emergency medicine (EM) teams; 2) leadership training for emergency physicians (EPs); 3) conducting comprehensive needs analyses at the organizational, personnel, and task levels; 4) development of training platforms to maximize knowledge transfer; 5) debriefing and provision of feedback; and 6) proper implementation of simulation technology. The authors believe that these six areas should form an EM team training research platform to advance the EM literature, while leveraging the unique team structures present in EM to expand team training theory and research.

Simulator training improves fiber-optic intubation proficiency among emergency medicine residents.

OBJECTIVES: The objective was to observe how a workshop using a virtual reality bronchoscopy simulator and computer-based tutorial affects emergency medicine (EM) resident skill in fiber-optic intubation. METHODS: In this observational before-and-after study, EM resident performance on three simulated pediatric difficult airway cases was observed before and after a short computer-based tutorial and 10 minutes of self-directed practice. The primary outcome was the total time required to place the endotracheal tube (ETT), secondary outcomes included the number of endoscope collisions with mucosa, and a calculated efficiency score measuring the proportion of time participants spent looking at correct central airway structures. Nonparametric Wilcoxon signed rank tests compared performance on the first versus the repeat attempt for each of the three simulated cases. Participants were surveyed regarding their assessments of the experience. RESULTS: Significant decreases in median procedure times and number of scope collisions and increases in median efficiency scores were seen for Cases 1 and 2. Case 3 showed no significant changes in outcomes between first and repeat attempts. Participants positively assessed the training and felt that its use would improve clinical practice. CONCLUSIONS: Participation in a simulation-based fiber-optic intubation skill workshop can improve fiber-optic intubation performance rapidly among EM residents. Future research should evaluate if this enhanced performance translates to improved clinical performance in the emergency department (ED).