Development and evaluation of a novel knowledge assessment tool for pediatric emergency medicine clerkships.

Objectives: This study seeks to determine validity evidence for a newly developed multiple-choice examination (MCE) tool to assess retention and application of medical knowledge of students enrolled in a pediatric emergency medicine (PEM) clerkship. Methods: A team of PEM physicians created a 110-item MCE covering the range of clinical topics in PEM relevant for medical students. The researchers determined examination content using the report of Clerkship Directors in Emergency Medicine and PEM Interest Group of the Society for Academic Emergency Medicine (SAEM). The authors administered the MCE to fourth-year medical students at the end of their PEM rotation from May 2020 to April 2023 at four institutions and then analyzed the examination using four of Messick's five sources of validity evidence: content, response process, internal structure, and relation to other variables. Results: A total of 158 students took the test. In academic year (AY)20-21, 47 students took the test and scored, on average, 81%. After revision of poor and indeterminate questions, the 111 medical students who took the revised version of the test in AY21-AY23 scored on average 77.3% with a standard deviation of 5.7% with a normal distribution in scores. The revised questions were rated as excellent (10.0%), good (26.4%), fair (34.5%), poor (24.5%), or indeterminate (4.5%) based on test item discrimination. There was a positive correlation between MCE scores and students' clinical evaluations but no correlation between MCE scores and scores that students received on their clinical notes or patient presentations during case conference. Conclusions: This novel PEM clerkship examination is a reliable test of medical knowledge. Future directions involve evaluating consequences of the MCE and offering the test to medical students in a dedicated PEM rotation at the national level.

Factors Associated With Improved Pediatric Resuscitative Care in General Emergency Departments.

OBJECTIVES: To describe the quality of pediatric resuscitative care in general emergency departments (GEDs) and to determine hospital-level factors associated with higher quality. METHODS: Prospective observational study of resuscitative care provided to 3 in situ simulated patients (infant seizure, infant sepsis, and child cardiac arrest) by interprofessional GED teams. A composite quality score (CQS) was measured and the association of this score with modifiable and nonmodifiable hospital-level factors was explored. RESULTS: A median CQS of 62.8 of 100 (interquartile range 50.5-71.1) was noted for 287 resuscitation teams from 175 emergency departments. In the unadjusted analyses, a higher score was associated with the modifiable factor of an affiliation with a pediatric academic medical center (PAMC) and the nonmodifiable factors of higher pediatric volume and location in the Northeast and Midwest. In the adjusted analyses, a higher CQS was associated with modifiable factors of an affiliation with a PAMC and the designation of both a nurse and physician pediatric emergency care coordinator, and nonmodifiable factors of higher pediatric volume and location in the Northeast and Midwest. A weak correlation was noted between quality and pediatric readiness scores. CONCLUSIONS: A low quality of pediatric resuscitative care, measured using simulation, was noted across a cohort of GEDs. Hospital factors associated with higher quality included: an affiliation with a PAMC, designation of a pediatric emergency care coordinator, higher pediatric volume, and geographic location. A weak correlation was noted between quality and pediatric readiness scores.

Development of a Physician Assistant Orientation Program in a Pediatric Emergency Department.

INTRODUCTION: The role of physician assistants (PAs) in the care of emergency department (ED) patients has been expanding in recent years. However, little is known about how PAs are prepared to practice in pediatric emergency medicine (PEM), and there is no published literature on how to train PAs in independently managing low-acuity visits in the pediatric emergency department (PED). METHODS: We created a preorientation, orientation, and postorientation program for PAs who are onboarding in the PED at a large, free-standing pediatric acute care hospital. We implemented an evaluation system that assessed readiness for independent practice based on number and type of patients seen as well as supervising physicians' feedback. RESULTS: On average, PAs took care of 877 patients over the course of their first year of employment at the hospital before achieving readiness for independent practice. Most PAs were deemed ready to see low-acuity patients without direct supervision by PEM attendings during their third or fourth quarter of employment. DISCUSSION: The successful implementation of a 12-month curriculum and individualized feedback allowed our PED to prepare PAs for independent management of the low-acuity PED patient.

Rapid Development and Testing of a COVID-19 Vaccine Curriculum for Pediatricians.

BACKGROUND AND OBJECTIVES: As the coronavirus disease 2019 (COVID-19) pandemic evolves and vaccines become available to children, pediatricians must navigate vaccination discussions in the setting of rapidly changing vaccine recommendations and approvals. We developed and evaluated an educational curriculum for pediatricians to improve their knowledge about COVID-19 vaccines and confidence in communicating with patients and families about COVID-19 vaccines. METHODS: Five institutions collaborated to develop an online educational curriculum. Utilizing the collaboration's multidisciplinary expertise, we developed a 3-module curriculum focused on the SARS-CoV-2 virus and vaccine basics, logistics and administration of COVID-19 vaccine, and COVID-19 vaccine communication principles. Surveys administered to clinician participants before and after completion of the curriculum assessed knowledge and confidence; a follow-up survey 1 month after the post-survey assessed persistence of initial findings. RESULTS: A total of 152 pediatric providers participated; 72 completed both pre- and post-surveys. The median knowledge score improved from the pre-survey to the post-survey (79%-93%, P < .001). There was an increase in providers' confidence after completing the curriculum, which persisted in the follow-up survey. In the post-survey, 98% of participants had had the opportunity to discuss the COVID-19 vaccine with patients, and most clinicians reported that the modules decreased apprehension some or significantly. CONCLUSIONS: This project demonstrates rapid and feasible deployment of a curriculum providing up-to-date information to front-line clinicians responsible for having complex conversations about COVID-19 vaccine decision-making. Clinicians who completed this curriculum had sustained increased confidence and decreased levels of apprehension when discussing the COVID-19 vaccine.

Developing and Integrating Asynchronous Web-Based Cases for Discussing and Learning Clinical Reasoning: Repeated Cross-sectional Study.

BACKGROUND: Trainees rely on clinical experience to learn clinical reasoning in pediatric emergency medicine (PEM). Outside of clinical experience, graduate medical education provides a handful of explicit activities focused on developing skills in clinical reasoning. OBJECTIVE: In this paper, we describe the development, use, and changing perceptions of a web-based asynchronous tool to facilitate clinical reasoning discussion for PEM providers. METHODS: We created a case-based web-based discussion tool for PEM clinicians and fellows to post and discuss cases. We examined website analytics for site use and collected user survey data over a 3-year period to assess the use and acceptability of the tool. RESULTS: The learning tool had more than 30,000 site visits and 172 case comments for the 55 published cases over 3 years. Self-reported engagement with the learning tool varied inversely with clinical experience in PEM. The tool was relevant to clinical practice and useful for learning PEM for most respondents. The most experienced clinicians were more likely than fellows to report posting commentary, although absolute rate of commentary was low. CONCLUSIONS: An asynchronous method of case presentation and web-based commentary may present an acceptable way to supplement clinical experience and traditional education methods for sharing clinical reasoning.

Evaluation of Communication and Safety Behaviors During Hospital-Wide Code Response Simulation.

INTRODUCTION: To understand the baseline quality of team communication behaviors at our organization, we implemented institution-wide simulation training and measured the performance of safety behaviors of ad hoc teams in emergent situations. METHODS: Clinicians participated in 2 interprofessional video-recorded simulation scenarios, each followed by debriefing. Using a standardized evaluation instrument, 2 reviewers independently evaluated the presence or absence of desired team safety behaviors, including escalating care, sharing a mental model, establishing leadership, thinking out loud, and identifying roles and responsibilities. We also scored the quality of sharing the mental model, closed-loop communication, and overall team performance on a 7-point scale. Discordant reviews were resolved with scoring by an additional reviewer. RESULTS: A total of 1404 clinicians participated in 398 simulation scenarios, resulting in 257 usable videos. Overall, teams exhibited desired behaviors at the following frequencies: escalating care, 85%; sharing mental models, 66%; verbally establishing leadership, 6%; thinking out loud, 87%; and identifying roles and responsibilities, 27%. Across all reviews, the quality of the graded behaviors (of 7 points) was 2.8 for shared mental models, 3.3 for closed-loop communication, and 3.2 for overall team performance. CONCLUSIONS: In a simulation setting with ad hoc teams, there was variable performance on completing safety behaviors and only a fair quality of graded communication behaviors. These results establish a baseline assessment of communication and teamwork behaviors and will guide future quality improvement interventions.

Preparedness for Pediatric Office Emergencies: A Multicenter, Simulation-Based Study.

OBJECTIVES: Pediatric emergencies can occur in pediatric primary care offices. However, few studies have measured emergency preparedness, or the processes of emergency care, provided in the pediatric office setting. In this study, we aimed to measure emergency preparedness and care in a national cohort of pediatric offices. METHODS: This was a multicenter study conducted over 15 months. Emergency preparedness scores were calculated as a percentage adherence to 2 checklists on the basis of the American Academy of Pediatrics guidelines (essential equipment and supplies and policies and protocols checklists). To measure the quality of emergency care, we recruited office teams for simulation sessions consisting of 2 patients: a child with respiratory distress and a child with a seizure. An unweighted percentage of adherence to checklists for each case was calculated. RESULTS: Forty-eight teams from 42 offices across 9 states participated. The mean emergency preparedness score was 74.7% (SD: 12.9). The mean essential equipment and supplies subscore was 82.2% (SD: 15.1), and the mean policies and protocols subscore was 57.1% (SD: 25.6). Multivariable analyses revealed that independent practices and smaller total staff size were associated with lower preparedness. The median asthma case performance score was 63.6% (interquartile range: 43.2-81.2), whereas the median seizure case score was 69.2% (interquartile range: 46.2-80.8). Offices that had a standardized process of contacting emergency medical services (EMS) had a higher rate of activating EMS during the simulations. CONCLUSIONS: Pediatric office preparedness remains suboptimal in a multicenter cohort, especially in smaller, independent practices. Academic and community partnerships using simulation can help address gaps and implement important processes like contacting EMS.

Code Response Training: Improving Interprofessional Communication.

Introduction: Using simulation to improve team performance in emergencies is commonplace. Decreasing codes hospital-wide can be challenging. To address these needs, hospital leaders requested a simulation program to provide team training across an institution focused on patient safety and communication techniques. Methods: We developed a multimodal approach pairing three online modules on communication techniques with a simulation-based learning session. The three modules required 1 hour, followed by a 1-hour, in-person, simulation-based, interprofessional, small-group session of clinical staff. In ad hoc teams, participants managed two cases: a toddler with airway obstruction and a child developing septic shock. A focused debriefing included discussion of mental models, team formation and expertise, and communication techniques to create a common language to use in ad hoc team formation and patient care. Results: Through more than 200 training sessions reaching over 1,400 staff members, we executed code response training. A nurse and physician facilitated each session, emphasizing the interprofessional nature needed for patient care. Participants rated the learning experience highly on a 5-point Likert scale (1 = low/poor, 5 = high/excellent), with an average rating of 4.3 for achieving objectives and an average rating of 4.8 for facilitator effectiveness. Discussion: Through engaging leadership and frontline clinicians, the simulation program provided code response training hospital-wide, emphasizing the importance of teamwork and communication in critical situations. Such hospital-wide training can emphasize a shared language to empower clinicians at all levels to deliver safe, quality patient care.

Simulation-based emergency medicine education in the era of physical distancing.

Background: The COVID-19 pandemic posed significant challenges to traditional simulation education. Because simulation is considered best practice for competency-based education, emergency medicine (EM) residencies adapted and innovated to accommodate to the new pandemic normal. Our objectives were to identify the impact of the pandemic on EM residency simulation training, to identify unique simulation adaptations and innovations implemented during the pandemic, and to analyze successes and failures through existing educational frameworks to offer guidance on the use of simulation in the COVID-19 era. Methods: The Society for Academic Emergency Medicine (SAEM)'s Simulation Academy formed the SimCOVID task force to examine the impact of COVID-19 on simulation didactics. A mixed-methods approach was employed. A literature search was conducted on the subject and used to develop an exploratory survey that was distributed on the Simulation Academy Listserv. The results were subjected to thematic analysis and examined through existing educational frameworks to better understand successes and failures and then used to generate suggestions on the use of simulation in the COVID-19 era. Results: Thirty programs responded to the survey. Strategies reported included adaptations to virtual teleconferencing and small-group in situ training with a focus on procedural training and COVID-19 preparedness. Successful continuation or relaunching of simulation programs was predicated on several factors including willingness for curricular pivots through rapid iterative prototyping, embracing teleconferencing software, technical know-how, and organizational and human capacity. In specific instances the use of in situ simulation for COVID-19 preparedness established the view of simulation as a "value add" to the organization. Conclusions: Whereas simulation educator's responses to the COVID-19 pandemic can be better appreciated through the lens of iterative curricular prototyping, their successes and failures depended on existing expertise in technological, pedagogical, and content knowledge. That knowledge needed to exist and synergize within a system that had the human and organizational capacity to prioritize and invest in strategies to respond to the rapidly evolving crisis in a proactive manner. Going forward, administrators and educators will need to advocate for continued investment in human and organizational capacity to support simulation-based efforts for the evolving clinical and educational landscape.

Using an eLearning Module to Facilitate Sepsis Knowledge Acquisition Across Multiple Institutions and Learner Disciplines.

OBJECTIVE: Guidelines exist for care of pediatric sepsis, but no study has assessed the benefit of electronic learning (eLearning) in this topic area. The objective of this multicenter study was to assess knowledge acquisition and retention for pediatric sepsis across multiple health care provider roles, using an adaptive and interactive eLearning module. METHODS: The study used pretest, posttest, and 90-day delayed test scores to evaluate provider knowledge after an adaptive and interactive eLearning module intervention. The eLearning module contained conditional logic-based assessments that allowed real-time adjustments of the displayed content according to each participant's demonstrated knowledge. Physicians, nurses, and advanced practice providers, primarily emergency department based, at 9 pediatric institutions were included. Changes in test scores were stratified by provider role. RESULTS: A total of 574 participants completed the posttest, and 296 (51.6%) of those completed the delayed test. Across all providers, there was an increase in test scores of 15.7% between the pretest and posttest (P < 0.001) with a large effect size as measured by Cramer's V. Across all providers, there was an overall test score increase of 5.2% (P < 0.001) between the pretest and delayed test, with a small effect size. CONCLUSIONS: An eLearning module improved immediate and delayed pediatric sepsis knowledge in pediatric health care providers across multiple institutions and provider roles. Immediate knowledge gain was meaningful as indicated by effect sizes, although by the time of the delayed test, the effect was smaller. This module fills an important gap in currently available pediatric sepsis education.

A Modified Delphi Study to Prioritize Content for a Simulation-based Pediatric Curriculum for Emergency Medicine Residency Training Programs.

OBJECTIVES: Pediatric training is an essential component of emergency medicine (EM) residency. The heterogeneity of pediatric experiences poses a significant challenge to training programs. A national simulation curriculum can assist in providing a standardized foundation of pediatric training experience to all EM trainees. Previously, a consensus-derived set of content for a pediatric curriculum for EM was published. This study aimed to prioritize that content to establish a pediatric simulation-based curriculum for all EM residency programs. METHODS: Seventy-three participants were recruited to participate in a three-round modified Delphi project from 10 stakeholder organizations. In round 1, participants ranked 275 content items from a published set of pediatric curricular items for EM residents into one of four categories: definitely must, probably should, possibly could, or should not be taught using simulation in all residency programs. Additionally, in round 1 participants were asked to contribute additional items. These items were then added to the survey in round 2. In round 2, participants were provided the ratings of the entire panel and asked to rerank the items. Round 3 involved participants dichotomously rating the items. RESULTS: A total of 73 participants participated and 98% completed all three rounds. Round 1 resulted in 61 items rated as definitely must, 72 as probably should, 56 as possibly could, 17 as should not, and 99 new items were suggested. Round 2 resulted in 52 items rated as definitely must, 91 as probably should, 120 as possibly could, and 42 as should not. Round 3 resulted in 56 items rated as definitely must be taught using simulation in all programs. CONCLUSIONS: The completed modified Delphi process developed a consensus on 56 pediatric items that definitely must be taught using simulation in all EM residency programs (20 resuscitation, nine nonresuscitation, and 26 skills). These data will serve as a targeted needs assessment to inform the development of a standard pediatric simulation curriculum for all EM residency programs.

Using Simulation to Measure and Improve Pediatric Primary Care Offices Emergency Readiness.

INTRODUCTION: Emergencies in the pediatric primary care office are high-risk, low-frequency events that offices may be ill-prepared to manage. We developed an intervention to improve pediatric primary care office emergency preparedness involving a baseline measurement, a customized report out with action plans for improvement (based on baseline measures), and a plan to repeat measurement at 6 months. This article reports on the baseline measurement. METHODS: This baseline measurement consisted of 2 components: preparedness checklists and in situ simulations. The preparedness checklists were completed in person to measure compliance with the American Academy of Pediatrics Policy Statement: preparation for emergencies in the offices of pediatricians and pediatric primary care providers, in the domains of equipment, supplies, medication, and guidelines. Two in situ simulations, a child in respiratory distress and a child with a seizure, were conducted with the offices' interprofessional teams; performance was scored using checklists. RESULTS: Baseline measurements were conducted in 12 pediatric offices from October to December 2018. Wide variability was noted for compliance with the American Academy of Pediatrics recommendations (range = 47%-87%) and performance during in situ simulations (range = 43%-100%). CONCLUSIONS: Pediatric primary care office emergency preparedness was found to be variable. Simulation can be used to augment existing measures of emergency preparedness, such as checklists. By using simulation to measure office emergency preparedness, areas of knowledge deficit and latent safety threats were identified and are being addressed through ongoing collaboration.

Development of a Simulation-Based Interprofessional Teamwork Assessment Tool.

BACKGROUND: The Accreditation Council for Graduate Medical Education (ACGME) Milestone projects required each specialty to identify essential skills and develop means of assessment with supporting validity evidence for trainees. Several specialties rate trainees on a milestone subcompetency related to working in interprofessional teams. A tool to assess trainee competence in any role on an interprofessional team in a variety of scenarios would be valuable and suitable for simulation-based assessment. OBJECTIVE: We developed a tool for simulation settings that assesses interprofessional teamwork in trainees. METHODS: In 2015, existing tools that assess teamwork or interprofessionalism using direct observation were systematically reviewed for appropriateness, generalizability, adaptability, ease of use, and resources required. Items from these tools were included in a Delphi method with multidisciplinary pediatrics experts using an iterative process from June 2016 to January 2017 to develop an assessment tool. RESULTS: Thirty-one unique tools were identified. A 2-stage review narrowed this list to 5 tools, and 81 items were extracted. Twenty-two pediatrics experts participated in 4 rounds of Delphi surveys, with response rates ranging from 82% to 100%. Sixteen items reached consensus for inclusion in the final tool. A global 4-point rating scale from novice to proficient was developed. CONCLUSIONS: A novel tool to assess interprofessional teamwork for individual trainees in a simulated setting was developed using a systematic review and Delphi methodology. This is the first step to establish the validity evidence necessary to use this tool for competency-based assessment.

Promoting interdisciplinary shared mental models.

BACKGROUND: Emergency Medicine (EM) and Hospital Medicine (HM) providers frequently interact when transitioning patients from the emergency department (ED) to the inpatient unit; however, there is infrequent collaboration between these subspecialties, and effective communication in EM-HM provider handover is an area for improvement. Shared mental models can enhance communication and safety. The purpose of this article is to describe the implementation of an interdisciplinary conference to allow providers to create shared mental models, and to assess the impact on attitudes and behaviours towards communication and collaboration outside the competing attentions of patient care environments. METHODS: The authors instituted a quarterly interactive case-based conference at an academic tertiary care children's hospital in which EM and HM subspecialty trainees co-facilitate evidence-based interactive presentations of clinical, diagnostic or management dilemmas. The conferences were evaluated via repeated cross-sectional surveys of EM and HM providers, as well as session evaluations. Surveys included multiple-choice, Likert-scale and free-text responses to assess the degree of interaction and collaboration between the divisions and provider attitudes toward changes in clinical practice. RESULTS: Assessment of the EM-HM collaborative conferences demonstrated trends towards increased interaction between divisions and increased discussion about management decisions among colleagues. Two-thirds of individuals who attended at least one conference felt that having attended an interdisciplinary conference influenced future management decisions. Effective communication in EM-HM provider handover is an area for improvement CONCLUSIONS: Interdisciplinary conferences provide a forum for cross-specialty communication to discuss management differences in a low-stress educational environment, and allow providers to develop shared mental models for effective, safe patient care.

Creating Consensus Educational Goals for Pediatric Sepsis via Multicenter Modified Delphi.

Clinical care of pediatric sepsis depends on early recognition and basic initial steps and thus focused educational materials for providers in these early phases of care are needed. The authors sought to identify educational goals and establish a framework for those materials. The authors conducted a Delphi process to create consensus educational goals for the recognition and treatment of pediatric sepsis, focused on the needs of emergency and acute care providers. Experts in pediatric sepsis quality improvement and education were recruited. In each round the experts determined if a particular goal was appropriate for inclusion and provided edits if required. Free-text responses and additional goals were accepted. The primary author deidentified and collated the responses and distributed an updated list of goals prior to each round. Five experts participated in the Delphi process. After three rounds the panelists unanimously agreed on 14 educational goals. These goals include three on recognition, five on early treatment, three on treatment response, and three on continued treatment. Using a Delphi process the authors established a list of educational goals for pediatric sepsis care. The experts largely agreed upon the initial set of goals and quickly came to consensus on the majority of topics. This highlights a foundation for any future educational interventions. In addition to creating content standards for pediatric sepsis education, this Delphi process represents a useful tool that may be adaptable to educational content development in other topics.

Script Concordance Testing to Determine Infant Lumbar Puncture Practice Variation.

OBJECTIVES: Script concordance testing (SCT) is used to assess clinical decision-making. We explore the use of SCT to (1) quantify practice variations in infant lumbar puncture (LP) and (2) analyze physician's characteristics affecting LP decision making. METHODS: Using standard SCT processes, a panel of pediatric subspecialty physicians constructed 15 infant LP case vignettes, each with 2 to 4 SCT questions (a total of 47). The vignettes were distributed to pediatric attending physicians and fellows at 10 hospitals within the INSPIRE Network. We determined both raw scores (tendency to perform LP) and SCT scores (agreement with the reference panel) as well as the variation with participant factors. RESULTS: Two hundred twenty-six respondents completed all 47 SCT questions. Pediatric emergency medicine physicians tended to select LP more frequently than did general pediatricians, with pediatric emergency medicine physicians showing significantly higher raw scores (20.2 ± 10.2) than general pediatricians (13 ± 15; 95% confidence interval for difference, 1, 13). Concordance with the reference panel varied among subspecialties and by the frequency with which practitioners perform LPs in their practices. CONCLUSION: Script concordance testing questions can be used as a tool to detect subspecialty practice variation. We are able to detect significant practice variation in the self-report of use of LP for infants among different pediatric subspecialties.

Are Graduating Pediatric Residents Prepared to Perform Infant Lumbar Punctures?: A Multi-Institutional Descriptive Study.

OBJECTIVES: Few published studies describe graduating pediatric residents' procedural skills or success rates. This information would help guide supervisors' decisions about graduating residents' preparedness, training, and supervision needs. This study aimed to measure success rates for graduating pediatric residents performing infant lumbar puncture (LP) during the final months of their training and to describe their experiences performing and supervising infant LPs during the course of their training. METHODS: This survey-based study was conducted at 10 academic medical institutions in 2013. The survey consisted of 4 domains: (1) demographics, (2) exposure to infant LP training as an intern, (3) number of LPs performed and supervised during residency, and (4) specific information on the most recent clinical infant LP. RESULTS: One hundred ninety-eight (82%) of 242 eligible graduating residents responded to the survey. A 54% success rate was noted for graduating residents when they were the first provider performing the infant LPs. Success rates were 24% if they were not the first provider to attempt the LP. Overall, graduating residents were supervised on 29% of their LPs, used anesthesia for 29%, and used the early stylet removal technique for 63%. The graduating residents performed a median of 12 infant LPs and supervised others on a median of 5 infant LPs throughout their residency. The vast majority reported feeling confident and prepared to perform this procedure. CONCLUSIONS: At the end of residency, graduating pediatric residents were rarely supervised and had low infant LP success rates despite confidence in their skills. However, graduating residents frequently supervised others performing this procedure.

Essentials of Pediatric Emergency Medicine Fellowship: Part 6: Program Administration.

This article is the sixth in a 7-part series that aims to comprehensively describe the current state and future directions of pediatric emergency medicine (PEM) fellowship training from the essential requirements to considerations for successfully administering and managing a program to the careers that may be anticipated upon program completion. This article provides a broad overview of administering and supervising a PEM fellowship program. It explores 3 topics: the principles of program administration, committee management, and recommendations for minimum time allocated for PEM fellowship program directors to administer their programs.