Survey of Current Simulation Based Training in the US Military Health System.

INTRODUCTION: Simulation-based medical training has been shown to be effective and is widely used in civilian hospitals; however, it is unclear how widely and how effectively simulation is utilized in the U.S. Military Health System (MHS). The current operational state of medical simulation in the MHS is unknown, and there remains a need for a system-wide assessment of whether and how the advances in simulation-based medical training are employed to meet the evolving needs of the present-day warfighter. Understanding the types of skills and methods used within simulation programs across the enterprise is important data for leaders as they plan for the future in terms of curriculum development and the investment of resources. The aim of the present study is to survey MHS simulation programs in order to determine the prevalence of skills taught, the types of learners served, and the most common methodologies employed in this worldwide health care system. MATERIALS AND METHODS: A cross-sectional survey of simulation activities was distributed to the medical directors of all 93 simulation programs in the MHS. The survey was developed by the authors based on lists of critical wartime skills published by the medical departments of the US Army, Navy, and Air Force. Respondents were asked to indicate the types of learners trained at their program, which of the 82 unique skills included in the survey are trained at their site, and for each skill the modalities of simulation used, i.e., mannequin, standardized patients, part task trainers, augmented/virtual reality tools, or cadaver/live tissue. RESULTS: Complete survey responses were obtained from 75 of the 93 (80%) MHS medical simulation training programs. Across all skills included in the survey, those most commonly taught belonged predominantly to the categories of medic skills and nursing skills. Across all sites, the most common category of learner was the medic/corpsman (95% of sites), followed by nurses (87%), physicians (83%), non-medical combat lifesavers (59%), and others (28%) that included on-base first responders, law enforcement, fire fighters, and civilians. The skills training offered by programs included most commonly the tasks associated with medics/corpsmen (97%) followed by nursing (81%), advanced provider (77%), and General Medical Officer (GMO) skills (47%). CONCLUSION: The survey demonstrated that the most common skills taught were all related to point of injury combat casualty care and addressed the most common causes of death on the battlefield. The availability of training in medic skills, nursing skills, and advanced provider skills were similar in small, medium, and large programs. However, medium and small programs were less likely to deliver training for advanced providers and GMOs compared to larger programs. Overall, this study found that simulation-based medical training in the MHS is focused on medic and nursing skills, and that large programs are more likely to offer training for advanced providers and GMOs. Potential gaps in the availability of existing training are identified as over 50% of skills included in the nursing, advanced provider, and GMO skill categories are not covered by at least 80% of sites serving those learners.

Assessing the Ability of a Large Language Model to Score Free-Text Medical Student Clinical Notes: Quantitative Study.

Background: Teaching medical students the skills required to acquire, interpret, apply, and communicate clinical information is an integral part of medical education. A crucial aspect of this process involves providing students with feedback regarding the quality of their free-text clinical notes. Objective: The goal of this study was to assess the ability of ChatGPT 3.5, a large language model, to score medical students' free-text history and physical notes. Methods: This is a single-institution, retrospective study. Standardized patients learned a prespecified clinical case and, acting as the patient, interacted with medical students. Each student wrote a free-text history and physical note of their interaction. The students' notes were scored independently by the standardized patients and ChatGPT using a prespecified scoring rubric that consisted of 85 case elements. The measure of accuracy was percent correct. Results: The study population consisted of 168 first-year medical students. There was a total of 14,280 scores. The ChatGPT incorrect scoring rate was 1.0%, and the standardized patient incorrect scoring rate was 7.2%. The ChatGPT error rate was 86%, lower than the standardized patient error rate. The ChatGPT mean incorrect scoring rate of 12 (SD 11) was significantly lower than the standardized patient mean incorrect scoring rate of 85 (SD 74; P=.002). Conclusions: ChatGPT demonstrated a significantly lower error rate compared to standardized patients. This is the first study to assess the ability of a generative pretrained transformer (GPT) program to score medical students' standardized patient-based free-text clinical notes. It is expected that, in the near future, large language models will provide real-time feedback to practicing physicians regarding their free-text notes. GPT artificial intelligence programs represent an important advance in medical education and medical practice.

Using an Innovative Model to Improve Performance of the Infant Hip Examination.

OBJECTIVES: Determine if a new teaching bundle targeting developmental dysplasia of the hip screening improved interns' examination skills across multiple pediatric residency programs. METHODS: This multicenter prospective cohort study included interns across 6 pediatric residency sites within the Academic Pediatric Association's Better Outcomes through Research for Newborns Network in 2022. Participants underwent a baseline hip examination assessment on models using a checklist derived from textbook descriptions of Galeazzi, Ortolani, and Barlow maneuvers before receiving a teaching bundle. Repeat testing occurred after instruction. Data were analyzed using t-test for continuous and χ2 test for categorical variables. Semistructured focus groups provided qualitative feedback regarding the teaching bundle. RESULTS: We enrolled 117 of 155 interns across 6 sites (76%) for participation in the teaching bundle. Only 2% of participants (n = 2) identified a positive Galeazzi sign at baseline, whereas 88% (n = 103, P < .001) did so on the postinstructional assessment. Although 27% of participants (n = 32) correctly identified a positive Barlow sign at baseline, 69% (n = 81, P < .001) did so on the postinstructional assessment. The ability to correctly detect a positive Ortolani sign increased from 22% (n = 26) to 92% (n = 108, P < .001). Fifteen interns participated in the semistructured focus groups, with resultant themes reinforcing the limited experience of the infant hip examination before this intervention and the positive impact of the teaching bundle. CONCLUSIONS: Most participants in this study did not have strong infant hip examination skills at entry into residency. A standardized teaching bundle significantly improved skills in examination technique and identifying abnormalities.

I-PASS Mentored Implementation Handoff Curriculum: Frontline Provider Training Materials.

Introduction: The I-PASS Handoff Program is a comprehensive handoff curriculum that has been shown to decrease rates of medical errors and adverse events during patient handoffs. Frontline providers are the key individuals participating in handoffs of patient care. It is important they receive robust handoff training. Methods: The I-PASS Mentored Implementation Handoff Curriculum frontline provider training materials were created as part of the original I-PASS Study and adapted for the Society of Hospital Medicine (SHM) I-PASS Mentored Implementation Program. The adapted materials embrace a flipped classroom approach with an emphasis on adult learning theory principles. The training includes an overview of I-PASS handoff techniques, TeamSTEPPS team communication strategies, verbal handoff simulation scenarios, and a printed handoff document exercise. Results: As part of the SHM I-PASS Mentored Implementation Program, 2,735 frontline providers were trained at 32 study sites (16 adult and 16 pediatric) across North America. At the end of their training, 1,762 frontline providers completed the workshop evaluation form (64% response rate). After receiving the training, over 90% agreed/strongly agreed that they were able to distinguish a good- from a poor-quality handoff, articulate the elements of the I-PASS mnemonic, construct a high-quality patient summary, advocate for an appropriate environment for handoffs, and participate in handoff simulations. Universally, the training provided them with knowledge and skills relevant to their patient care activities. Discussion: The I-PASS frontline training materials were rated highly by those trained and are an integral part of a successful I-PASS Handoff Program implementation.

Rapid Application Temporary Tattoos for Medical Moulage: From Development to Testing and Commercialization.

Military medical education and training often utilize theatrical makeup, or moulage, to simulate injuries or pathologies. Traditional methods of moulage application are incredibly realistic when expertly applied. However, moulage can be expensive in terms of supplies, manpower, and time. We proposed that by creating a library of illustrations for use with temporary tattoos, the overall cost of moulage could decrease with little to no impact on training goals and objectives. The development, detailed testing and plans for commercialization are outlined herein.

A Focus on Feedback: Improving Learner Engagement and Faculty Delivery of Feedback in Hospital Medicine.

Feedback is an integral part of medical education. However, there is great variation of training and effectiveness of feedback delivery, especially in the inpatient setting. The unique learning environment provided in hospital medicine allows teachers the opportunity to provide feedback on learner performance under several longitudinal observations in areas such as direct patient care, procedural tasks, and interdisciplinary team leadership skills. Most important, feedback should occur on more than one occasion to truly empower change in knowledge, attitude, and skills. This article aims to provide the reader with foundational theories on feedback and strategies to use best practices for delivery.

I-PASS Mentored Implementation Handoff Curriculum: Champion Training Materials.

Introduction: The I-PASS Handoff Program is a comprehensive handoff curriculum that has been shown to decrease rates of medical errors and adverse events during patient handoffs. I-PASS champions are a critical part of the implementation and sustainment of this curriculum, and therefore, a rigorous program to support their training is necessary. Methods: The I-PASS Handoff champion training materials were created for the original I-PASS Study and adapted for the Society of Hospital Medicine (SHM) I-PASS Mentored Implementation Program. The adapted materials embrace a flipped classroom approach and adult learning theory. The training includes an overview of I-PASS handoff techniques, an opportunity to practice evaluating handoffs with the I-PASS observation tools using a handoff video vignette, and other key implementation principles. Results: As part of the SHM I-PASS Mentored Implementation Program, 366 champions were trained at 32 sites across North America and participated in a total of 3,491 handoff observations. A total of 346 champions completed the I-PASS Champion Workshop evaluation form at the end of their training (response rate: 94.5%). After receiving the training, over 90% agreed/strongly agreed that it provided them with knowledge or skills critical to their patient care activities and that they were able to distinguish the difference between high- and poor-quality handoffs, competently use the I-PASS handoff assessment tools, and articulate the importance of handoff observations. Conclusion: The I-PASS champion training materials were rated highly by those trained and are an integral part of a successful I-PASS Handoff Program implementation.

Patient safety after implementation of a coproduced family centered communication programme: multicenter before and after intervention study

OBJECTIVE: To determine whether medical errors, family experience, and communication processes improved after implementation of an intervention to standardize the structure of healthcare provider-family communication on family centered rounds. DESIGN: Prospective, multicenter before and after intervention study. SETTING: Pediatric inpatient units in seven North American hospitals, 17 December 2014 to 3 January 2017. PARTICIPANTS: All patients admitted to study units (3106 admissions, 13171 patient days); 2148 parents or caregivers, 435 nurses, 203 medical students, and 586 residents. INTERVENTION: Families, nurses, and physicians coproduced an intervention to standardize healthcare provider-family communication on ward rounds ("family centered rounds"), which included structured, high reliability communication on bedside rounds emphasizing health literacy, family engagement, and bidirectional communication; structured, written real-time summaries of rounds; a formal training programme for healthcare providers; and strategies to support teamwork, implementation, and process improvement. MAIN OUTCOME MEASURES: Medical errors (primary outcome), including harmful errors (preventable adverse events) and non-harmful errors, modeled using Poisson regression and generalized estimating equations clustered by site; family experience; and communication processes (eg, family engagement on rounds). Errors were measured via an established systematic surveillance methodology including family safety reporting. RESULTS: The overall rate of medical errors (per 1000 patient days) was unchanged (41.2 (95% confidence interval 31.2 to 54.5) pre-intervention v 35.8 (26.9 to 47.7) post-intervention, P=0.21), but harmful errors (preventable adverse events) decreased by 37.9% (20.7 (15.3 to 28.1) v 12.9 (8.9 to 18.6), P=0.01) post-intervention. Non-preventable adverse events also decreased (12.6 (8.9 to 17.9) v 5.2 (3.1 to 8.8), P=0.003). Top box (eg, "excellent") ratings for six of 25 components of family reported experience improved; none worsened. Family centered rounds occurred more frequently (72.2% (53.5% to 85.4%) v 82.8% (64.9% to 92.6%), P=0.02). Family engagement 55.6% (32.9% to 76.2%) v 66.7% (43.0% to 84.1%), P=0.04) and nurse engagement (20.4% (7.0% to 46.6%) v 35.5% (17.0% to 59.6%), P=0.03) on rounds improved. Families expressing concerns at the start of rounds (18.2% (5.6% to 45.3%) v 37.7% (17.6% to 63.3%), P=0.03) and reading back plans (4.7% (0.7% to 25.2%) v 26.5% (12.7% to 7.3%), P=0.02) increased. Trainee teaching and the duration of rounds did not change significantly. CONCLUSIONS: Although overall errors were unchanged, harmful medical errors decreased and family experience and communication processes improved after implementation of a structured communication intervention for family centered rounds coproduced by families, nurses, and physicians. Family centered care processes may improve safety and quality of care without negatively impacting teaching or duration of rounds. TRIAL REGISTRATION: ClinicalTrials.gov NCT02320175.

Validity Evidence for a Serious Game to Assess Performance on Critical Pediatric Emergency Medicine Scenarios.

INTRODUCTION: We developed a first-person serious game, PediatricSim, to teach and assess performances on seven critical pediatric scenarios (anaphylaxis, bronchiolitis, diabetic ketoacidosis, respiratory failure, seizure, septic shock, and supraventricular tachycardia). In the game, players are placed in the role of a code leader and direct patient management by selecting from various assessment and treatment options. The objective of this study was to obtain supportive validity evidence for the PediatricSim game scores. METHODS: Game content was developed by 11 subject matter experts and followed the American Heart Association's 2011 Pediatric Advanced Life Support Provider Manual and other authoritative references. Sixty subjects with three different levels of experience were enrolled to play the game. Before game play, subjects completed a 40-item written pretest of knowledge. Game scores were compared between subject groups using scoring rubrics developed for the scenarios. Validity evidence was established and interpreted according to Messick's framework. RESULTS: Content validity was supported by a game development process that involved expert experience, focused literature review, and pilot testing. Subjects rated the game favorably for engagement, realism, and educational value. Interrater agreement on game scoring was excellent (intraclass correlation coefficient = 0.91, 95% confidence interval = 0.89-0.9). Game scores were higher for attendings followed by residents then medical students (Pc < 0.01) with large effect sizes (1.6-4.4) for each comparison. There was a very strong, positive correlation between game and written test scores (r = 0.84, P < 0.01). CONCLUSIONS: These findings contribute validity evidence for PediatricSim game scores to assess knowledge of pediatric emergency medicine resuscitation.

The Creation of Standard-Setting Videos to Support Faculty Observations of Learner Performance and Entrustment Decisions.

Entrustable professional activities (EPAs) provide a framework to standardize medical education outcomes and advance competency-based assessment. Direct observation of performance plays a central role in entrustment decisions; however, data obtained from these observations are often insufficient to draw valid high-stakes conclusions. One approach to enhancing the reliability and validity of these assessments is to create videos that establish performance standards to train faculty observers. Little is known about how to create videos that can serve as standards for assessment of EPAs.The authors report their experience developing videos that represent five levels of performance for an EPA for patient handoffs. The authors describe a process that begins with mapping the EPA to the critical competencies needed to make an entrustment decision. Each competency is then defined by five milestones (behavioral descriptors of performance at five advancing levels). Integration of the milestones at each level across competencies enabled the creation of clinical vignettes that were converted into video scripts and ultimately videos. Each video represented a performance standard from novice to expert. The process included multiple assessments by experts to guide iterative improvements, provide evidence of content validity, and ensure that the authors successfully translated behavioral descriptions and vignettes into videos that represented the intended performance level for a learner. The steps outlined are generalizable to other EPAs, serving as a guide for others to develop videos to train faculty. This process provides the level of content validity evidence necessary to support using videos as standards for high-stakes entrustment decisions.

Simulation-based medical education in pediatrics.

The use of simulation-based medical education (SBME) in pediatrics has grown rapidly over the past 2 decades and is expected to continue to grow. Similar to other instructional formats used in medical education, SBME is an instructional methodology that facilitates learning. Successful use of SBME in pediatrics requires attention to basic educational principles, including the incorporation of clear learning objectives. To facilitate learning during simulation the psychological safety of the participants must be ensured, and when done correctly, SBME is a powerful tool to enhance patient safety in pediatrics. Here we provide an overview of SBME in pediatrics and review key topics in the field. We first review the tools of the trade and examine various types of simulators used in pediatric SBME, including human patient simulators, task trainers, standardized patients, and virtual reality simulation. Then we explore several uses of simulation that have been shown to lead to effective learning, including curriculum integration, feedback and debriefing, deliberate practice, mastery learning, and range of difficulty and clinical variation. Examples of how these practices have been successfully used in pediatrics are provided. Finally, we discuss the future of pediatric SBME. As a community, pediatric simulation educators and researchers have been a leading force in the advancement of simulation in medicine. As the use of SBME in pediatrics expands, we hope this perspective will serve as a guide for those interested in improving the state of pediatric SBME.

Changes in medical errors after implementation of a handoff program.

BACKGROUND: Miscommunications are a leading cause of serious medical errors. Data from multicenter studies assessing programs designed to improve handoff of information about patient care are lacking. METHODS: We conducted a prospective intervention study of a resident handoff-improvement program in nine hospitals, measuring rates of medical errors, preventable adverse events, and miscommunications, as well as resident workflow. The intervention included a mnemonic to standardize oral and written handoffs, handoff and communication training, a faculty development and observation program, and a sustainability campaign. Error rates were measured through active surveillance. Handoffs were assessed by means of evaluation of printed handoff documents and audio recordings. Workflow was assessed through time-motion observations. The primary outcome had two components: medical errors and preventable adverse events. RESULTS: In 10,740 patient admissions, the medical-error rate decreased by 23% from the preintervention period to the postintervention period (24.5 vs. 18.8 per 100 admissions, P<0.001), and the rate of preventable adverse events decreased by 30% (4.7 vs. 3.3 events per 100 admissions, P<0.001). The rate of nonpreventable adverse events did not change significantly (3.0 and 2.8 events per 100 admissions, P=0.79). Site-level analyses showed significant error reductions at six of nine sites. Across sites, significant increases were observed in the inclusion of all prespecified key elements in written documents and oral communication during handoff (nine written and five oral elements; P<0.001 for all 14 comparisons). There were no significant changes from the preintervention period to the postintervention period in the duration of oral handoffs (2.4 and 2.5 minutes per patient, respectively; P=0.55) or in resident workflow, including patient-family contact and computer time. CONCLUSIONS: Implementation of the handoff program was associated with reductions in medical errors and in preventable adverse events and with improvements in communication, without a negative effect on workflow. (Funded by the Office of the Assistant Secretary for Planning and Evaluation, U.S. Department of Health and Human Services, and others.).

Introduction of a direct observation program into a pediatric resident continuity clinic: feasibility, acceptability, and effect on resident feedback.

BACKGROUND: Direct observations of trainee performance are essential to the evaluation of clinical skills, and are now required by the Accreditation Council on Graduate Medical Education (ACGME). PURPOSE: We sought to describe the feasibility and acceptability of a direct observation program in a pediatric resident clinic, and evaluate its effect on feedback. METHODS: We modified the Structured Clinical Observation (SCO) for use in a pediatric resident clinic. Preceptors and residents were asked about the frequency and content of feedback given before and after the introduction of the observations. RESULTS: One-hundred-sixty-six structured clinical observations were performed. Their use increased the frequency of feedback provided, and resulted in more feedback on listening skills and less feedback on medical knowledge and decision making. CONCLUSIONS: Structured clinical observations can be successfully introduced into a pediatric resident continuity clinic. This is a valuable supplement to the traditional precepting, and fulfills the mandate to include direct observations in the assessment of residents' clinical competence.

Teaching breaking bad news using mixed reality simulation.

BACKGROUND: Our novel teaching approach involved having students actively participate in an unsuccessful resuscitation of a high fidelity human patient simulator with a gun shot wound to the chest, followed immediately by breaking bad news (BBN) to a standardized patient wife (SPW) portrayed by an actress. METHODS: Brief education interventions to include viewing a brief video on the SPIKES protocol on how to break bad news, a didactic lecture plus a demonstration, or both, was compared to no pretraining by dividing 553 students into four groups prior to their BBN to the SPW. The students then self-assessed their abilities, and were also evaluated by the SPW on 21 items related to appearance, communication skills, and emotional affect. All received cross-over training. RESULTS: Groups were equal in prior training (2 h) and belief that this was an important skill to be learned. Students rated the experience highly, and demonstrated marked improvement of self-assessed skills over baseline, which was maintained for the duration of the 12-wk clerkship. Additionally, students who received any of the above training prior to BBN were rated superior to those who had no training on several communication skills, and the observation of the video seemed to offer the most efficient way of teaching this skill in a time delimited curriculum. CONCLUSION: This novel approach was well received and resulted in improvement over baseline. Lessons learned from this study have enhanced our curricular approach to this vital component of medical education.

Cooperative sequence modules determine replication initiation sites at the human beta-globin locus.

The human beta globin locus contains two adjacent replicators, each capable of initiating DNA replication when transferred from its native locus to ectopic sites. Here, we report a detailed analysis of the sequence requirements for replication initiation from these replicators. In both replicators, initiation required a combination of an asymmetric purine:pyrimidine sequence and several AT-rich stretches. Modules from the two replicators could combine to initiate replication. AT-rich sequences were essential for replicator activity: a low frequency of initiation was observed in DNA fragments that included a short stretch of AT-rich sequences, whereas inclusion of additional AT-rich stretches increased initiation efficiency. By contrast, replication initiated at a low level without the asymmetric purine:pyrimidine modules but they were required in synergy to achieve efficient initiation. These data support a combinatorial model for replicator activity and suggest that the initiation of DNA replication requires interaction between at least two distinct sequence modules.