Evaluation of urological and gynecological surgeons as force multipliers for mass Casualty trauma care.

BACKGROUND: The clinical demands of mass casualty events strain even the most well-equipped trauma centers and are especially challenging in resource-limited rural, remote, or austere environments. Gynecologists and urologists care for patients with pelvic and abdominal injuries, but the extent to which they are able to serve as "force multipliers" for trauma care is unclear. This study examined the abilities of urologists and gynecologists to perform 32 trauma procedures after mentored training by expert trauma educators to inform the potential for these specialists to independently care for trauma patients. METHODS: Urological (6), gynecological surgeons (6), senior (PGY5) general surgery residents (6), and non-trauma trained general surgeons (8) completed a rigorous trauma training program (ASSET+). All participants were assessed in their trauma knowledge and surgical abilities performing 32 trauma procedures pre/post mentored training by expert trauma surgeons. Performance benchmarks were set for knowledge (80%) and independent, accurate completion of all procedural components within a realistic time window (90%). RESULTS: General surgery participants demonstrated greater trauma knowledge than gynecologists and urologists; however, none of the specialties reached the 80% benchmark. Pre-training, general surgery and urology participants outperformed gynecologists for overall procedural abilities. Post-training, only general surgeons met the 90% benchmark. Post-hoc analysis revealed no differences between the groups performing most pelvic and abdominal procedures, however knowledge associated with decision making and judgment in the provision of trauma care was significantly below the benchmark for gynecologists and urologists, even after training. CONCLUSION: For physiologically stable patients with traumatic injuries to the abdomen, pelvis or retroperitoneum, these specialists might be able to provide appropriate care; however, they would best benefit trauma patients in the capacity of highly skilled assisting surgeons to trauma specialists. These specialists should not be considered for solo resuscitative surgical care. LEVEL OF EVIDENCE: Therapeutic/Care Management, Level III/IV.

Putting the ready in readiness: A post-hoc analysis of surgeon performance during a military MASCAL in Afghanistan.

BACKGROUND: All military surgeons must maintain trauma capabilities for expeditionary care contexts, yet most are not trauma specialists. Maintaining clinical readiness for trauma and mass casualty care is a significant challenge for military and civilian surgeons. We examined the effect of a prescribed clinical readiness program for expeditionary trauma care on the surgical performance of 12 surgeons during a 60-patient MASCAL event. METHODS: The sample included orthopaedic (4) and general surgeons (8) who cared for MASCAL victims at Hamad Karzai International Airport, Kabul, Afghanistan on 26 August 2021. One orthopaedic and two general surgeons had prior deployment experience. The prescribed program included three primary measures of clinical readiness: 1. expeditionary knowledge (exam score), 2. procedural skills competencies (performance assessment score), 3. clinical activity (operative practice profile metric). Data were attained from program records for each surgeon in the sample. Each of the 60 patient cases were reviewed and rated (performance score) by The Joint Trauma System's Performance Improvement Branch; a military-wide performance improvement organization. All scores were normalized to facilitate direct comparisons using effect size calculations between each pre-deployment measure and MASCAL surgical care. RESULTS: Pre-deployment knowledge and clinical activity measures met program benchmarks. Baseline pre-deployment procedural skills competency scores did not meet program benchmarks, however those gaps were closed through re-training, ensuring all surgeons met or exceeded the program benchmarks pre-deployment. There were very large effect sizes (Cohen's d) between all program measures and surgical care score, confirming the relationship between the program measures and MASCAL trauma care provided by the 12 surgeons. CONCLUSION: The prescribed program measures ensured all surgeons achieved pre-deployment performance benchmarks and provided high quality trauma care to our nation's servicemembers. LEVEL OF EVIDENCE: Prognostic, Level III/IV.

Measuring Competency: Improving the validity of your procedural performance assessments.

Objective: The objective of the study was to compare the use of ordinal scales and interval scales for capturing surgical competency information for general surgeons performing 3 complex trauma procedures. Background: Surgical performance assessment is typically captured using nonparametric data (eg, checklists) that do not support inferential analyses. Interval scales support parametric analyses that are essential for determining competency. We compared assessment outcomes for surgeons performing 3 complex trauma procedures using ordinal and interval scales. Methods: All participants were board-certified or eligible general surgeons. Each participant was assessed by an experienced trauma surgeon while performing 3 trauma procedures on cadavers. All assessors completed a rigorous assessment certification process. We calculated descriptive statistics to examine the differences between interval (parametric) and ordinal (nonparametric) outcomes. Results: Ordinal scales overestimated competence in up to 100% of the participants and did not identify specific performance gaps. Interval scales provided more granularity and identified specific capability gaps. Conclusions: Imprecise instrumentation conveys a false sense of competence and deprives surgeons of opportunities to close capability gaps. Measuring discrete procedural components with interval scales provides a more precise measurement of surgical competency.

Circumventing Acute Compartment Syndrome: Outcomes from a Cadaver-based Course in Fasciotomy Procedural Skills.

Acute compartment syndrome (ACS) represents a surgical emergency requiring effective, complete fasciotomy. The purpose of this study is to evaluate cadaver-based training on the ability of practicing general surgeons to effectively perform upper extremity, thigh, and leg fasciotomies. One hundred seventeen general surgeons underwent a 2-day, cadaver-based course with formative and summative assessments based on validated scoring tools. Overall performance and critical item scores were recorded and compared utilizing analysis of variance with repeated measures and eta-squared values to evaluate effect size. For all three procedures, post-training scores were significantly improved when compared with pre-training scores (p < 0.001). Mean pre-training score for lower leg fasciotomy met the standard for competent performance of the procedure (total score > 80), whereas neither thigh nor upper extremity performance scores met this standard before training. This 2-day, cadaver-based course improves the ability of practicing general surgeons to effectively and independently perform upper extremity, thigh, and lower leg fasciotomies. (Journal of Surgical Orthopaedic Advances 32(4):238-241, 2023).

Five Influential Factors for Clinical Team Performance in Urgent, Emergency Care Contexts.

INTRODUCTION: In deployed contexts, military medical care is provided through the coordinated efforts of multiple interdisciplinary teams that work across and between a continuum of widely distributed role theaters. The forms these teams take, and functional demands, vary by roles of care, location, and mission requirements. Understanding the requirements for optimal performance of these teams to provide emergency, urgent, and trauma care for multiple patients simultaneously is critical. A team's collective ability to function is dependent on the clinical expertise (knowledge and skills), authority, experience, and affective management capabilities of the team members. Identifying the relative impacts of multiple performance factors on the accuracy of care provided by interdisciplinary clinical teams will inform targeted development requirements. MATERIALS AND METHODS: A regression study design determined the extent to which factors known to influence team performance impacted the effectiveness of small, six to eight people, interdisciplinary teams tasked with concurrently caring for multiple patients with urgent, emergency care needs. Linear regression analysis was used to distinguish which of the 11 identified predictors individually and collectively contributed to the clinical accuracy of team performance in simulated emergency care contexts. RESULTS: All data met the assumptions for regression analyses. Stepwise linear regression analysis of the 11 predictors on team performance yielded a model of five predictors accounting for 82.30% of the variance. The five predictors of team performance include (1) clinical skills, (2) team size, (3) authority profile, (4) clinical knowledge, and (5) familiarity with team members. The analysis of variance confirmed a significant linear relationship between team performance and the five predictors, F(5, 240) = 218.34, P < .001. CONCLUSIONS: The outcomes of this study demonstrate that the collective knowledge, skills, and abilities within an urgent, emergency care team must be developed to the extent that each team member is able to competently perform their role functions and that smaller teams benefit by being composed of clinical authorities who are familiar with each other. Ideally, smaller, forward-deployed military teams will be an expert team of individual experts, with the collective expertise and abilities required for their patients. This expertise and familiarity are advantageous for collective consideration of significant clinical details, potential alternatives for treatment, decision-making, and effective implementation of clinical skills during patient care. Identifying the most influential team performance factors narrows the focus of team development strategies to precisely what is needed for a team to optimally perform.

Upper-Extremity Vascular Exposures for Trauma: Comparative Performance Outcomes for General Surgeons and Orthopedic Surgeons.

INTRODUCTION: As combat-related trauma decreases, there remains an increasing need to maintain the ability to care for trauma victims from other casualty events around the world (e.g., terrorism, natural disasters, and infrastructure failures). During these events, military surgeons often work closely with their civilian counterparts, often in austere and expeditionary contexts. In these environments, the primary aim of the surgical team is to implement damage control principles to avert blood loss, optimize oxygenation, and improve survival. Upper-extremity vascular injuries are associated with high rates of morbidity and mortality resulting from exsanguination and ischemic complications; however, fatalities may be avoided if hemorrhage is rapidly controlled. In austere contexts, deployed surgical teams typically include one general surgeon and one orthopedic surgeon, neither of which have acquired the expertise to manage these vascular injuries. The purpose of this study was to examine the baseline capabilities of general surgeons and orthopedic surgeons to surgically expose and control axillary and brachial arteries and to determine if the abilities of both groups could be increased through a focused cadaver-based training intervention. METHODS: This study received IRB approval at our institution. Study methods included the use of cadavers for baseline assessment of procedural capabilities to expose and control axillary and brachial vessels, followed by 1:1 procedural training and posttraining re-assessment of procedural capabilities. Inferential analyses included ANOVA/MANOVA for within- and between-group effects (P < .05). Effect sizes were calculated using Cohen's d. RESULTS: Study outcomes demonstrated significant differences between the baseline performance abilities of the two groups, with general surgeons outperforming orthopedic surgeons. Before training, neither group reached performance benchmarks for overall or critical procedural abilities in exposing axillary and brachial vessels. Training led to increased abilities for both groups. There were statistically significant gains for overall procedural abilities, as well as for critical procedural elements that are directly associated with morbidity and mortality. These outcomes were consistent for both general and orthopedic surgeons. Effect sizes ranged between medium (general surgeons) and very large (orthopedic surgeons). CONCLUSION: There was a baseline capability gap for both general surgeons and orthopedic surgeons to surgically expose and control the axillary and brachial vessels. Outcomes from the course suggest that the methodology facilitates the acquisition of accurate and independent vascular procedural capabilities in the management of upper-extremity trauma injuries. The impact of this training for surgeons situated in expeditionary or remote contexts has direct relevance for caring for victims of extremity trauma. These outcomes underscore the need to train all surgeons serving in rural, remote, expeditionary, combat, or global health contexts to be able to competently manage extremity trauma and concurrent vascular injuries to increase the quality of care in those settings.

Dunning-Kruger Effect Between Self-Peer Ratings of Surgical Performance During a MASCAL Event and Pre-Event Assessed Trauma Procedural Capabilities.

Objectives: The research question asked to what extent do self-rated performance scores of individual surgeons correspond to assessed procedural performance abilities and to peer ratings of procedural performance during a mass casualty (MASCAL) event? Background: Self-assessment using performance rating scales is ubiquitous in surgical education as a proxy for direct measurement of competence. The validity and reliability of self-ratings as competency measures are susceptible to cognitive biases such as Dunning-Kruger effects, which describe how individuals over/underestimate their own performance compared to assessments from independent sources. The ability of surgeons to accurately self-assess their procedural performance remains undetermined. Methods: A purposive sample of military surgeons (N = 13) who collectively cared for trauma patients during a MASCAL event participated in the study. Pre-event performance assessment scores for 32 trauma procedures were compared with post-event self and peer performance ratings using F tests (P < 0.05) and effect sizes (Cohen's d). Results: There were no significant differences between peer ratings and performance assessment scores. There were significant differences between self-ratings and both peer ratings (P < 0.001) and performance assessment scores (P < 0.001). Effect sizes were very large for self to peer rating comparison (Cohen's d = 2.34) and self to performance assessment comparison (Cohen's d = 2.77). Conclusions: The outcomes demonstrate that self-ratings were significantly lower than the independently determined assessment scores for each surgeon, revealing a Dunning-Kruger effect for highly skilled individuals underestimating their abilities. These outcomes underscore the limitations of self-assessment for measuring competence.

A Novel Paradigm for Surgical Skills Training and Assessment of Competency.

Importance: Sustainment of comprehensive procedural skills in trauma surgery is a particular problem for surgeons in rural, global, and combat settings. Trauma care often requires open surgical procedures for low-frequency/high-risk injuries at a time when open surgical experience is declining in general and trauma surgery training. Objective: To determine whether general surgeons participating in a 2-day standardized trauma skills course demonstrate measurable improvement in accuracy and independent performance of specific trauma skills. Design, Setting, and Participants: General surgeons in active surgical practice were enrolled from a simulation center with anatomic laboratory from October 2019 to October 2020. Differences in pretraining/training and posttraining performance outcomes were examined using (1) pretraining/posttraining surveys, (2) pretraining/posttraining knowledge assessment, and (3) training/posttraining faculty assessment. Analysis took place in November 2020. Interventions: A 2-day standardized, immersive, cadaver-based skills course, developed with best practices in instructional design, that teaches and assesses 24 trauma surgical procedures was used. Main Outcomes and Measures: Trauma surgery capability, as measured by confidence, knowledge, abilities, and independent performance of specific trauma surgical procedures; 3-month posttraining skill transfer. Results: The study cohort included 65 active-duty general surgeons, of which 16 (25%) were women and 49 (75%) were men. The mean (SD) age was 38.5 (4.2) years. Before and during training, 1 of 65 participants (1%) were able to accurately perform all 24 procedures without guidance. After course training, 64 participants (99%) met the benchmark performance requirements for the 24 trauma procedures, and 51 (78%) were able to perform them without guidance. Procedural confidence and knowledge increased significantly from before to after the course. At 3 months after training, 37 participants (56%) reported skill transfer to trauma or other procedures. Conclusions and Relevance: In this study, direct measurement of procedural performance following standardized training demonstrated significant improvement in skill performance in a broad array of trauma procedures. This model may be useful for assessment of procedural competence in other specialties.

Retention Curves for Pediatric and Neonatal Intubation Skills After Simulation-Based Training.

OBJECTIVES: We evaluated the retention of pediatric and neonatal intubation performance abilities of clinicians trained on a simulated or live tissue model at 3 intervals after initial training to assess competency degradation related to either training modality or retention interval. METHODS: We implemented a quasi-experimental design with purposive sampling to assess performance differences between 171 subjects randomly assigned to 1 of 3 intervals after initial training: 6 weeks, 18 weeks, or 52 weeks. Training followed the American Heart Association Pediatric Advanced Life Support and Neonatal Resuscitation Program protocols with hands-on practice using 1 of 2 models (live feline or simulated feline). Assessment data were captured using validated instruments and analyzed using analysis of variance with repeated measures (statistical significance set at P < 0.05). RESULTS: Cognitive retention scores decreased significantly (P = 0.000) from posttraining cognitive scores. There were no significant differences between posttraining and retention scores for pediatric and neonatal performances. Both affect and self-efficacy retention scores decreased significantly (P = 0.000) from posttraining scores at 18 and 52 weeks, but remained constant at 6 weeks. Retention scores for all dimensions showed a significant difference between subjects with varying amounts of experience performing pediatric and neonatal intubation, such that those with more experience scored higher those with less (P < 0.003). CONCLUSIONS: Retention performance outcomes decreased sufficiently from posttraining scores to suggest that training refreshment could serve to maintain posttraining competency in the ability to perform pediatric and neonatal intubation. Retraining intervals may be best aligned with provider experience levels. Future research focusing on the effect of variable interval refresher training on retention in pediatric and neonatal intubation is merited.

Texting From the Bush: Data Collection Using SMS Text Messaging in Areas of Low Network Coverage From Low-Literacy Providers.

Mobile health technology, specifically Short Message Service (SMS), provides a low-cost medium to transmit data in real time. SMS has been used for data collection by highly literate and educated health care workers in low-resource countries; however, no previous studies have evaluated implementation of an SMS intervention by low-literacy providers. The Liberian Ministry of Health and Social Welfare identified a lack of accurate data on the number of pregnancies from rural areas. To capture these data from 11 rural communities in Liberia, 66 low-literate traditional midwives and 15 high-literate certified midwives were trained to report data via SMS. Data were reported via a 9-digit code sent from Java-based mobile phones. Study aims included determining the following components of SMS transmission: success rate, accuracy, predictors of successful transmission, and acceptance. Success rate of SMS transmission was significantly higher for certified midwives than for traditional midwives. The error rate was significantly higher for traditional midwives than for certified midwives. Years of education was the only predictor of successful SMS transmission. Traditional midwives and certified midwives accepted the intervention, although certified midwives found it easier to use. Certified midwives performed significantly better than did traditional midwives. SMS texting interventions should be targeted to health care workers with higher rates of literacy.

Performance-based comparison of neonatal intubation training outcomes: simulator and live animal.

The purpose of this article was to establish psychometric validity evidence for competency assessment instruments and to evaluate the impact of 2 forms of training on the abilities of clinicians to perform neonatal intubation. To inform the development of assessment instruments, we conducted comprehensive task analyses including each performance domain associated with neonatal intubation. Expert review confirmed content validity. Construct validity was established using the instruments to differentiate between the intubation performance abilities of practitioners (N = 294) with variable experience (novice through expert). Training outcomes were evaluated using a quasi-experimental design to evaluate performance differences between 294 subjects randomly assigned to 1 of 2 training groups. The training intervention followed American Heart Association Pediatric Advanced Life Support and Neonatal Resuscitation Program protocols with hands-on practice using either (1) live feline or (2) simulated feline models. Performance assessment data were captured before and directly following the training. All data were analyzed using analysis of variance with repeated measures and statistical significance set at P < .05. Content validity, reliability, and consistency evidence were established for each assessment instrument. Construct validity for each assessment instrument was supported by significantly higher scores for subjects with greater levels of experience, as compared with those with less experience (P = .000). Overall, subjects performed significantly better in each assessment domain, following the training intervention (P = .000). After controlling for experience level, there were no significant differences among the cognitive, performance, and self-efficacy outcomes between clinicians trained with live animal model or simulator model. Analysis of retention scores showed that simulator trained subjects had significantly higher performance scores after 18 weeks (P = .01) and 52 weeks (P = .001) and cognitive scores after 52 weeks (P = .001). The results of this study demonstrate the feasibility of using valid, reliable assessment instruments to assess clinician competency and self-efficacy in the performance of neonatal intubation. We demonstrated the relative equivalency of live animal and simulation-based models as tools to support acquisition of neonatal intubation skills. Retention of performance abilities was greater for subjects trained using the simulator, likely because it afforded greater opportunity for repeated practice. Outcomes in each assessment area were influenced by the previous intubation experience of participants. This suggests that neonatal intubation training programs could be tailored to the level of provider experience to make efficient use of time and educational resources. Future research focusing on the uses of assessment in the applied clinical environment, as well as identification of optimal training cycles for performance retention, is merited.

Pediatric and neonatal intubation training gap analysis: instruction, assessment, and technology.

Summary statement: Intubating adult patients presents numerous challenges for clinicians. Procedural complexities associated with performing pediatric and neonatal intubation, along with a lower frequency of a need for intubating pediatric and neonatal patients, further amplifies the difficulties associated with acquiring and maintaining relevant clinical skills. Clinicians must develop and maintain competency through training and continuing medical education. A systematic review was conducted through June 2012 to examine current instructional methods, assessment tools, and training models in pediatric and neonatal intubation training. Variability among the included literature on instruction, assessment, and models used to support training makes direct comparison problematic. The results of this review emphasize the need for improvement of current instructional methods to promote competency acquisition; development of valid, reliable assessment tools to evaluate clinician competency; and identification of a superior training model to promote acquisition and maintenance of skills associated with intubation of pediatric patients.

Evaluation of a precise and measurable model for learning laparoscopic tissue handling.

INTRODUCTION: Laparoscopic tissue handling is quite difficult to measure using virtual-reality laparoscopic simulators and box-trainer exercises, and therefore, completion time is the predominant performance measure for simulation-based laparoscopic training exercises. The purpose of this study was to evaluate the construct validity of a training and assessment model for precise laparoscopic handling of delicate tissue. METHODS: Participants (n = 35) completed 2 progressively challenging laparoscopic tissue translocation exercises using delicate foam pieces and templates. Deidentified performances were scored using objective measures for tissue damage, accuracy, percentage complete, and completion time. Evaluation included multiple analysis of variance with repeated measures among the 3 groups as follows: medical students, residents and faculty who perform laparoscopic surgery less than once per week, and faculty members who perform laparoscopic surgery at least once per week. RESULTS: The model demonstrated significant construct validity by discriminating performances between the types of shapes and templates and across the levels of surgical experience on all dimensions. A significant interaction effect between the level of expertise and the difficulty of the exercise revealed excellent discrimination between experienced laparoscopic surgeons and others. DISCUSSION: This low-cost model provides an alternative or adjunct platform for laparoscopic training and assessment that requires precise and measurable handling of a delicate tissue.

Postinterview Communication between Obstetrics and Gynecology Residency Programs and Candidates.

INTRODUCTION: Residency programs seek to match the best candidates with their positions. To avoid ethical conflicts in this process, the National Residency Matching Program (NRMP or Match) has rules regarding appropriate conduct, including guidelines on contact between candidates and programs. Our study examined communication between obstetrics and gynecology (Ob-Gyn) programs and residency candidates after interviewing and prior to ranking. METHODS: Ob-Gyn program directors in the United States were sent a self-administered survey via e-mail. Data were collected and analyzed using descriptive methods to examine communication practices of these programs. RESULTS: The response rate was 40%. The findings showed that respondents had variable interpretations of the NRMP rules and suggest that programs may be communicating their match intentions especially to favored candidates. Respondents' open text comments highlighted program directors' frustrations with current NRMP rules. DISCUSSION: NRMP communication rules are intended to minimize pressure on residency candidates. Our findings suggest they may be leading to unforeseen stresses on program directors and candidates. CONCLUSIONS: As educational leaders in medicine, we must consider what professional communications are acceptable without increasing the pressure on candidates during the ranking and match process.

Validity: what does it mean for competency-based assessment in obstetrics and gynecology?

Validity refers to an evidence-based claim about the trustworthiness of decisions made from context-specific performance data. Validity requirements for competency-based assessments in obstetrics and gynecology have not been defined in the literature. We explain why validity is intrinsic to any discussion about competency assessment and provide a model for obstetrics and gynecology programs to use in determining the essential validity evidence for various forms of assessments. The implications of decisions made from assessment results influence the requisite level and precision of validity evidence. Although validity evidence is essential, it is also flexibly tied to the implications of decisions made from assessment results and not all assessments require the same degree of validity. We propose a model for considering validity, and build a discussion around specific assessment examples targeting progressive levels of expertise along the training continuum.

A typology for health care teams.

BACKGROUND: Effective interdisciplinary health care teamwork improves clinical and financial outcomes, and training and assessment of team competencies are central to establishing high-functioning health care teams. The roles that team members assume in the provision of patient care are important contributors to effective health care team performance; however, variability among health care practitioners can lead to philosophical, political, social, and clinical differences in perceptions and recommendations for patient care as well as expected communication patterns and protocols. PURPOSE: The purpose of this study was to describe the roles and behaviors within variable health care teams in the provision of patient care across multiple clinical practice areas to inform a model for team development strategies. METHODOLOGY: Interdisciplinary health care teams were observed in vivo during the routine course of their work in multiple patient care contexts. Data were collected and analyzed using qualitative methods of observation and categorization, with supplemental interviews to substantiate, to clarify, and to verify observations. The constant comparative method of data analyses was used to derive a compositional typology for health care teams. FINDINGS: A compositional typology for health care teams emerged from the data specifying four types of health care teams: stable role, stable personnel (Type SRSP); stable role, variable personnel (Type SRVP); variable role, stable personnel (Type VRSP); and variable role, variable personnel (Type VRVP). IMPLICATIONS: Results suggest that health care teams may be more complicated than non-health care teams, and team models with associated derived competencies from other professions may not wholly transfer to health care. A singular model to inform best practices for health care team development may not adequately address the specific performance challenges of each team type. Adaptable development strategies for each type of team and its associated role membership may be required to optimize team performance. The health care team typology derived from this study may help inform the selection of appropriate team development strategies and define associated team competencies.

Simulation and team training.

The use of simulation-based methods for clinical and team training provides an opportunity for health care professionals to develop and maintain the skills required to effectively manage patient care. This is especially true for those rare events when emergency interventions require urgent, accurate, and cohesive team functioning. We present a framework for considering simulation-based training, examine contextual factors and the outcomes of research conducted to date in this area, and provide suggestions for selecting simulation-based approaches for developing obstetrics and gynecology teams in multiple contexts.