Designing formative assessments to improve anatomy exam performance.

Formative assessments are primarily used as a tool to gauge learning throughout an anatomy course. They have also been demonstrated to improve student mastery and exam performance, although the precise nature of this relationship is poorly understood. In this study, it is hypothesized that formative assessment questions targeting higher cognitive levels, integrating topics from multiple lessons, and including visuospatial elements will increase student exam performance. Formative and summative questions provided to students during the Clinical Anatomy block at the University of Arizona College of Medicine-Phoenix between 2015 and 2018 were assessed for cognitive level, integration of targeted learning objectives, and presence or absence of visuospatial elements. These variables were entered into a hierarchical linear model along with demographic variables for each cohort to assess the relationships between these variables and cohort performance on exam questions. The best predictor of exam performance was the inclusion of constituent learning objectives within the formative assessment. Additionally, students performed better on exam questions with visuospatial elements when the targeted learning objectives were also associated with visuospatial elements on the formative assessment. Surprisingly, the cognitive level of formative questions and the integration of learning objectives within them were not correlated with student exam performance. This study demonstrates the importance of including a broad range of topics in formative assessments and highlights a potential benefit of adopting consistent question formats for formative assessments and exams.

Enhanced Training Benefits of Video Recording Surgery With Automated Hand Motion Analysis.

BACKGROUND: Hand motion analysis by video recording during surgery has potential for evaluation of surgical performance. The aim was to identify how technical skill during open surgery can be measured unobtrusively by video recording during a surgical procedure. We hypothesized that procedural-step timing, hand movements, instrument use and Shannon entropy differ with expertise and training and are concordant with a performance-based validated individual procedure score. METHODS: Surgeon and non-surgeon participants with varying training and levels of expertise were video recorded performing axillary artery exposure and control (AA) on un-preserved cadavers. Color-coded gloves permitted motion-tracking and automated extraction of entropy data from recordings. Timing and instrument-use metrics were obtained through observational video reviews. Shannon entropy measured speed, acceleration and direction by computer-vision algorithms. Findings were compared with individual procedure score for AA performance RESULTS: Experts had lowest entropy values, idle time, active time and shorter time to divide pectoralis minor, using fewer instruments. Residents improved with training, without reaching expert levels, and showed deterioration 12-18 months later. Individual procedure scores mirrored these results. Non-surgeons differed substantially. CONCLUSIONS: Hand motion entropy and timing metrics discriminate levels of surgical skill and training, and these findings are congruent with individual procedure score evaluations. These measures can be collected using consumer-level cameras and analyzed automatically with free software. Hand motion with video timing data may have widespread application to evaluate resident performance and can contribute to the range of evaluation and testing modalities available to educators, training course designers and surgical quality assurance programs.

Bioarchaeology of Neolithic Çatalhöyük reveals fundamental transitions in health, mobility, and lifestyle in early farmers.

The transition from a human diet based exclusively on wild plants and animals to one involving dependence on domesticated plants and animals beginning 10,000 to 11,000 y ago in Southwest Asia set into motion a series of profound health, lifestyle, social, and economic changes affecting human populations throughout most of the world. However, the social, cultural, behavioral, and other factors surrounding health and lifestyle associated with the foraging-to-farming transition are vague, owing to an incomplete or poorly understood contextual archaeological record of living conditions. Bioarchaeological investigation of the extraordinary record of human remains and their context from Neolithic Çatalhöyük (7100-5950 cal BCE), a massive archaeological site in south-central Anatolia (Turkey), provides important perspectives on population dynamics, health outcomes, behavioral adaptations, interpersonal conflict, and a record of community resilience over the life of this single early farming settlement having the attributes of a protocity. Study of Çatalhöyük human biology reveals increasing costs to members of the settlement, including elevated exposure to disease and labor demands in response to community dependence on and production of domesticated plant carbohydrates, growing population size and density fueled by elevated fertility, and increasing stresses due to heightened workload and greater mobility required for caprine herding and other resource acquisition activities over the nearly 12 centuries of settlement occupation. These changes in life conditions foreshadow developments that would take place worldwide over the millennia following the abandonment of Neolithic Çatalhöyük, including health challenges, adaptive patterns, physical activity, and emerging social behaviors involving interpersonal violence.

Cadaver-Based Trauma Procedural Skills Training: Skills Retention 30 Months after Training among Practicing Surgeons in Comparison to Experts or More Recently Trained Residents.

BACKGROUND: Long-term retention of trauma procedural core-competency skills and need for re-training after a 1-day cadaver-based course remains unknown. We measured and compared technical skills for trauma core competencies at mean 14 months (38 residents), 30 months (35 practicing surgeons), and 46 months (10 experts) after training to determine if skill degradation occurs with time. Technical performance during extremity vascular exposures and lower-extremity fasciotomy in fresh cadavers measured by validated individual procedure score (IPS) was the primary outcome. STUDY DESIGN: We performed a prospective study between May 2013 and September 2016. RESULTS: Practicing surgeons had lower IPS and IPS component scores (p = 0.02 to 0.001) than residents (p < 0.05) and experts (p < 0.002) for vascular procedures. Frequencies of errors were no different among residents and experts. Practicing surgeons made more critical errors (p < 0.05) than experts or residents. Experts had shortest time to proximal vascular control. Fasciotomy procedural errors occurred in all participants. Cluster analysis of anatomy vs procedural steps identified tertiles of performance and wide variance (32.5% practicing surgeons, 26.5% residents vs 13% experts) for vascular procedures. Vascular control duration > 20 minutes (n = 21) and failure to decompress fasciotomy compartments were correlated with incorrect landmarks and skin incisions. Modeling found interval trauma skills experience, not time since training, was associated with lower IPS. CONCLUSIONS: Practicing surgeons with low trauma skills experience since training had lower IPS and component scores (p = 0.02 to 0.001) and more errors compared with experts and residents (p < 0.05). Surgeons, including experts with low interval experience performing trauma procedures, may benefit from refreshing of correct landmarks and skin incision placement identification.

Assessment of Anatomical Knowledge and Core Trauma Competency Vascular Skills.

Objectives: Surgical residents express confidence in performing specific vascular exposures before training, but such self-reported confidence did not correlate with co-located evaluator ratings. This study reports residents' self-confidence evaluated before and after Advanced Surgical Skills for Exposure in Trauma (ASSET) cadaver-based training, and 12-18 mo later. We hypothesize that residents will better judge their own skill after ASSET than before when compared with evaluator ratings. Methods: Forty PGY2-7 surgical residents performed four procedures: axillary artery (AA), brachial artery (BA), femoral artery exposure and control (FA), and lower extremity fasciotomy (FAS) at the three evaluations. Using 5-point Likert scales, surgeons self-assessed their confidence in anatomical understanding and procedure performance after each procedure and evaluators rated each surgeon accordingly. Results: For all the three evaluations, residents consistently rated their anatomical understanding (p < 0.04) and surgical performance (p < 0.03) higher than evaluators for both FA and FAS. Residents rated their anatomical understanding and surgical performance higher (p < 0.005) than evaluators for BA after training and up to 18 mo later. Only for third AA evaluation were there no rating differences. Conclusions: Residents overrate their anatomical understanding and performance abilities for BA, FA, and FAS even after performing the procedures and being debriefed three times in 18 mo.

Head-camera video recordings of trauma core competency procedures can evaluate surgical resident's technical performance as well as colocated evaluators.

BACKGROUND: Unbiased evaluation of trauma core competency procedures is necessary to determine if residency and predeployment training courses are useful. We tested whether a previously validated individual procedure score (IPS) for individual procedure vascular exposure and fasciotomy (FAS) performance skills could discriminate training status by comparing IPS of evaluators colocated with surgeons to blind video evaluations. METHODS: Performance of axillary artery (AA), brachial artery (BA), and femoral artery (FA) vascular exposures and lower extremity FAS on fresh cadavers by 40 PGY-2 to PGY-6 residents was video-recorded from head-mounted cameras. Two colocated trained evaluators assessed IPS before and after training. One surgeon in each pretraining tertile of IPS for each procedure was randomly identified for blind video review. The same 12 surgeons were video-recorded repeating the procedures less than 4 weeks after training. Five evaluators independently reviewed all 96 randomly arranged deidentified videos. Inter-rater reliability/consistency, intraclass correlation coefficients were compared by colocated versus video review of IPS, and errors. Study methodology and bias were judged by Medical Education Research Study Quality Instrument and the Quality Assessment of Diagnostic Accuracy Studies criteria. RESULTS: There were no differences (p ≥ 0.5) in IPS for AA, FA, FAS, whether evaluators were colocated or reviewed video recordings. Evaluator consistency was 0.29 (BA) - 0.77 (FA). Video and colocated evaluators were in total agreement (p = 1.0) for error recognition. Intraclass correlation coefficient was 0.73 to 0.92, dependent on procedure. Correlations video versus colocated evaluations were 0.5 to 0.9. Except for BA, blinded video evaluators discriminated (p < 0.002) whether procedures were performed before training versus after training. Study methodology by Medical Education Research Study Quality Instrument criteria scored 15.5/19, Quality Assessment of Diagnostic Accuracy Studies 2 showed low bias risk. CONCLUSION: Video evaluations of AA, FA, and FAS procedures with IPS are unbiased, valid, and have potential for formative assessments of competency. LEVEL OF EVIDENCE: Prognostic study, level II.

Performance of Vascular Exposure and Fasciotomy Among Surgical Residents Before and After Training Compared With Experts.

Importance: Surgical patient outcomes are related to surgeon skills. Objective: To measure resident surgeon technical and nontechnical skills for trauma core competencies before and after training and up to 18 months later and to compare resident performance with the performance of expert traumatologists. Design, Setting, and Participants: This longitudinal study performed from May 1, 2013, through February 29, 2016, at Maryland State Anatomy Board cadaver laboratories included 40 surgical residents and 10 expert traumatologists. Interventions: Performance was measured during extremity vascular exposures and lower extremity fasciotomy in fresh cadavers before and after taking the Advanced Surgical Skills for Exposure in Trauma (ASSET) course. Main Outcomes and Measures: The primary outcome variable was individual procedure score (IPS), with secondary outcomes of IPSs on 5 components of technical and nontechnical skills, Global Rating Scale scores, errors, and time to complete the procedure. Two trained evaluators located in the same laboratory evaluated performance with a standardized script and mobile touch-screen data collection. Results: Thirty-eight (95%) of 40 surgical residents (mean [SD] age, 31 [2.9] years) who were evaluated before and within 4 weeks of ASSET training completed follow-up evaluations 12 to 18 months later (mean [SD], 14 [2.7] months). The experts (mean [SD] age, 52 [10.0] years) were significantly older and had a longer (mean [SD], 46 [16.3] months) interval since taking the ASSET course (both P < .001). Overall resident cohort performance improved with increased anatomy knowledge, correct procedural steps, and decreased errors from 60% to 19% after the ASSET course regardless of clinical year of training (P < .001). For 21 of 40 residents (52%), correct vascular procedural steps plotted against anatomy knowledge (the 2 IPS components most improved with training) indicates the resident's performance was within 1 nearest-neighbor classifier of experts after ASSET training. Five residents had no improvement with training. The Trauma Readiness Index for experts (mean [SD], 74 [4]) was significantly different compared with the trained residents (mean [SD], 48 [7] before training vs 63 [7] after training [P = .004] and vs 64 [6] 14 months later [P = .002]). Critical errors that might lead to patient death were identified by pretraining IPS decile of less than 0.5. At follow-up, frequency of resident critical errors was no different from experts. The IPSs ranged from 31.6% to 76.9% among residents for core trauma competency procedures. Modeling revealed that interval experience, rather than time since training, affected skill retention up to 18 months later. Only 4 experts and 16 residents (40%) adequately decompressed and confirmed entry into all 4 lower extremity compartments. Conclusions and Relevance: This study found that ASSET training improved resident procedural skills for up to 18 months. Performance was highly variable. Interval experience after training affected performance. Pretraining skill identified competency of residents vs experts. Extremity vascular and fasciotomy performance evaluations suggest the need for specific anatomical training interventions in residents with IPS deciles less than 0.5.

Using an Individual Procedure Score Before and After the Advanced Surgical Skills Exposure for Trauma Course Training to Benchmark a Hemorrhage-Control Performance Metric.

OBJECTIVE: Test with an individual procedure score (IPS) to assess whether an unpreserved cadaver trauma training course, including upper and lower limb vascular exposure, improves correct identification of surgical landmarks, underlying anatomy, and shortens time to vascular control. DESIGN: Prospective study of performance of 3 vascular exposure and control procedures (axillary, brachial, and femoral arteries) using IPS metrics by 2 colocated and trained evaluators before and after training with the Advanced Surgical Skills Exposure for Trauma (ASSET) course. IPS, including identification of anatomical landmarks, incisions, underlying structures, and time to completion of each procedure was compared before and after training using repeated measurement models. SETTING: Audio-video instrumented cadaver laboratory at University of Maryland School of Medicine. PARTICIPANTS: A total of 41 second to sixth year surgical residents from surgical programs throughout Mid-Atlantic States who had not previously taken the ASSET course were enrolled, 40 completed the pre- and post-ASSET performance evaluations. RESULTS: After ASSET training, all components of IPS increased and time shortened for each of the 3 artery exposures. Procedure steps performed correctly increased 57%, anatomical knowledge increased 43% and skin incision to passage of a vessel loop twice around the correct vessel decreased by a mean of 2.5 minutes. An overall vascular trauma readiness index, a comprehensive IPS score for 3 procedures increased 28% with ASSET Training. CONCLUSIONS: Improved knowledge of surface landmarks and underlying anatomy is associated with increased IPS, faster procedures, more accurate incision placement, and successful vascular control. Structural recognition during specific procedural steps and anatomical knowledge were key points learned during the ASSET course. Such training may accelerate acquisition of specific trauma surgery skills to compensate for shortened training hours, infrequent exposure to major vascular injuries, or when just-in-time training is necessary. IPS is a benchmark for competence in extremity vascular control.

Gradual decline in mobility with the adoption of food production in Europe.

Increased sedentism during the Holocene has been proposed as a major cause of decreased skeletal robusticity (bone strength relative to body size) in modern humans. When and why declining mobility occurred has profound implications for reconstructing past population history and health, but it has proven difficult to characterize archaeologically. In this study we evaluate temporal trends in relative strength of the upper and lower limb bones in a sample of 1,842 individuals from across Europe extending from the Upper Paleolithic [11,000-33,000 calibrated years (Cal y) B.P.] through the 20th century. A large decline in anteroposterior bending strength of the femur and tibia occurs beginning in the Neolithic (∼ 4,000-7,000 Cal y B.P.) and continues through the Iron/Roman period (∼ 2,000 Cal y B.P.), with no subsequent directional change. Declines in mediolateral bending strength of the lower limb bones and strength of the humerus are much smaller and less consistent. Together these results strongly implicate declining mobility as the specific behavioral factor underlying these changes. Mobility levels first declined at the onset of food production, but the transition to a more sedentary lifestyle was gradual, extending through later agricultural intensification. This finding only partially supports models that tie increased sedentism to a relatively abrupt Neolithic Demographic Transition in Europe. The lack of subsequent change in relative bone strength indicates that increasing mechanization and urbanization had only relatively small effects on skeletal robusticity, suggesting that moderate changes in activity level are not sufficient stimuli for bone deposition or resorption.

Development and validation of trauma surgical skills metrics: Preliminary assessment of performance after training.

BACKGROUND: Maintaining trauma-specific surgical skills is an ongoing challenge for surgical training programs. An objective assessment of surgical skills is needed. We hypothesized that a validated surgical performance assessment tool could detect differences following a training intervention. METHODS: We developed surgical performance assessment metrics based on discussion with expert trauma surgeons, video review of 10 experts and 10 novice surgeons performing three vascular exposure procedures and lower extremity fasciotomy on cadavers, and validated the metrics with interrater reliability testing by five reviewers blinded to level of expertise and a consensus conference. We tested these performance metrics in 12 surgical residents (Year 3-7) before and 2 weeks after vascular exposure skills training in the Advanced Surgical Skills for Exposure in Trauma (ASSET) course. Performance was assessed in three areas as follows: knowledge (anatomic, management), procedure steps, and technical skills. Time to completion of procedures was recorded, and these metrics were combined into a single performance score, the Trauma Readiness Index (TRI). Wilcoxon matched-pairs signed-ranks test compared pretraining/posttraining effects. RESULTS: Mean time to complete procedures decreased by 4.3 minutes (from 13.4 minutes to 9.1 minutes). The performance component most improved by the 1-day skills training was procedure steps, completion of which increased by 21%. Technical skill scores improved by 12%. Overall knowledge improved by 3%, with 18% improvement in anatomic knowledge. TRI increased significantly from 50% to 64% with ASSET training. Interrater reliability of the surgical performance assessment metrics was validated with single intraclass correlation coefficient of 0.7 to 0.98. CONCLUSION: A trauma-relevant surgical performance assessment detected improvements in specific procedure steps and anatomic knowledge taught during a 1-day course, quantified by the TRI. ASSET training reduced time to complete vascular control by one third. Future applications include assessing specific skills in a larger surgeon cohort, assessing military surgical readiness, and quantifying skill degradation with time since training.

Perception does not equal reality for resident vascular trauma skills.

BACKGROUND: Experience with the management of vascular trauma by senior surgical residents is increasingly limited. When queried about their understanding of anatomy and ability to perform specific vascular exposures, residents express a moderately high level of confidence. We hypothesized that this perception does not equal reality. METHODS: A total of 42 senior surgical residents participating in an ongoing validation study of the Advanced Surgical Skills for Exposures in Trauma course were asked to self-assess their baseline (precourse) confidence of their understanding of the anatomy required to perform and their ability to perform exposure and control of the axillary, brachial, and femoral arteries, as well as lower extremity fasciotomy using a 5-point Likert scale. Residents then performed the four procedures on a fresh cadaver model and were scored in real time by experts using a global assessment of anatomic knowledge and readiness to perform." The Student t-test was used with α set at P < 0.05. RESULTS: Residents consistently rated their understanding of anatomy and their ability to perform the procedures significantly higher than expert evaluator ultimately scored them. Evaluators also deemed that residents would be unable to perform without help 65%-86% of the time. CONCLUSIONS: Senior residents are ill-prepared to perform the procedures studied and have an unwarranted confidence in their knowledge and abilities. Perception clearly does not equal reality in preparing these trainees to perform as advertized. The low global scores for anatomy and performance should be a wake-up call for surgical educators prompting curricular reform and evaluation.

Anemia or scurvy: A pilot study on differential diagnosis of porous and hyperostotic lesions using differential cranial vault thickness in subadult humans.

Metabolic disorders, such as scurvy, manifested in human skeletal remains provide insight into health, nutrition, and environmental quality in past populations. Porous cranial vault lesions are often used to diagnose metabolic conditions in subadult remains, but overlapping gross lesion expressions have led to over-diagnosis of anemia and under-diagnosis of scurvy. Studies by Ortner and colleagues have suggested that specific porous cranial lesions are pathognomonic of scurvy, but additional diagnostic tools are necessary. In this technical report, we offer a preliminary assessment of cranial vault thickness (CVT) at the site of porous lesions (sensu lato porotic hyperostosis, cribra orbitalia) as a method for distinguishing between scurvy and anemia in subadult crania. Computed Tomography (CT) was used to measure CVT at various landmarks associated with porotic hyperostosis and cribra orbitalia, complemented by lesion scores, from scorbutic (N=11), anemic (N=3), and non-pathological (N=28) subadult crania used as a control group. Results indicate that CVT consistently distinguishes scorbutic from non-pathological individuals, while anemic individuals overlap with both - likely a function of small sample size in this study. Despite current limitations, CVT has the potential to be an objective diagnostic tool for distinguishing scurvy and expanding reconstructions of nutritional adequacy over the life course in past populations.

Interpreting skeletal growth in the past from a functional and physiological perspective.

The study of juvenile skeletal remains can yield important insights into the health, behavior, and biological relationships of past populations. However, most studies of past skeletal growth have been limited to relatively simple metrics. Considering additional skeletal parameters and taking a broader physiological perspective can provide a more complete assessment of growth patterns and environmental and genetic effects on those patterns. We review here some alternative approaches to ontogenetic studies of archaeological and paleontological skeletal material, including analyses of body size (stature and body mass) and cortical bone structure of long bone diaphyses and the mandibular corpus. Together such analyses can shed new light on both systemic and localized influences on bone growth, and the metabolic and mechanical factors underlying variation in growth.

Stature and body mass estimation from skeletal remains in the European Holocene.

Techniques that are currently available for estimating stature and body mass from European skeletal remains are all subject to various limitations. Here, we develop new prediction equations based on large skeletal samples representing much of the continent and temporal periods ranging from the Mesolithic to the 20th century. Anatomical reconstruction of stature is carried out for 501 individuals, and body mass is calculated from estimated stature and biiliac breadth in 1,145 individuals. These data are used to derive stature estimation formulae based on long bone lengths and body mass estimation formulae based on femoral head breadth. Prediction accuracy is superior to that of previously available methods. No systematic geographic or temporal variation in prediction errors is apparent, except in tibial estimation of stature, where northern and southern European formulae are necessary because of the presence of relatively longer tibiae in southern samples. Thus, these equations should bebroadly applicable to European Holocene skeletal samples.

Technical note: an R program for automating bone cross section reconstruction.

Many recent studies have used long bone cross-sectional geometric properties in various comparative analyses. Methods have been described for reconstructing diaphyseal cross sections from external molds and biplanar radiographs that produce accurate results (within 5% of true values on average). The manual image processing required, however, is both time and labor intensive. A new freely available program developed here for the computational freeware, R, automates much of the process. This study compares cross-sectional properties calculated using the new R program to those from peripheral quantitative CT (pQCT) and the original manual method. We find that the R program works aswell as the original manual image processing for most cross sections eliminates the chance for entry errors at several steps and greatly speeds up data collection.

Student laboratory presentations as a learning tool in anatomy education.

Previous studies have shown that anatomy students who complete oral laboratory presentations believe they understand the material better and retain it longer than they otherwise would if they only took examinations on the material; however, we have found no studies that empirically test such outcomes. The purpose of this study was to assess the effectiveness of oral presentations through comparisons with other methods of assessment, most notably, examination performance. Specifically, we tested whether students (n = 256) performed better on examination questions on topics covered by their oral presentations than on other topics. Each student completed two graded, 12-minute laboratory presentations on two different assigned topics during the course and took three examinations, each of which covered a third of the course material. Examination questions were characterized by type (memorization, pathway, analytical, spatial). A two-way repeated measures analysis of variance revealed that students performed better on topics covered by their presentations than on topics not covered by their presentations (P < 0.005), regardless of presentation grade (P > 0.05) and question type (P > 0.05). These results demonstrate empirically that oral presentations are an effective learning tool.