Evaluating validity evidence for 2 instruments developed to assess students' surgical skills in a simulated environment.

OBJECTIVE: To gather and evaluate validity evidence in the form of content and reliability of scores produced by 2 surgical skills assessment instruments, 1) a checklist, and 2) a modified form of the Objective Structured Assessment of Technical Skills (OSATS) global rating scale (GRS). STUDY DESIGN: Prospective randomized blinded study. SAMPLE POPULATION: Veterinary surgical skills educators (n =10) evaluated content validity. Scores from students in their third preclinical year of veterinary school (n = 16) were used to assess reliability. METHODS: Content validity was assessed using Lawshe's method to calculate the Content Validity Index (CVI) for the checklist and modified OSATS GRS. The importance and relevance of each item was determined in relation to skills needed to successfully perform supervised surgical procedures. The reliability of scores produced by both instruments was determined using generalizability (G) theory. RESULTS: Based on the results of the content validation, 39 of 40 checklist items were included. The 39-item checklist CVI was 0.81. One of the 6 OSATS GRS items was included. The 1-item GRS CVI was 0.80. The G-coefficients for the 40-item checklist and 6-item GRS were 0.85 and 0.79, respectively. CONCLUSION: Content validity was very good for the 39-item checklist and good for the 1-item OSATS GRS. The reliability of scores from both instruments was acceptable for a moderate stakes examination. IMPACT: These results provide evidence to support the use of the checklist described and a modified 1-item OSAT GRS in moderate stakes examinations when evaluating preclinical third-year veterinary students' technical surgical skills on low-fidelity models.

Comparison of 2 canine celiotomy closure models for training novice veterinary students.

OBJECTIVE: To compare a low-fidelity foam and fabric (FF) model to a high fidelity silicone (SI) model for teaching canine celiotomy closure. STUDY DESIGN: Prospective blinded comparison of learning outcomes. SAMPLE POPULATION: Second-year veterinary students who had never performed surgery as a primary surgeon (n = 46) and veterinarians experienced in performing canine celiotomy (n = 10). METHODS: Veterinary students performed a digitally recorded celiotomy closure on a canine cadaver before and after participation in 4 facilitated laboratory training sessions on their randomly assigned model. Recordings were scored by masked, trained educators with an 8-item task-specific rubric. Students completed surveys evaluating the models. Experienced veterinarians tested the models and provided feedback on their features. RESULTS: Completed pretest and posttest recordings were available for 38 of 46 students. Students' performance improved regardless of the model used to practice (P = .04). The magnitude of improvement did not differ between the 2 groups (P = .10). All students (n = 46) described their models favorably. Ninety percent of veterinarians thought both models were helpful for training students and gave similar ratings on all measures except for realism, which was rated higher for the SI model's skin (median, agree) compared with the FF model (median, neutral, P = .02). CONCLUSION: Model-based training was effective at improving students' surgical skills. Less experienced learners achieved similar skill gains after practicing with FF or SI models. CLINICAL SIGNIFICANCE: The acquisition of surgical skills required to perform celiotomy closure in companion animals occurs similarly well on models made of foam and fabric or of silicone, providing flexibility in model selection.

Development of a Dermatology Diagnostic Techniques Teaching Session: A Nine-Step Approach.

Creating effective learning experiences for veterinary clinical skills and procedures can be a challenging task. Drawing from both medical and veterinary education literature and personal experiences as practicing veterinarians and educators, the authors share nine key steps that describe the development process of a pre-clinical veterinary clinical skills teaching session. Relevant research and pedagogical principles supporting the effectiveness of the proposed nine-step process were identified and discussed. The aims of this article were to describe the development of a dermatology techniques teaching session and to provide the reader with a structured approach that can be used as a template to design or audit other clinical skills teaching sessions.

Evaluation of Veterinary Student Surgical Skills Preparation for Ovariohysterectomy Using Simulators: A Pilot Study.

This paper describes the development and evaluation of training intended to enhance students' performance on their first live-animal ovariohysterectomy (OVH). Cognitive task analysis informed a seven-page lab manual, 30-minute video, and 46-item OVH checklist (categorized into nine surgery components and three phases of surgery). We compared two spay simulator models (higher-fidelity silicone versus lower-fidelity cloth and foam). Third-year veterinary students were randomly assigned to a training intervention: lab manual and video only; lab manual, video, and $675 silicone-based model; lab manual, video, and $64 cloth and foam model. We then assessed transfer of training to a live-animal OVH. Chi-square analyses determined statistically significant differences between the interventions on four of nine surgery components, all three phases of surgery, and overall score. Odds ratio analyses indicated that training with a spay model improved the odds of attaining an excellent or good rating on 25 of 46 checklist items, six of nine surgery components, all three phases of surgery, and the overall score. Odds ratio analyses comparing the spay models indicated an advantage for the $675 silicon-based model on only 6 of 46 checklist items, three of nine surgery components, and one phase of surgery. Training with a spay model improved performance when compared to training with a manual and video only. Results suggested that training with a lower-fidelity/cost model might be as effective when compared to a higher-fidelity/cost model. Further research is required to investigate simulator fidelity and costs on transfer of training to the operational environment.