Collecting Validity Evidence for the Assessment of Mastery Learning in Simulation-Based Ultrasound Training.

PURPOSE: To collect validity evidence for the assessment of mastery learning on a virtual reality transabdominal ultrasound simulator. MATERIALS AND METHODS: We assessed the validity evidence using Messick's framework for validity. The study included 20 novices and 9 ultrasound experts who all completed 10 obstetric training modules on a transabdominal ultrasound simulator that provided automated measures of performance for each completed module (i. e., simulator metrics). Differences in the performance of the two groups were used to identify simulator metrics with validity evidence for the assessment of mastery learning. The novices continued to practice until they had attained mastery learning level. RESULTS: One-third of the simulator metrics discriminated between the two groups. The median simulator scores from a maximum of 40 metrics were 17.5 percent (range 0 - 45.0 percent) for novices and 90.0 percent (range 85.0 - 97.5) for experts, p < 0.001. Internal consistency was high, with a Cronbach's alpha value of 0.98. The test/retest reliability gave an intra-class correlation coefficient (ICC) of 0.62 for novices who reached the mastery learning level twice. Novices reached the mastery learning level within a median of 4 attempts (range 3 - 8) corresponding to a median of 252 minutes of simulator training (range 211 - 394 minutes). CONCLUSION: This study found that validity evidence for the assessment of mastery learning in simulation-based ultrasound training can be demonstrated and that ultrasound novices can attain mastery learning levels with less than 5 hours of training. Only one-third of the standard simulator metrics discriminated between different levels of competence.

Sustained effect of simulation-based ultrasound training on clinical performance: a randomized trial.

OBJECTIVE: To study the effect of initial simulation-based transvaginal sonography (TVS) training compared with clinical training only, on the clinical performance of residents in obstetrics and gynecology (Ob-Gyn), assessed 2 months into their residency. METHODS: In a randomized study, new Ob-Gyn residents (n = 33) with no prior ultrasound experience were recruited from three teaching hospitals. Participants were allocated to either simulation-based training followed by clinical training (intervention group; n = 18) or clinical training only (control group; n = 15). The simulation-based training was performed using a virtual-reality TVS simulator until an expert performance level was attained, and was followed by training on a pelvic mannequin. After 2 months of clinical training, one TVS examination was recorded for assessment of each resident's clinical performance (n = 26). Two ultrasound experts blinded to group allocation rated the scans using the Objective Structured Assessment of Ultrasound Skills (OSAUS) scale. RESULTS: During the 2 months of clinical training, participants in the intervention and control groups completed an average ± SD of 58 ± 41 and 63 ± 47 scans, respectively (P = 0.67). In the subsequent clinical performance test, the intervention group achieved higher OSAUS scores than did the control group (mean score, 59.1% vs 37.6%, respectively; P < 0.001). A greater proportion of the intervention group passed a pre-established pass/fail level than did controls (85.7% vs 8.3%, respectively; P < 0.001). CONCLUSION: Simulation-based ultrasound training leads to substantial improvement in clinical performance that is sustained after 2 months of clinical training. © 2015 The Authors. Ultrasound in Obstetrics & Gynecology published by John Wiley & Sons Ltd on behalf of the International Society of Ultrasound in Obstetrics and Gynecology.

Assessment of performance measures and learning curves for use of a virtual-reality ultrasound simulator in transvaginal ultrasound examination.

OBJECTIVE: To assess the validity and reliability of performance measures, develop credible performance standards and explore learning curves for a virtual-reality simulator designed for transvaginal gynecological ultrasound examination. METHODS: A group of 16 ultrasound novices, along with a group of 12 obstetrics/gynecology (Ob/Gyn) consultants, were included in this experimental study. The first two performances of the two groups on seven selected modules on a high-fidelity ultrasound simulator were used to identify valid and reliable metrics. Performance standards were determined and novices were instructed to continue practicing until they attained the performance level of an expert subgroup (n = 4). RESULTS: All 28 participants completed the selected modules twice and all novices reached the expert performance level. Of 153 metrics, 48 were able to be used to discriminate between the two groups' performance. The ultrasound novices scored a median of 43.8% (range, 17.9-68.9%) and the Ob/Gyn consultants scored a median of 82.8% (range, 60.4-91.7%) of the maximum sum score (P < 0.001). The ultrasound novices reached the expert level (88.4%) within a median of five iterations (range, 5-6), corresponding to an average of 219 min (range, 150-251 min) of training. The test/retest reliability was high, with an intraclass correlation coefficient of 0.93. CONCLUSIONS: Competence in the performance of gynecological ultrasound examination can be assessed in a valid and reliable way using virtual-reality simulation. The novices' performance improved with practice and their learning curves plateaued at the level of expert performance, following between 3 and 4 h of simulator training.

Evaluation of pig lungs following an experimental challenge with Actinobacillus pleuropneumoniae serotype 1 and 5 in pigs inoculated with either hemolysin protein and/or outer membrane proteins.

Histopathological changes were compared in pigs challenged with Actinobacillus pleuropneumoniae serotype 1 and serotype 5 after inoculation with subunit vaccines. The vaccines consisted of outer membrane protein and/or hemolysin protein isolated from Actinobacillus pleuropneumoniae serotype 1 or both subunits combined. Twenty-seven cross-bred pigs were separated into six groups: Groups I and IV were vaccinated and boostered with 1500 micrograms outer membrane protein; Groups II and V were vaccinated and boostered with 250 micrograms hemolysin protein; Groups III and VI were vaccinated and boostered with a combination of 1500 micrograms outer membrane protein and 250 micrograms hemolysin protein. Groups I, II and III were challenged with A. pleuropneumoniae serotype 1; and Groups IV, V and VI were challenged with A. pleuropneumoniae serotype 5. Groups III and VI demonstrated the least severe lung tissue damage, with significantly lower (P < 0.05) lung involvement as compared to the other groups. Lesions were noted in all six groups. These results showed that complete protection against A. pleuropneumoniae infection was not feasible using a subunit vaccine consisting of just outer membrane protein and hemolysin protein, and that some cross-protection did occur.