Development and validation of a procedure-specific assessment tool for hands-on surgical training in congenital heart surgery.

BACKGROUND: Hands-on surgical simulation has been sought to address training limitations within congenital heart surgery (CHS). However, there is a need for objective assessment methods to measure surgeons' performance to justify its global adoption. This study aimed to validate a procedure-specific assessment tool for the simulation of the arterial switch operation on 3D-printed models and to evaluate the consistency of scoring among evaluators with different levels of experience in CHS. METHODS: Five "expert" and 5 "junior" surgeons performed the arterial switch procedure on 3D-printed models with transposition of the great arteries during 2 hands-on surgical training courses. Their performance was retrospectively assessed by 9 evaluators with varying experience in CHS (staff surgeons, resident surgeons, and non-MD raters). Assessments were done using 2 assessment tools: the Hands-On Surgical Training-Congenital Heart Surgery (HOST-CHS) assessment tool and the global rating scale (GRS). RESULTS: The HOST-CHS tool showed a higher interrater and intrarater reliability compared with the GRS. Total scores for expert surgeons were highly consistent across all evaluators. Non-MD raters' total scores for junior surgeons were slightly higher than those of residents and staff evaluators. All grades of evaluator were able to discriminate between junior and expert surgeons. CONCLUSIONS: This study demonstrates the development and validation of an objective, procedure-specific assessment tool for the arterial switch operation with consistency among evaluators with different experience. There is now a platform for quantifying and accurately evaluating performance, which will be highly beneficial in training and developing the next generation of congenital heart surgeons.

Quantitative assessment of technical performance during hands-on surgical training of the arterial switch operation using 3-dimensional printed heart models.

OBJECTIVE: Data supporting the use of hands-on simulation in congenital heart surgery are promising but primarily qualitative. This study aimed to demonstrate if there was an objective improvement in time and technical performance of the arterial switch procedure on 3-dimensional printed heart models by surgeons using a validated assessment method. METHODS: A total of 30 surgeons of varying experience performed the arterial switch procedure twice on 3-dimensional printed models with transposition of the great arteries during the Hands-on Surgical Training courses. Surgeons' performances were recorded and retrospectively assessed for both time and performance using the Hands-on Surgical Training-Congenital Heart Surgery tool, a validated procedure-specific assessment tool for the arterial switch. RESULTS: A total of 60 videos were scored. Eighty percent of surgeons (24/30) had improved from their first attempt. The mean total score of the first attempt performance compared with the second was 103 and 120, respectively, with a mean difference in score of 17 (95% confidence interval, 10-24). All surgeons were statistically significantly quicker in their second attempt. The mean time for the first attempt compared with the second was 1 hour, 28 minutes, 4 seconds and 1 hour, 5 minutes, and 45 seconds, respectively, with a mean difference of 0 hours, 22 minutes, 19 seconds (95% confidence interval, 0 hours, 15 minutes, 22 seconds to 0 hours, 25 minutes, 34 seconds). CONCLUSIONS: This is the first study to demonstrate an objective improvement in time and technical performance of the arterial switch procedure on 3-dimensional printed heart models. This supports the evidence that simulation in the form of deliberate practice with constructive, objective feedback is fundamental in the training of future congenital heart surgeons. These simulations and assessments should be incorporated to create structured, standardized training curricula within congenital heart surgery.