International Implementation of a PROficiency based StePwise Endovascular Curricular Training (PROSPECT) in Daily Practice.

OBJECTIVE: A randomised controlled trial (RCT) showed superior real life endovascular performance of trainees after PROficiency based StePwise Endovascular Curricular Training (PROSPECT) compared with traditional training. This study aimed to determine whether PROSPECT can be implemented routinely in surgical training (registry) and to identify the associated effects, facilitators, and barriers to implementation. METHODS: This was a multicentre prospective registry to evaluate real life PROSPECT implementation effects. Cognitive ability was tested with a Multiple Choice Question test (MCQ) (max. score 20 points), while technical skills were assessed with a Global Rating Scale (GRS) (max. score 55 points), Examiner's Checklist (max, score 85 points), and simulator metrics. A cross sectional anonymous survey for both trainees and faculty evaluated the programme's implementation. RESULTS: The PROSPECT registry was implemented in five centres in four countries over a period of three years. Only 17 of 48 starting trainees completed PROSPECT, resulting in a dropout rate of 65%. Participants who completed the programme showed significant improvement in the GRS (median 26 vs. 44, p = .018) and Examiner's Checklist (median 53.5 vs. 80, p = .028) after the programme. The survey was completed by 13/17 faculty members (76%) and 38/57 trainees who participated either in the initial PROSPECT RCT or the registry (67%). PROSPECT was perceived to provide more endovascular training opportunities, but non-availability of logistical/technical support, lack of training time within the working schedule, and little trainee motivation hindered broad implementation. CONCLUSION: PROSPECT has a significant effect on performance, and trainees and faculty agree that PROSPECT is a valuable addition to surgical training. However, external and internal factors pose significant barriers to integration of this simulation based programme into daily practice leading to a high dropout rate. To integrate PROSPECT into contemporary surgical training, it should be a prerequisite to treating real patients and protected training time should be provided, combined with accessible support.

Training non-intensivist doctors to work with COVID-19 patients in intensive care units.

BACKGROUND: Due to an expected surge of COVID-19 patients in need of mechanical ventilation, the intensive care capacity was doubled at Rigshospitalet, Copenhagen, in March 2020. This resulted in an urgent need for doctors with competence in working with critically ill COVID-19 patients. A training course and a theoretical test for non-intensivist doctors were developed. The aims of this study were to gather validity evidence for the theoretical test and explore the effects of the course. METHODS: The 1-day course was comprised of theoretical sessions and hands-on training in ventilator use, hemodynamic monitoring, vascular access, and use of personal protective equipment. Validity evidence was gathered for the test by comparing answers from novices and experts in intensive care. Doctors who participated in the course completed the test before (pretest), after (posttest), and again within 8 weeks following the course (retention test). RESULTS: Fifty-four non-intensivist doctors from 15 different specialties with a wide range in clinical experience level completed the course. The test consisted of 23 questions and demonstrated a credible pass-fail standard at 16 points. Mean pretest score was 11.9 (SD 3.0), mean posttest score 20.6 (1.8), and mean retention test score 17.4 (2.2). All doctors passed the posttest. CONCLUSION: Non-intensivist doctors, irrespective of experience level, can acquire relevant knowledge for working in the ICU through a focused 1-day evidence-based course. This knowledge was largely retained as shown by a multiple-choice test supported by validity evidence. The test is available in appendix and online.

Simulation Based Training and Assessment in Open Vascular Surgery: A Systematic Review.

OBJECTIVE: The aim of this study was to systematically review the literature and give evidence based recommendations for future initiatives for simulation based training (SBT) and assessment in open vascular surgery. DATA SOURCES: PubMed, Embase, and the Cochrane Library. REVIEW METHODS: A systematic review of PubMed, Embase, and the Cochrane Library was performed, with the last search on 31 March 2020, to identify studies describing SBT and assessment in open vascular surgery. Kirkpatrick's levels for efficacy of training were evaluated. Validity evidence for assessment tools was evaluated according to the recommended contemporary framework by Messick. RESULTS: Of 2 844 studies, 51 were included for data extraction. A high degree of heterogeneity in reporting standards and varying types of simulation was found. Vascular anastomosis was the most frequently simulated technical skill (43%). Assessment was mostly carried out using the Objective Structured Assessment of Technical Skills (55%). Validity evidence for assessment tools was found using outdated frameworks, and only one study used Messick's framework. Self directed training is valuable, the low trainer to trainee ratio is important to maximise efficiency, and experienced vascular surgeons are the most effective trainers. CONCLUSION: Carefully designed and structured SBT is effective and can improve technical skills, especially in less experienced trainees. However, the supporting evidence lacks homogeneity in the reporting standards and types of simulations. Pass/fail standards that support proficiency based learning and studies investigating skills transfer should be the focus in future studies. Validity evidence of assessment tools needs to be addressed using contemporary frameworks.

European Society for Vascular Surgery (ESVS) Certification of Theoretical and Practical Competences in Basic Vascular Ultrasound: Validity Investigation of the Assessment Tools.

OBJECTIVE: The aim of this study was to gather validity evidence for the Assessment of basic Vascular Ultrasound Expertise (AVAUSE) tool, and to establish a pass/fail score for each component, to support decisions for certification. METHODS: A cross sectional validation study performed during the European Society for Vascular Surgery's annual meeting. Validity evidence was sought for the theoretical test and two practical tests based on Messick's framework. The participants were vascular surgeons, vascular surgical trainees, sonographers, and nurses with varying experience levels. Five vascular ultrasound experts developed the theoretical and two practical test components of the AVAUSE tool for each test component. Two stations were set up for carotid examinations and two for superficial venous incompetence (SVI) examinations. Eight raters were assigned in pairs to each station. Three methods were used to set pass/fail scores: contrasting groups' method; rater consensus; and extended Angoff. RESULTS: Nineteen participants were enrolled. Acceptable internal consistency reliability (Cronbach's alpha) for the AVAUSE theoretical (0.93), carotid (0.84), and SVI (0.65) practical test were shown. In the carotid examination, inter-rater reliability (IRR) for the two rater pairs was good: 0.68 and 0.78, respectively. The carotid scores correlated significantly with years of experience (Pearson's r = 0.56, p = .013) but not with number of examinations in the last five years. For SVI, IRR was excellent at 0.81 and 0.87. SVI performance scores did not correlate with years of experience and number of examinations. The pass/fail score set by the contrasting groups' method was 29 points out of 50. The rater set pass/fail scores were 3.0 points for both carotid and SVI examinations and were used to determine successful participants. Ten of 19 participants passed the tests and were certified. CONCLUSION: Validity evidence was sought and established for the AVAUSE comprehensive tool, including pass/fail standards. AVAUSE can be used to assess competences in basic vascular ultrasound, allowing operators to progress towards independent practice.

Ensuring Competency in Open Aortic Aneurysm Repair - Development and Validation of a New Assessment Tool.

OBJECTIVE: The aims of this study were to develop a procedure specific assessment tool for open abdominal aortic aneurysm (AAA) repair, gather validity evidence for the tool and establish a pass/fail standard. METHODS: Validity was studied based on the contemporary framework by Messick. Three vascular surgeons experienced in open AAA repair and an expert in assessment and validation within medical education developed the OPEn aortic aneurysm Repair Assessment of Technical Expertise (OPERATE) tool. Vascular surgeons with varying experiences performed open AAA repair in a standardised simulation based setting. All procedures were video recorded with the faces anonymised and scored independently by three experts in a mutual blinded setup. The Angoff standard setting method was used to establish a credible pass/fail score. RESULTS: Sixteen novices and nine experienced open vascular surgeons were enrolled. The OPERATE tool achieved high internal consistency (Cronbach's alpha .92) and inter-rater reliability (Cronbach's alpha .95) and was able to differentiate novices and experienced surgeons with mean scores (higher score is better) of 13.4 ± 12 and 25.6 ± 6, respectively (p = .01). The pass/fail score was set high (27.7). One novice passed the test while six experienced surgeons failed. CONCLUSION: Validity evidence was established for the newly developed OPERATE tool and was able to differentiate between novices and experienced surgeons providing a good argument that this tool can be used for both formative and summative assessment in a simulation based environment. The high pass/fail score emphasises the need for novices to train in a simulation based environment up to a certain level of competency before apprenticeship training in the clinical environment under the tutelage of a supervisor. Familiarisation with the simulation equipment must be ensured before performance is assessed as reflected by the low scores in the experienced group's first attempt.

Achieving Consensus to Define Curricular Content for Simulation Based Education in Vascular Surgery: A Europe Wide Needs Assessment Initiative.

OBJECTIVE: To gather consensus among European educators about technical procedures that should be included in a future simulation based curriculum in vascular surgery. METHODS: A three round modified Delphi survey was initiated among 189 key opinion leaders (KOL) from 34 countries across Europe who were identified according to their positions in the European Society for Vascular Surgery, the European Journal of Vascular and Endovascular Surgery, and Union Européenne des Médecins Spécialistes Section and Board of Vascular Surgery. The first round was a brainstorming phase to identify technical procedures that a newly qualified vascular surgeon should be able to perform. The answers were analysed qualitatively. The second round investigated how often the identified procedures are performed, the number of vascular surgeons that should be able to perform these procedures, whether the procedures pose a risk to the patients, and whether simulation based education (SBE) is feasible. In the third round, elimination and re-ranking of procedures were performed. Only procedures that gained more than 70% support were included. An international steering group consisting of open and endovascular surgeons and medical educators governed the process. RESULTS: Response rates in the three rounds were 75% (142/189), 89% (126/142), and 85% (107/126), respectively. In the final prioritised list of 30 technical procedures for SBE, the top five procedures focus on basic open vascular skills, basic endovascular skills, vascular imaging interpretation, femoral endarterectomy, and open peripheral bypass. Twenty-six procedures were eliminated, including peripheral pressure measurement, wound management, open management of complications, major amputations, and highly advanced endovascular skills. CONCLUSION: The prioritised list of technical procedures from this ESVS supported project could be used to guide planning and development of future SBE programs to meet the needs of vascular surgeons across Europe.