Validation of a novel duplex ultrasound objective structured assessment of technical skills (DUOSATS) for arterial stenosis detection.

INTRODUCTION: Duplex ultrasound facilitates bedside diagnosis and hence timely patient care. Its uptake has been hampered by training and accreditation issues. We have developed an assessment tool for Duplex arterial stenosis measurement for both simulator and patient based training. METHODS: A novel assessment tool: duplex ultrasound assessment of technical skills was developed. A modified duplex ultrasound assessment of technical skills was used for simulator training. Novice, intermediate experience and expert users of duplex ultrasound were invited to participate. Participants viewed an instructional video and were allowed ample time to familiarize with the equipment. Participants' attempts were recorded and independently assessed by four experts using the modified duplex ultrasound assessment of technical skills. 'Global' assessment was also done on a four point Likert scale. Content, construct and concurrent validity as well as reliability were evaluated. RESULTS: Content and construct validity as well as reliability were demonstrated. The simulator had good satisfaction rating from participants: median 4; range 3-5. Receiver operator characteristic analysis has established a cut point of 22/ 34 and 25/ 40 were most appropriate for simulator and patient based assessment respectively. We have validated a novel assessment tool for duplex arterial stenosis detection. Further work is underway to establish transference validity of simulator training to improved skill in scanning patients. CONCLUSIONS: We have developed and validated duplex ultrasound assessment of technical skills for simulator training.

Superficial venous disease treatment--is there still a role for open surgery in 2011?

The treatment of superficial venous disease (commonly described as varicose veins by the general public) has remained relatively constant over the past 100 years until the refinements of endovenous treatments such as sclerotherapy and more recently, the development of endovenous ablation. This has radically changed the treatment profile of this disease with treatments easily administered and well tolerated even in those patients who would not be considered fit for open surgery previously. With the advent of day surgery and improved general and local anaesthetic techniques, venous surgery has forged a path towards the end goal of outpatient treatment with no requirement for inpatient stay. The end goal of all superficial venous surgery is an improvement in quality of life, and with such new treatments reducing the impact of the actual intervention, such gains are easier to make. This review assesses and presents the current literature describing superficial venous disease treatments covering all treatment modalities. With endovenous treatment, true ambulatory treatment is available, providing high quality treatment at speed and convenience for patients.

Self-assessment of technical skill in surgery: the need for expert feedback.

INTRODUCTION: Technical skill has been formally assessed in the Fellow of the European Board of Vascular Surgery Examinations (FEBVS) since 2002. The aim of this study was to examine the relationship between expert assessment and trainee self-assessment. MATERIALS AND METHODS: Forty-two examination candidates performed a saphenofemoral junction (SFJ) ligation and an anterior tibial anastomosis on a synthetic simulation. Each candidate was rated by two examiners using a validated rating scale for their generic surgical skill for both procedures. Candidates then anonymously rated their own performance using the same scale. Parametric tests were used in the statistical analysis; a P-value < 0.05 was considered significant. RESULTS: The maximum mark in each assessment was 40; 24 was considered a competent score. The interobserver correlation for examiners marks were high (SFJ ligation, alpha = 0.68; distal anastomosis, alpha = 0.76). Examiners' marks were averaged. The mean examiner score for the SFJ ligation station was 27.8 (SD = 4.1) with 36 candidates (85.8%) attaining a competent score. The mean self-assessment score for this station was 30.7 (SD = 4.66). The mean examiners' marks for the distal anastomosis station was 29.2 (SD = 4.2); 39 candidates (92.8%) attained a competent score. The mean self-assessment score was 32.1 (SD = 4.0). There was no correlation between examiner and self-assessment scores in either station (Pearson's correlation coefficient: SFJ, r = 0.045, P = NS); distal anastomosis, r = 0.089, P = NS). Bland and Altman plots assessed the agreement between examiner and self-assessment. These showed candidates marked themselves higher than examiners with a mean difference of 2.9 marks in each station. CONCLUSIONS: Candidates' self-assessment and expert independent assessment correlate poorly. Trainees overestimate their ability according to independent assessment; regular technical feedback during training is, therefore, essential.

Competence assessment of senior vascular trainees using a carotid endarterectomy bench model.

BACKGROUND: Competency-based assessment is being introduced to surgical training. The value of bench-top technical skills assessment using a synthetic carotid endarterectomy (CEA) model was evaluated in vascular trainees and consultants. METHODS: Forty-one surgeons (13 junior trainees, 15 senior trainees and 13 experienced consultants with experience of more than 50 CEAs) performed a three-throw knot-tying exercise on a jig and a CEA on the bench model. A composite score for knot-tying was calculated, incorporating electromagnetic motion analysis. CEA technical skill was assessed using validated rating scales by blinded video analysis. RESULTS: Senior trainees performed better than junior trainees in knot-tying (P = 0.025) as well as generic (P < 0.001) and procedural (P < 0.001) skills on CEA model assessment. There was no difference between senior trainees and consultants on any of these measures. The CEA model interobserver reliability was high for all rating scales (generic alpha = 0.974, procedural alpha = 0.952, end-product alpha = 0.976). CONCLUSION: Senior trainees achieved the same score as consultants, suggesting a similar level of basic technical skill and knowledge required to perform CEA, and were significantly better than junior trainees. Performance on the bench model could provide an early assessment for suitability to proceed to operative training in a competency-based training and assessment programme.

AAA benchmarking by Dr Foster: a cause for concern?

INTRODUCTION: The publication of interpretable performance data for hospitals is an important service. In November 2002, the medical benchmarking company Dr Foster published a league table based on the results of abdominal aortic aneurysm (AAA) repair. The purpose of our study was to establish the validity of the data used in benchmarking. PATIENTS AND METHODS: Data on elective infra-renal AAA (IRAAA) repair was obtained from three sources. Data used by Dr Foster was based on the hospital PAS system. The databases for both Dr Foster and PAS were analysed and cross-referenced to the vascular unit database maintained by a separately employed audit co-ordinator. RESULTS: Of 395 total aortic aneurysm repairs, 223 (56%) were identified as elective IRAAA repairs on the unit database. Of these, 125 were identified on the PAS database and 115 on the Dr Foster database. The number of deaths was the same in both the unit and Dr Foster databases (n = 11) but the Dr Foster database included deaths in patients who had undergone juxtarenal (n = 1), Type III TAAA (n = 2) and Type IV TAAA (n = 4) repairs and omitted 7 deaths following IRAAA. The sensitivity and specificity for the PAS dataset was 0.51 and 0.93, respectively. For Dr Foster, the results were worse with a sensitivity and specificity of 0.41 and 0.86, respectively. The accuracy of the data was 0.6 and 0.69 for Dr Foster and PAS, respectively. Standardised mortality ratios (SMRs) were used to rank hospitals. Dr Foster's published SMR for elective AAA repair for our unit was 160. The actual SMR was 67. CONCLUSIONS: Robust and accurate published league tables should be supported and commended but currently available data appear to be misleading and may cause unnecessary concern to patients.

Training for carotid intervention: preparing the next generation.

BACKGROUND: Carotid interventions are performed to reduce the cumulative risk of stroke. The success of the procedure is dependent upon maintaining low operative risk. This article reviews the current state of training for both carotid endarterectomy (CEA) and carotid angioplasty and stenting (CAS). METHODS: Medline searches were performed to identify articles with the combination of the following key words: carotid, endarterectomy, stent, training, assessment and simulation. Manual searches of the reference lists and related papers was conducted. RESULTS: Training and assessment for CEA and CAS follows the traditional apprenticeship model. There is no formal training protocol or objective means of assessment for either carotid endarterectomy or stenting. Models and simulators to allow for training and assessment away from the operative theatre have been developed, and exist for both CEA and CAS. CONCLUSION: The technology exists to allow for both training and assessment of competency to take place in a controlled and objective environment for both CEA and CAS. The use of simulation needs to be robustly evaluated and assessed to both complement and augment existing training programs to ensure that the highest standards of care are maintained for treatment of carotid territory disease. Objective competency based training and assessment is no longer unattainable. Simulators augment this process and without them operative exposure is sporadic and crisis management infrequent.

The examination assessment of technical competence in vascular surgery.

BACKGROUND: The European Board of Surgery Qualification in Vascular Surgery is a pan-European examination for vascular surgeons who have attained a national certificate of completion of specialist training. A 2-year study was conducted before the introduction of a technical skills assessment in the examination. METHODS: The study included 30 surgeons: 22 candidates and eight examiners. They were tested on dissection (on a synthetic saphenofemoral junction model), anastomosis (on to anterior tibial artery of a synthetic leg model) and dexterity (a knot-tying simulator with electromagnetic motion analysis). Validated rating scales were used by two independent examiners. Composite knot-tying scores were calculated for the computerized station. The stations were weighted 35, 45 and 20 percent, respectively. RESULTS: Examiners performed better than candidates in the dissection (P<0.001), anastomosis (P=0.002) and dexterity (P=0.005) stations. Participants performed consistently in the examination (dissection versus anastomosis: r=0.79, P<0.001; dexterity versus total operative score: r=-0.73, P<0.001). Interobserver reliability was high (alpha=0.91). No correlation was seen between a candidate's technical skill and oral examination performance or logbook-accredited scores. CONCLUSION: Current surgical examinations do not address technical competence. This model appears to be a valid assessment of technical skills in an examination setting. The standards are set at a level appropriate for a specialist vascular surgeon.

Do workshops improved the technical skill of vascular surgical trainees?

AIMS: Adjuncts to conventional surgical training are needed in order to address the reduction in working hours. This purpose of this study was to objectively assess the efficacy of workshop training on simulators. METHODS: Fifteen consecutive participants of the European Vascular Workshop in 2003 and 2004 were recruited to this study. Participants performed a proximal anastomosis on a commercially available abdominal aortic aneurysm simulator, were then given intensive training on sophisticated models for 3 days and re-assessed. Pre- and post-course procedures were videotaped and independently reviewed by three assessors (tapes were blinded and in random order). The operative end product was similarly assessed. Four measures of technical skill were used: generic skill, procedural skill; a five point technical rating of the anastomosis (assessed using validated rating scales) and procedure time. Non-parametric tests were used in the statistical analysis. RESULTS: The video assessment scores for aneurysm repair increased significantly following completion of the course (p=0.006 and p=0.004 for generic and procedural skill, respectively). End product assessment scores increased significantly post-course (p=0.001) and participants performed aneurysm repair faster following the course (p<0.05). Inter-observer reliability ranged from alpha=0.84-0.98 for the three rating scales pre- and post-course. CONCLUSION: Objective improvements in technical performance follow intensive workshop training. Participants' perform better, faster, and with an improved end product following the course. Such adjuncts to training play an important part in a focused integrated programme that addresses reduced work hours.