A Descriptive Analysis of the Use of Workplace-Based Assessments in UK Surgical Training.

BACKGROUND: Workplace-based assessments (WBAs) were introduced formally in the UK in 2007. The aim of the study was to describe the use of WBAs by UK surgical trainees and examine variations by training region, specialty, and level of training. METHODS: The database of the Intercollegiate Surgical Curriculum Programme was examined for WBAs between August 2007 and July 2013, with in-depth analysis of 2 periods: August 2011 to July 2012 and August 2012 to July 2013. RESULTS: The numbers of validated WBAs per trainee per year increased more than 7-fold, from median 6 per trainee in 2007 to 2008, to 39 in 2011 to 2012, and 44 in 2012 to 2013. In the period 2011 to 2012, 58.4% of core trainees completed the recommended 40 WBAs, with only 38.1% of specialty trainees achieving 40 validated WBAs. In the period 2012 to 2013, these proportions increased to 67.7% and 57.0% for core and specialty trainees, respectively. Core trainees completed more WBAs per year than specialty trainees in the same training region. London core trainees completed the highest numbers of WBAs in both the periods 2011 to 2012 (median 67) and 2012 to 2013 (median 74). There was a peak in WBAs completed by London specialty trainees in the period 2012 to 2013 (median 63). The most validated WBAs were completed by ST1/CT1 (specialty surgical training year, core surgical training year), with a gradual decrease in median WBAs to ST4, followed by a plateau; in the period 2012 to 2013, there was an increase in WBAs at ST8. Core surgical trainees complete ~50% "operative" (procedure-based assessment/direct observation of procedural skills) and ~50% "nonoperative" assessments (case-based discussion/clinical evaluation exercise). During specialty training, procedure-based assessments represented ~46% of WBAs, direct observation of procedural skills 11.2%, case-based discussion ~23%, and clinical evaluation exercise ~15%. CONCLUSIONS: UK surgical trainees are, on an average, undertaking 1 WBA per week. Variation exists in use of WBAs between training regions. Core trainees tend to use the spectrum of WBAs more frequently than their senior colleagues do. Further work is required to examine the role of WBAs in assessment, and engagement and training of trainers in processes and validation of WBAs.

Workplace-based assessment in surgical training: experiences from the Intercollegiate Surgical Curriculum Programme.

BACKGROUND: The Intercollegiate Surgical Curriculum Programme was launched in the United Kingdom in 2007. At its heart was the reliance upon clear, defined curricula, competence-based training and the use of workplace-based assessments to assess the competence. The principle assessments used were Case-based Discussion, Procedure-based Assessments (PBA), Direct Observation of Procedural Skills, and Clinical Evaluation Exercise and a Multisource Feedback tool. METHODS: We report the initial experience with that system, and most importantly, the experience with workplace-based assessment. RESULTS: Themes include issues around faculty development, misuse of assessments, inappropriate timing of assessments, concerns about validity and reliability of the assessments and concerns about the actual process of workplace-based assessments. Of the assessments, the PBA performed best. CONCLUSIONS: As a consequence, there has been an increased focus upon faculty development, while some of the assessments have been redesigned in line with the PBA. A global rating scale has been introduced that uses clinical anchors. The rating scales have also been altered with a reduction in the number of ratings while an enhanced description of the complexity of the case has been introduced within the Case-based Discussion and the Clinical Evaluation Exercise. A re-evaluation will take place in the near future.

Workplace-based assessment: assessing technical skill throughout the continuum of surgical training.

The Royal Colleges of Surgeons and Surgical Specialty Associations in the UK have introduced competence-based syllabi and curricula for surgical training. The syllabi of the Intercollegiate Surgical Curriculum Programme (ISCP) and Orthopaedic Curriculum and Assessment Programme (OCAP) define the core competencies, that is, the observable and measureable behaviours required of a surgical trainee. The curricula define when, where and how these will be assessed. Procedure-based assessment (PBA) has been adopted as the principal method of assessing surgical skills. It combines competencies specific to the procedure with generic competencies such as safe handling of instruments. It covers the entire procedure, including preoperative and postoperative planning. A global summary of the level at which the trainee performed the assessed elements of the procedure is also included. The form has been designed to be completed quickly by the assessor (clinical supervisor) and fed-back to the trainee between operations. PBA forms have been developed for all index procedures in all surgical specialties. The forms are intended to be used as frequently as possible when performing index procedures, as their primary aim is to aid learning. At the end of a training placement the aggregated PBA forms, together with the logbook, enable the Educational Supervisor and/or Programme Director to make a summary judgement about the competence of a trainee to perform index procedures to a given standard.