Development and validation of the TOCO-TURBT tool: a summative assessment tool that measures surgical competency in transurethral resection of bladder tumour.

BACKGROUND: The current shift towards competency-based residency training has increased the need for objective assessment of skills. In this study, we developed and validated an assessment tool that measures technical and non-technical competency in transurethral resection of bladder tumour (TURBT). METHODS: The 'Test Objective Competency' (TOCO)-TURBT tool was designed by means of cognitive task analysis (CTA), which included expert consensus. The tool consists of 51 items, divided into 3 phases: preparatory (n = 15), procedural (n = 21), and completion (n = 15). For validation of the TOCO-TURBT tool, 2 TURBT procedures were performed and videotaped by 25 urologists and 51 residents in a simulated setting. The participants' degree of competence was assessed by a panel of eight independent expert urologists using the TOCO-TURBT tool. Each procedure was assessed by two raters. Feasibility, acceptability and content validity were evaluated by means of a quantitative cross-sectional survey. Regression analyses were performed to assess the strength of the relation between experience and test scores (construct validity). Reliability was analysed by generalizability theory. RESULTS: The majority of assessors and urologists indicated the TOCO-TURBT tool to be a valid assessment of competency and would support the implementation of the TOCO-TURBT assessment as a certification method for residents. Construct validity was clearly established for all outcome measures of the procedural phase (all r > 0.5, p < 0.01). Generalizability-theory analysis showed high reliability (coefficient Phi ≥ 0.8) when using the format of two assessors and two cases. CONCLUSIONS: This study provides first evidence that the TOCO-TURBT tool is a feasible, valid and reliable assessment tool for measuring competency in TURBT. The tool has the potential to be used for future certification of competencies for residents and urologists. The methodology of CTA might be valuable in the development of assessment tools in other areas of clinical practice.

High educational impact of a national simulation-based urological curriculum including technical and non-technical skills.

BACKGROUND: Although simulation training is increasingly used to meet modern technology and patient safety demands, its successful integration within surgical curricula is still rare. The Dutch Urological Practical Skills (D-UPS) curriculum provides modular simulation-based training of technical and non-technical basic urological skills in the local hospital setting. This study aims to assess the educational impact of implementing the D-UPS curriculum in the Netherlands and to provide focus points for improvement of the D-UPS curriculum according to the participants. METHODS: Educational impact was assessed by means of qualitative individual module-specific feedback and a quantitative cross-sectional survey among residents and supervisors. Twenty out of 26 Dutch teaching hospitals participated. The survey focussed on practical aspects, the D-UPS curriculum in general, and the impact of the D-UPS curriculum on the development of technical and non-technical skills. RESULTS: A considerable survey response of 95 % for residents and 76 % for supervisors was obtained. Modules were attended by junior and senior residents, supervised by a urologist, and peer teaching was used. Ninety percent of supervisors versus 67 % of residents judged the D-UPS curriculum as an important addition to current residency training (p = 0.007). Participants' aggregated general judgement of the modules showed a substantial percentage favorable score (M ± SE: 57 ± 4 %). The impact of training on, e.g., knowledge of materials/equipment and ability to anticipate on complications was high, especially for junior residents (77 ± 5 and 71 ± 7 %, respectively). Focus points for improvement of the D-UPS curriculum according to the participants include adaptation of the training level to residents' level of experience and focus on logistics. CONCLUSION: The simulation-based D-UPS curriculum has a high educational impact. Residents and supervisors consider the curriculum to be an important addition to current residency training. Focus points for improvement of the D-UPS curriculum according to the participants include increased attention to logistics and integration of a spiral learning approach.

The Simbla TURBT Simulator in Urological Residency Training: From Needs Analysis to Validation.

OBJECTIVE: To investigate the value of the physical Simbla Transurethral Resection of a Bladder Tumor (TURBT) simulator as an educational tool within urological residency training, by means of a training needs analysis (TNA) and assessment of its feasibility, acceptability, and face, content, and construct validity. METHODS: To analyze the training needs for TURBT, procedural steps and pitfalls were identified and the TNA was completed during an expert consensus meeting. Participants (n = 76) were divided into three groups based on their experience in TURBT: novices, intermediates, and experts. Participants performed two standardized TURBT procedures on the simulator. Face validity and content validity, as well as feasibility and acceptability, were assessed with a quantitative survey. Construct validity was assessed by comparing the performance of novices, intermediates, and experts on resection time, quality of tumor resection, and overall performance. RESULTS: Of the 21 procedural steps and 17 pitfalls defined in TNA, 13 steps and 8 pitfalls were covered by the Simbla. Participants rated the Simbla's overall realism (face validity) with a score of 8 of 10 (range 6-9). The simulator was judged to be most useful (content validity) for learning eye-hand coordination: score 8 (6-10). All aspects regarding realism and usefulness were rated above the acceptability threshold of 6/10. Intermediates (100%) and experts (96%) considered the Simbla to be a useful educational tool within the urological curriculum. Resection time was longer for novices than for experts (p < 0.05; construct validity). In addition, the overall performance of novices was rated lower compared with intermediates and experts, and novices showed more irradical resections and bladder perforations (all p < 0.05). CONCLUSIONS: The Simbla TURBT simulator is a valid, feasible, and acceptable educational tool for training procedural skills and may be implemented in the urological curriculum to complement learning in clinical practice. TNA is valuable in defining training objectives and evaluating the educational value of a simulator.

High acceptability of a newly developed urological practical skills training program.

BACKGROUND: Benefits of simulation training are widely recognized, but its structural implementation into urological curricula remains challenging. This study aims to gain insight into current and ideal urological practical skills training and presents the outline of a newly developed skills training program, including an assessment of the design characteristics that may increase its acceptability. METHODS: A questionnaire was sent to the urology residents (n = 87) and program directors (n = 45) of all Dutch teaching hospitals. Open- and close-ended questions were used to determine the views on current and ideal skills training and the newly developed skills training program. Eight semi-structured interviews were conducted with 39 residents and 15 program directors. All interviews were audiotaped, fully transcribed, and thereafter analyzed. RESULTS: Response was 87.4% for residents and 86.7% for program directors. Residents appeared to be still predominantly trained 'by doing'. Structured practical skills training in local hospitals takes place according to 12% of the residents versus 44% of the program directors (p < 0.001). Ideally, residents prefer to practice certain procedures on simulation models first, especially in endourology. The majority of residents (92%) and program directors (87%) approved of implementing the newly developed skills training program (p = 0.51). 'Structured scheduling', 'use of peer teaching' and 'high fidelity models' were indicated as design characteristics that increase its acceptability. CONCLUSIONS: Current urological residency training consists of patient-related 'learning by doing', although more practice on simulation models is desired. The acceptability of implementing the presented skills-training program is high. Design characteristics that increase its acceptability are structured scheduling, the use of peer teaching and high fidelity models.

Patient Safety Risks of Basic Urological Procedures Performed by Junior and Senior Residents.

OBJECTIVE: To investigate the current performance of urological residents regarding basic urological procedures in relation to patient safety issues and the identification of specific training needs. DESIGN: Observational data of 146 urethrocystoscopies (UCSs), 27 transrectal ultrasounds of the prostate (TRUSs), 38 transrectal ultrasound-guided prostatic biopsies (TRUSPs), and 30 transurethral resections of bladder tumor (TURBTs) were collected. Performance was evaluated using scoring lists including details on completeness of procedural steps, level of independence, time, and the incidence of unintended events. The causal factors contributing to the unintended events were identified by 2 expert urologists and classified according to the recognized PRISMA method. SETTING: This study was performed in 5 teaching hospitals in the Netherlands. PARTICIPANTS: We included 11 junior residents and 5 senior residents in urology in the final study cohort. RESULTS: Senior residents showed a lower degree of completeness in material usage than junior residents did during UCS (p < 0.01) and in preparation, material usage, and procedure during TRUSP (all p < 0.05). In UCS and TURBT, senior residents received significantly less feedback than junior residents did (both p < 0.01). Incidence of unintended events for junior vs senior residents was 11% and 4% in UCS, 0% and 7% in transrectal ultrasound of the prostate, 36% and 62% in TRUSP, and 41% and 23% in TURBT, respectively. Overall, unintended events were mainly caused by human factors, in particular, verification and skills-based issues. CONCLUSION: Present performance of basic urological procedures involves a high percentage of unintended events, especially in TRUSP and TURBT, which are mainly caused by human factors and are a potential threat for patient safety. Junior residents are less independent but more thorough in the performance of UCS and TRUSP than senior residents are. Targeted skills training including assessment should be implemented before privileges for independent practice are granted to reduce the incidence of unintended events and optimize patient safety.