Assessing colonoscopic inspection skill using a virtual withdrawal simulation: a preliminary validation of performance metrics.

BACKGROUND: The effectiveness of colonoscopy for diagnosing and preventing colon cancer is largely dependent on the ability of endoscopists to fully inspect the colonic mucosa, which they achieve primarily through skilled manipulation of the colonoscope during withdrawal. Performance assessment during live procedures is problematic. However, a virtual withdrawal simulation can help identify and parameterise actions linked to successful inspection, and offer standardised assessments for trainees. METHODS: Eleven experienced endoscopists and 18 endoscopy novices (medical students) completed a mucosal inspection task during three simulated colonoscopic withdrawals. The two groups were compared on 10 performance metrics to preliminarily assess the validity of these measures to describe inspection quality. Four metrics were related to aspects of polyp detection: percentage of polyp markers found; number of polyp markers found per minute; percentage of the mucosal surface illuminated by the colonoscope (≥0.5 s); and percentage of polyp markers illuminated (≥2.5 s) but not identified. A further six metrics described the movement of the colonoscope: withdrawal time; linear distance travelled by the colonoscope tip; total distance travelled by the colonoscope tip; and distance travelled by the colonoscope tip due to movement of the up/down angulation control, movement of the left/right angulation control, and axial shaft rotation. RESULTS: Statistically significant experienced-novice differences were found for 8 of the 10 performance metrics (p's < .005). Compared with novices, experienced endoscopists inspected more of the mucosa and detected more polyp markers, at a faster rate. Despite completing the withdrawals more quickly than the novices, the experienced endoscopists also moved the colonoscope more in terms of linear distance travelled and overall tip movement, with greater use of both the up/down angulation control and axial shaft rotation. However, the groups did not differ in the number of polyp markers visible on the monitor but not identified, or movement of the left/right angulation control. All metrics that yielded significant group differences had adequate to excellent internal consistency reliability (α = .79 to .90). CONCLUSIONS: These systematic differences confirm the potential of the simulated withdrawal task for evaluating inspection skills and strategies. It may be useful for training, and assessment of trainee competence.

A novel training device for tip control in colonoscopy: preliminary validation and efficacy as a training tool.

BACKGROUND: Effective control of the colonoscope tip is one of the most fundamental components of colonoscopy skill. Mastering fine tip control can be problematic for novice trainees, yet no validated training regimes exist for developing this specific skill component in isolation. We aimed to conduct a preliminary validation of a novel training device for colonoscopic tip control, and to assess its efficacy as a training tool. METHODS: In study 1 (validation), 13 experienced colonoscopists and 16 novices used a colonoscope to accurately track 28 targets on each of four concave "training surfaces" as quickly as possible, and we compared their performance. In study 2 (pre-post-training study), another 16 novices were tested before and after a six-session training program. In both studies, the main outcome measurements were completion time (measured automatically by the device) and variability of individual performance (the SD of each individual's completion times across trials). RESULTS: Compared with novices, experienced colonoscopists were faster (P < 0.0001) and their performances less variable (P < 0.0001). With training, novices became faster (P < 0.0001) and more consistent (P = 0.003), and these improvements also generalized to novel training surfaces (P's < 0.01). After training, the novices' tip control performance was indistinguishable from that of the experienced colonoscopists (P's > 0.05). The composite measures of completion time used in both studies all had acceptable to excellent internal consistency reliability (α's ranged from 0.72 to 0.93). CONCLUSIONS: We found that performance measures derived from using the device to assess skill can discriminate between experienced colonoscopists and novices in terms of their ability to control and guide the colonoscope tip precisely, providing preliminary evidence to support the construct validity of the metrics. The device is also an effective training tool for this fundamental component of colonoscopy skill.

Assessment of colorectal polyp recognition skill: development and validation of an objective test.

BACKGROUND: The quality of colonoscopy is known to vary. The extent to which colonoscopists can recognize the presence of subtle colorectal lesions by visually distinguishing them from the surrounding mucosa (i.e., polyp recognition skill) may be one of several attributes that influence polyp detection rates. The aim of the present study was to develop and validate the first objective test of polyp recognition skill. METHODS: Validation study. Twenty-eight experienced colonoscopists and eighty novices took a preliminary 280-item computer-based polyp recognition test. Items were genuine endoscopic images which participants assessed for the presence of "likely polyps." Half included clinically identified polyps. Participants clicked on a suspected lesion or a button marked "no likely polyp", and the main outcome measures were accuracy and response latency. The best items were selected for the final 50-item test. RESULTS: In the preliminary test, experienced colonoscopists correctly identified more polyps than novices (P < .0001) and better discriminated between clinically identified polyps and non-polyp features (as measured by d', P < .0001). For polyp items, the experienced group also responded faster (P < .01). Effect sizes were large for accuracy (Cohen's d = 3.22) and d' (Cohen's d = 3.22). The 50 final test items produced comparable results for accuracy, d', and response latency. For both versions of the test, score scale reliability was high for both polyp and non-polyp items (α = .82 to .97). CONCLUSIONS: The observed experienced-novice differences support the construct validity of the performance measures derived from the tests, indicating that polyp recognition skill can be quantified objectively. The final test may potentially be used to assess trainees, but test sensitivity may be insufficient to make fine-grained distinctions between different skill levels among experienced colonoscopists. More sensitive future tests may provide a valuable supplement to clinical detection rates, allowing objective comparisons between skilled colonoscopists.

The efficacy of training insertion skill on a physical model colonoscopy simulator.

Background and study aims: Prior research supports the validity of performance measures derived from the use of a physical model colonoscopy simulator - the Kyoto Kagaku Colonoscope Training Model (Kyoto Kagaku Co. Ltd., Kyoto, Japan) - for assessing insertion skill. However, its use as a training tool has received little research attention. We assessed the efficacy of a brief structured program to develop basic colonoscope insertion skill through unsupervised practice on the model. Participants and methods: This was a training study with pretesting and post-testing. Thirty-two colonoscopy novices completed an 11-hour training regime in which they practiced cases on the model in a colonoscopy simulation research laboratory. They also attempted a series of test cases before and after training. For each outcome measure (completion rates, time to cecum and peak force applied to the model), we compared trainees' post-test performance with the untrained novices and experienced colonoscopists from a previously-reported validation study. Results: Compared with untrained novices, trained novices had higher completion rates and shorter times to cecum overall (Ps < .001), but were out-performed by the experienced colono-scopists on these metrics (Ps < .001). Nevertheless, their performance was generally closer to that of the experienced group. Overall, trained novices did not differ from either experience-level comparison group in the peak forces they applied (P > .05). We also present the results broken down by case. Conclusions: The program can be used to teach trainees basic insertion skill in a more or less self-directed way. Individuals who have completed the program (or similar training on the model) are better prepared to progress to supervised live cases.

A competency framework for colonoscopy training derived from cognitive task analysis techniques and expert review.

BACKGROUND: Colonoscopy is a difficult cognitive-perceptual-motor task. Designing an appropriate instructional program for such a task requires an understanding of the knowledge, skills and attitudes underpinning the competency required to perform the task. Cognitive task analysis techniques provide an empirical means of deriving this information. METHODS: Video recording and a think-aloud protocol were conducted while 20 experienced endoscopists performed colonoscopy procedures. "Cued-recall" interviews were also carried out post-procedure with nine of the endoscopists. Analysis of the resulting transcripts employed the constant comparative coding method within a grounded theory framework. The resulting draft competency framework was modified after review during semi-structured interviews conducted with six expert endoscopists. RESULTS: The proposed colonoscopy competency framework consists of twenty-seven skill, knowledge and attitude components, grouped into six categories (clinical knowledge; colonoscope handling; situation awareness; heuristics and strategies; clinical reasoning; and intra- and inter-personal). CONCLUSIONS: The colonoscopy competency framework provides a principled basis for the design of a training program, and for the design of formative assessment to gauge progress towards attaining the knowledge, skills and attitudes underpinning the achievement of colonoscopy competence.

Construct validation of a physical model colonoscopy simulator.

BACKGROUND: Previous studies have demonstrated the construct validity of virtual reality colonoscopy simulators by showing that they can distinguish between users according to their level of endoscopic experience. Although physical model simulators are known to simulate looping more realistically than these devices, they lack published validation evidence. OBJECTIVE: To assess the construct validity of a physical model simulator, the Kyoto Kagaku Colonoscope Training Model (Kyoto Kagaku Co. Ltd, Kyoto, Japan) and to determine its suitability for assessing the insertion skill of trainee colonoscopists. DESIGN: Validation study; 21 experienced colonoscopists and 18 novices made 2 attempts at each of 4 standard cases on the Kyoto Kagaku physical model simulator, and we compared their performance on each case. SETTING: A medical simulation center in a large tertiary hospital. MAIN OUTCOME MEASUREMENTS: Completion rates, times to cecum, and peak forces applied to the colon model. RESULTS: Compared with novices, experienced colonoscopists had significantly higher completion to cecum rates and shorter times to cecum for each of the 4 cases (all P < .005). For 2 cases, experienced colonoscopists also exerted significantly lower peak forces than did novices (both P = .01). LIMITATIONS: Two of the model's 6 "standard cases" were not included in the study. CONCLUSIONS: The 4 cases included in the study have construct validity in that they can distinguish between the performance of experienced colonoscopists and novices, reproducing experienced/novice differences found in real colonoscopy. These cases can be used to validly assess the insertion skill of colonoscopy trainees.

The influence of visual motion on interceptive actions and perception.

Visual information is an essential guide when interacting with moving objects, yet it can also be deceiving. For instance, motion can induce illusory position shifts, such that a moving ball can seem to have bounced past its true point of contact with the ground. Some evidence suggests illusory motion-induced position shifts bias pointing tasks to a greater extent than they do perceptual judgments. This, however, appears at odds with other findings and with our success when intercepting moving objects. Here we examined the accuracy of interceptive movements and of perceptual judgments in relation to simulated bounces. Participants were asked to intercept a moving disc at its bounce location by positioning a virtual paddle, and then to report where the disc had landed. Results showed that interceptive actions were accurate whereas perceptual judgments were inaccurate, biased in the direction of motion. Successful interceptions necessitated accurate information concerning both the location and timing of the bounce, so motor planning evidently had privileged access to an accurate forward model of bounce timing and location. This would explain why people can be accurate when intercepting a moving object, but lack insight into the accurate information that had guided their actions when asked to make a perceptual judgment.

Assessing the realism of colonoscopy simulation: the development of an instrument and systematic comparison of 4 simulators.

BACKGROUND: No useful comparative data exist on the relative realism of commercially available devices for simulating colonoscopy. OBJECTIVES: To develop an instrument for quantifying realism and provide the first wide-ranging empiric comparison. DESIGN: Repeated measures, observational study. Nineteen experienced colonoscopists completed cases on 4 colonoscopy simulators (AccuTouch, GI Mentor II, Koken, and Kyoto Kagaku) and evaluated each device. SETTING: A medical simulation center in a large tertiary hospital. MAIN OUTCOME MEASURES: For each device, colonoscopists completed the newly developed Colonoscopy Simulator Realism Questionnaire (CSRQ), which contains 58 items grouped into 10 subscales measuring the realism of different aspects of the simulation. Subscale scores are weighted and combined into an aggregated score, and there is also a single overall realism item. RESULTS: Overall, current colonoscopy simulators were rated as only moderately realistic compared with real human colonoscopy (mean aggregated score, 56.28/100; range, 48.39-60.45, where 0 = "extremely unrealistic" and 100 = "extremely realistic"). On both overall realism measures, the GI Mentor II was rated significantly less realistic than the AccuTouch, Kyoto Kagaku, and Koken (P < .001). There were also significant differences between simulators on 9 subscales, and the pattern of results varied between subscales. LIMITATIONS: The study was limited to commercially available simulators, excluding ex-vivo models. The CSRQ does not assess simulated therapeutic procedures. CONCLUSIONS: The CSRQ is a useful instrument for quantifying simulator realism. There is no clear "first choice" simulator among those assessed. Each has unique strengths and weaknesses, reflected in the differing results observed across 9 subscales. These findings may facilitate the targeted selection of simulators for various aspects of colonoscopy training.

The effect of priming on interceptive actions.

Time constraints in ball sports encourage players to take advantage of any relevant advance information available to prepare their actions. Advance information, therefore, can serve to prime movement parameters (e.g. movement direction) and reduce the amount of time required to prepare the upcoming movement. Regularly, however, players face situations in which the information used to prepare the action turns out to be outdated just prior to movement initiation and the prepared action needs to be changed as soon as possible. The aim of the experiment presented here was to determine whether the priming effect, generally reported for reaction time tasks, could be generalised to interceptive actions. A secondary aim was to examine the strategies employed by the participants to cope with valid, invalid, or no advance information. The results indicate that, when available, the participants used advance information to prepare their movements. More specifically, in comparison with valid advance information, hit rate and spatial accuracy were reduced when the participants had no advance information and were even smaller when the information conveyed was invalid. The results also suggest that in the absence of valid advance information, the strategies employed to intercept the moving target were tuned to the time remaining until the interception was due to occur.

Delayed inhibition of an anticipatory action during motion extrapolation.

BACKGROUND: Continuous visual information is important for movement initiation in a variety of motor tasks. However, even in the absence of visual information people are able to initiate their responses by using motion extrapolation processes. Initiation of actions based on these cognitive processes, however, can demand more attentional resources than that required in situations in which visual information is uninterrupted. In the experiment reported we sought to determine whether the absence of visual information would affect the latency to inhibit an anticipatory action. METHODS: The participants performed an anticipatory timing task where they were instructed to move in synchrony with the arrival of a moving object at a determined contact point. On 50% of the trials, a stop sign appeared on the screen and it served as a signal for the participants to halt their movements. They performed the anticipatory task under two different viewing conditions: Full-View (uninterrupted) and Occluded-View (occlusion of the last 500 ms prior to the arrival at the contact point). RESULTS: The results indicated that the absence of visual information prolonged the latency to suppress the anticipatory movement. CONCLUSION: We suggest that the absence of visual information requires additional cortical processing that creates competing demand for neural resources. Reduced neural resources potentially causes increased reaction time to the inhibitory input or increased time estimation variability, which in combination would account for prolonged latency.

The time course of direction specification in brief interceptive actions.

In fastball sports such as baseball and tennis people are required to produce accurate responses following brief observations of the ball. This limits the time available to prepare the movement. To cope with constrained viewing periods which precede the interception of fast approaching balls, performers are likely to prepare their responses in advance. Although motor preparation may begin before the moving object is seen, accuracy requires that certain program parameters are determined from observations of the target. The aim of the experiment reported here was to determine the last moment at which information about the direction of the target can be incorporated into a motor program. The empirical protocol used in this study allowed us to examine whether new direction information is incorporated discretely or continuously into the program during short intervals prior to movement onset (MO) - the preparation interval. Participants were trained to hit moving targets at two directions with movements of a specific duration (180 ms). This method permitted an estimate of MO. Preparation intervals were controlled by issuing a stimulus cue for movement direction at various times prior to the estimated MO. Results showed that direction information could be fully incorporated into the program with a preparation interval as brief as 250 ms. In addition, the results indicated that direction was specified predominantly in a discrete fashion even at short preparation intervals.

Preparation and inhibition of interceptive actions.

Two experiments aimed to provide an estimate of the last moment at which visual information needs to be obtained in order for it to be used to initiate execution of an interceptive movement or to withhold execution of such a movement. In experiment 1, we sought to estimate the minimum time required to suppress the movement when the participants were first asked to intercept a moving target. In experiment 2, we sought to determine the minimum time required to initiate an interceptive movement when the participants were initially asked to keep stationary. Participants were trained to hit moving targets using movements of a pre-specified duration. This permitted an estimate of movement onset (MO) time. In both experiments the requirement to switch from one prepared course of action to the other was indicated by changing the colour of the moving target at times prior to the estimated MO. The results of the experiments showed that the decision to execute or suppress the interception must be made no less than about 200 ms before MO.

The utilisation of visual information in the control of rapid interceptive actions.

When intercepting a moving target, accurate timing depends, in part, upon starting to move at the right moment. It is generally believed that this is achieved by triggering motor command generation when a visually perceived quantity such as the target's time-to-arrival reaches a specific criterion value. An experimental method that could be used to determine the moment when this visual event happens was introduced by Whiting and coworkers in the 1970s, and it involves occluding the vision of the target at different times prior to the time of movement onset (MO). This method is limited because the experimenter has no control over MO time. We suggest a method which provides the needed control by having people make interceptive movements of a specific duration. We tested the efficacy of this method in two experiments in which the accuracy of interception was examined under different occlusion conditions. In the first experiment, we examined the effect of changing the timing of an occlusion period (OP) of fixed duration (200 ms). In the second experiment, we varied the duration of the OP (180-430 ms) as well as its timing. The results demonstrated the utility of the proposed method and showed that performance deteriorated only when the participants had their vision occluded from 200 ms prior to MO. The results of Experiment 2 were able to narrow down the critical interval to trigger the interceptive action to within the period from 200 to 150 ms prior to MO, probably closer to 150 ms. In addition, the results showed that the execution of brief interceptive movements (180 ms) was not affected by the range of OPs used in the experiments. This indicates that the whole movement was prepared in advance and triggered by a visual stimulus event that occurred at about 150 ms before onset.

Reducing contrast makes speeds in a video-based driving simulator harder to discriminate as well as making them appear slower.

We investigated the effect of reducing image contrast on speed perception using a video-based driving simulator in which participants viewed pairs of scenes and were asked to judge whether the second scene was faster or slower than the first scene. We predicted two outcomes: (i) that vehicle speeds would become harder to discriminate, and (ii) that vehicle speeds would appear slower. There is previous evidence confirming the latter prediction in a less realistic computer-based driving simulation, but none demonstrating the former. Our results supported both predictions, each of which may have traffic-safety implications when reduced-contrast conditions are experienced in real life, such as with fog or when the driver has cataracts.

Auditory feedback influences perceived driving speeds.

Reducing the level of internal noise is seen as a goal when designing modern cars. One danger of such a philosophy is that one is systematically attempting to alter one of the cues that can be used by drivers to estimate speed and this could bias speed judgments and driving behaviour. Seven participants were presented with pairs of video-based driving scenes and asked to judge whether the second scene appeared faster or slower than the first (2-alternative forced-choice task using the method of constant stimuli). They either heard in-car noise at the level it occurred in the real world or reduced in volume by 5 dB. The reduction in noise led to participants judging speeds to be significantly slower and this effect was evident for all participants. This finding indicates that, when in-car noise is attenuated, drivers are likely to underestimate their speed, potentially encouraging them to drive faster and placing them at greater risk of crashing.