A surgical instrument motion measurement system for skill evaluation in practical laparoscopic surgery training.

This study developed and validated a surgical instrument motion measurement system for skill evaluation during practical laparoscopic surgery training. Owing to the various advantages of laparoscopic surgery including minimal invasiveness, this technique has been widely used. However, expert surgeons have insufficient time for providing training to beginners due to the shortage of surgeons and limited working hours. Skill transfer efficiency has to be improved for which there is an urgent need to develop objective surgical skill evaluation methods. Therefore, a simple motion capture-based surgical instrument motion measurement system that could be easily installed in an operating room for skill assessment during practical surgical training was developed. The tip positions and orientations of the instruments were calculated based on the marker positions attached to the root of the instrument. Because the patterns of these markers are individual, this system can track multiple instruments simultaneously and detect exchanges. However due to the many obstacles in the operating room, the measurement data included noise and outliers. In this study, the effect of this decrease in measurement accuracy on feature calculation was determined. Accuracy verification experiments were conducted during wet-lab training to demonstrate the capability of this system to measure the motion of surgical instruments with practical accuracy. A surgical training experiment on a cadaver was conducted, and the motions of six surgical instruments were measured in 36 cases of laparoscopic radical nephrectomy. Outlier removal and smoothing methods were also developed and applied to remove the noise and outliers in the obtained data. The questionnaire survey conducted during the experiment confirmed that the measurement system did not interfere with the surgical operation. Thus, the proposed system was capable of making reliable measurements with minimal impact on surgery. The system will facilitate surgical education by enabling the evaluation of skill transfer of surgical skills.

Single-Camera-Based Bridge Structural Displacement Measurement with Traffic Counting.

Vision-based structural displacement methods allow convenient monitoring of civil structures such as bridges, though they are often limited due to the small number of measurement points, constrained spatial resolution, and inability to identify the acting forces of the measured displacement. To increase the number of measurement points in vision-based bridge displacement measurement, this study introduces a front-view tandem marker motion capture system with side-view traffic counting to identify the force-inducing passing vehicles on the bridge's deck. The proposed system was able to measure structural displacement at submillimeter resolution on eight measurement points at once at a distance of 40.8-64.2 m from a front-view camera. The traffic counting system with a side-view camera recorded the passing vehicles from two opposing lanes. We conducted a 35-min experiment for a 25 m-span steel road bridge with hundreds of cars passing over it and confirmed dynamic displacement distributions with amplitudes of several millimeters when large vehicles passed.

HFR Projector Camera Based Visible Light Communication System for Real-Time Video Streaming.

This study develops a projector-camera-based visible light communication (VLC) system for real-time broadband video streaming, in which a high frame rate (HFR) projector can encode and project a color input video sequence into binary image patterns modulated at thousands of frames per second and an HFR vision system can capture and decode these binary patterns into the input color video sequence with real-time video processing. For maximum utilization of the high-throughput transmission ability of the HFR projector, we introduce a projector-camera VLC protocol, wherein a multi-level color video sequence is binary-modulated with a gray code for encoding and decoding instead of pure-code-based binary modulation. Gray code encoding is introduced to address the ambiguity with mismatched pixel alignments along the gradients between the projector and vision system. Our proposed VLC system consists of an HFR projector, which can project 590 × 1060 binary images at 1041 fps via HDMI streaming and a monochrome HFR camera system, which can capture and process 12-bit 512 × 512 images in real time at 3125 fps; it can simultaneously decode and reconstruct 24-bit RGB video sequences at 31 fps, including an error correction process. The effectiveness of the proposed VLC system was verified via several experiments by streaming offline and live video sequences.

Real-Time Traffic Light Detection with Frequency Patterns Using a High-speed Camera.

LEDs are widely employed as traffic lights. Because most LED traffic lights are driven by alternative power, they blink at high frequencies, even at twice their frequencies. We propose a method to detect a traffic light from images captured by a high-speed camera that can recognize a blinking traffic light. This technique is robust under various illuminations because it can detect traffic lights by extracting information from the blinking pixels at a specific frequency. The method is composed of six modules, which includes a band-pass filter and a Kalman filter. All the modules run simultaneously to achieve real-time processing and can run at 500 fps for images with a resolution of 800 × 600. This technique was verified on an original dataset captured by a high-speed camera under different illumination conditions such as a sunset or night scene. The recall and accuracy justify the generalization of the proposed detection system. In particular, it can detect traffic lights with a different appearance without tuning parameters and without datasets having to be learned.

Gain-compensation Methodology for a Sinusoidal Scan of a Galvanometer Mirror in Proportional-Integral-Differential Control Using Pre-emphasis Techniques.

Galvanometer mirrors are used for optical applications such as target tracking, drawing, and scanning control because of their high speed and accuracy. However, the responsiveness of a galvanometer mirror is limited by its inertia; hence, the gain of a galvanometer mirror is reduced when the control path is steep. In this research, we propose a method to extend the corresponding frequency using a pre-emphasis technique to compensate for the gain reduction of galvanometer mirrors in sine-wave path tracking using proportional-integral-differential (PID) control. The pre-emphasis technique obtains an input value for a desired output value in advance. Applying this method to control the galvanometer mirror, the raw gain of a galvanometer mirror in each frequency and amplitude for sine-wave path tracking using a PID controller was calculated. Where PID control is not effective, maintaining a gain of 0 dB to improve the trajectory tracking accuracy, it is possible to expand the speed range in which a gain of 0 dB can be obtained without tuning the PID control parameters. However, if there is only one frequency, amplification is possible with a single pre-emphasis coefficient. Therefore, a sine wave is suitable for this technique, unlike triangular and sawtooth waves. Hence, we can adopt a pre-emphasis technique to configure the parameters in advance, and we need not prepare additional active control models and hardware. The parameters are updated immediately within the next cycle because of the open loop after the pre-emphasis coefficients are set. In other words, to regard the controller as a black box, we need to know only the input-to-output ratio, and detailed modeling is not required. This simplicity allows our system to be easily embedded in applications. Our method using the pre-emphasis technique for a motion-blur compensation system and the experiment conducted to evaluate the method are explained.

Applying High-Speed Vision Sensing to an Industrial Robot for High-Performance Position Regulation under Uncertainties.

It is traditionally difficult to implement fast and accurate position regulation on an industrial robot in the presence of uncertainties. The uncertain factors can be attributed either to the industrial robot itself (e.g., a mismatch of dynamics, mechanical defects such as backlash, etc.) or to the external environment (e.g., calibration errors, misalignment or perturbations of a workpiece, etc.). This paper proposes a systematic approach to implement high-performance position regulation under uncertainties on a general industrial robot (referred to as the main robot) with minimal or no manual teaching. The method is based on a coarse-to-fine strategy that involves configuring an add-on module for the main robot's end effector. The add-on module consists of a 1000 Hz vision sensor and a high-speed actuator to compensate for accumulated uncertainties. The main robot only focuses on fast and coarse motion, with its trajectories automatically planned by image information from a static low-cost camera. Fast and accurate peg-and-hole alignment in one dimension was implemented as an application scenario by using a commercial parallel-link robot and an add-on compensation module with one degree of freedom (DoF). Experimental results yielded an almost 100% success rate for fast peg-in-hole manipulation (with regulation accuracy at about 0.1 mm) when the workpiece was randomly placed.

Visual encoder: robust and precise measurement method of rotation angle via high-speed RGB vision.

In this paper, a new measurement method of the rotation angle of a rotor, which is named the 'visual encoder,' was proposed. This method is based on the principles of the vision-based method and the optical encoder, and realized by using a high-speed vision system. The visual encoder shows advantageous features such as non-contact, high-resolution and robustness against the free motion and the fluctuation of the rotation axis. A high resolution method to increase the measurement resolution was also suggested. The accuracy and the robustness of the visual encoder were confirmed through the experimental verifications, and the operation was possible at 6,000 rpm even under the fluctuation of rotation axis.

Gain-compensated sinusoidal scanning of a galvanometer mirror in proportional-integral-differential control using the pre-emphasis technique for motion-blur compensation.

We propose a method to achieve precise sine-wave path tracking for real-time motion-blur compensation to extend the corresponding frequency spectrum in proportional-integral-differential (PID) control by using a pre-emphasis technique. We calculate pre-emphasis coefficients in advance to follow a sine wave with a gain of 0 dB and multiply the input signal by these pre-emphasis coefficients. In experiments, we were thus able to extend the greatest frequency from 100 to 500 Hz and achieve gain improvement of approximately 3 dB at 400 and 500 Hz. For the application of inspection, we confirmed that motion blur is significantly reduced when the system operates at high frequency, and we achieved a responsiveness 3.3 times higher than that of our previous system.