Automated three-dimensional vessel reconstruction based on deep segmentation and bi-plane angiographic projections.

Automated three-dimensional (3D) blood vessel reconstruction to improve vascular diagnosis and therapeutics is a challenging task in which the real-time implementation of automatic segmentation and specific vessel tracking for matching artery sequences is essential. Recently, a deep learning-based segmentation technique has been proposed; however, existing state-of-the-art deep architectures exhibit reduced performance when they are employed using real in-vivo imaging because of serious issues such as low contrast and noise contamination of the X-ray images. To overcome these limitations, we propose a novel methodology composed of the de-haze image enhancement technique as pre-processing and multi-level thresholding as post-processing to be applied to the lightweight multi-resolution U-shaped architecture. Specifically, (1) bi-plane two-dimensional (2D) vessel images were extracted simultaneously using the deep architecture, (2) skeletons of the vessels were computed via a morphology operation, (3) the corresponding skeleton structure between image sequences was matched using the shape-context technique, and (4) the 3D centerline was reconstructed using stereo geometry. The method was validated using both in-vivo and in-vitro models. The results show that the proposed technique could improve the segmentation quality, reduce computation time, and reconstruct the 3D skeleton automatically. The algorithm accurately reconstructed the phantom model and the real mouse vessel in 3D in 2 s. Our proposed technique has the potential to allow therapeutic micro-agent navigation in clinical practice, thereby providing the 3D position and orientation of the vessel.

Real-time microrobot posture recognition via biplane X-ray imaging system for external electromagnetic actuation.

PURPOSE: As a promising intravascular therapeutic approach for autonomous catheterization, especially for thrombosis treatment, a microrobot or robotic catheter driven by an external electromagnetic actuation system has been recently investigated. However, the three-dimensional (3D) real-time position and orientation tracking of the microrobot remains a challenge for precise feedback control in clinical applications owing to the micro-size of the microrobot geometry in vessels, along with bifurcation and vulnerability. Therefore, in this paper, we propose a 3D posture recognition method for the unmanned microrobotic surgery driven by an external electromagnetic actuator system. METHODS: We propose a real-time position and spatial orientation tracking method for a millimeter-sized intravascular object or microrobot using a principal component analysis algorithm and an X-ray reconstruction. The suggested algorithm was implemented to an actual controllable wireless microrobot system composed of a bullet-shaped object, a biplane X-ray imaging device, and an electromagnetic actuation system. Numerical computations and experiments were conducted for the performance verification. RESULTS: The experimental results showed a good performance of the implemented system with tracking errors less than 0.4 mm in position and 2° in orientation. The proposed tracking technique accomplished a fast processing time, ~ 0.125 ms/frame, and high-precision recognition of the micro-sized object. CONCLUSIONS: Since the suggested method does not require pre-information of the object geometry in the human body for its 3D shape and position recognition, it could be applied to various elliptical shapes of the microrobot system with computation time efficacy and recognition accuracy. Hence, the method can be used for therapeutic millimeter- or micron-sized manipulator recognition in vascular, as well as implanted objects in the human body.