Guidewire Endpoint Detection Based on Pixel Adjacent Relation in Robot-assisted Cardiovascular Interventions.

Visualization of endovascular tools like guidewire and catheter is essential for procedural success of endovascular interventions. This requires tracking the tool pixels and motion during catheterization; however, detecting the endpoints of the endovascular tools is challenging due to their small size, thin appearance, and flexibility. As this still limit the performances of existing methods used for endovascular tool segmentation, predicting correct object location could provide ways forward. In this paper, we proposed a neighborhood-based method for detecting guidewire endpoints in X-ray angiograms. Typically, it consists of pixel-level segmentation and a post-segmentation step that is based on adjacency relationships of pixels in a given neighborhood. The latter includes skeletonization to predict endpoint pixels of guidewire. The method is evaluated with proprietary guidewire dataset obtained during in-vivo study in six rabbits, and it shows a high segmentation performance characterized with precision of 87.87% and recall of 90.53%, and low detection error with a mean pixel error of 2.26±0.14 pixels. We compared our method with four state-of-the-art detection methods and found it to exhibit the best detection performance. This neighborhood-based detection method can be generalized for other surgical tool detection and in related computer vision tasks.Clinical Relevance- The proposed method can be provided with better tool tracking and visualization systems during robot-assisted intravascular interventional surgery.

Endovascular Tool Segmentation with Multi-lateral Branched Network during Robot-assisted Catheterization.

Robot-assisted catheterization is routinely carried out for intervention of cardiovascular diseases. Meanwhile, the success of endovascular tool navigation depends on visualization and tracking cues available in the robotic platform. Currently, real-time motion analytics are lacking, while poor illumination during fluoroscopy affects existing physics- and learning-based methods used for tool segmentation. A multi-lateral branched network (MLB-Net) is herein proposed for tool segmentation in cardiovascular angiograms. The model has an encoder with multi-lateral separable convolutions and a pyramid decoder. Model training and validation are done on 1320 angiograms obtained during robot-assisted catheterization in rabbit. Model performance, explained with F1-score of 89.01% and mean intersection-over-union of 90.05% on 330 frames, indicates the model's robustness for guidewire segmentation in angiograms. The MLB-Net offers better performance than the state-of-the-art segmentation models such as U-Net, U-Net++ and DeepLabV3. Thus, it could provide basis for endovascular tool tracking and surgical scene analytics during cardiovascular interventions.

Noninvasive Blood Glucose Monitoring Using Spatiotemporal ECG and PPG Feature Fusion and Weight-Based Choquet Integral Multimodel Approach.

change of blood glucose (BG) level stimulates the autonomic nervous system leading to variation in both human's electrocardiogram (ECG) and photoplethysmogram (PPG). In this article, we aimed to construct a novel multimodal framework based on ECG and PPG signal fusion to establish a universal BG monitoring model. This is proposed as a spatiotemporal decision fusion strategy that uses weight-based Choquet integral for BG monitoring. Specifically, the multimodal framework performs three-level fusion. First, ECG and PPG signals are collected and coupled into different pools. Second, the temporal statistical features and spatial morphological features in the ECG and PPG signals are extracted through numerical analysis and residual networks, respectively. Furthermore, the suitable temporal statistical features are determined with three feature selection techniques, and the spatial morphological features are compressed by deep neural networks (DNNs). Lastly, weight-based Choquet integral multimodel fusion is integrated for coupling different BG monitoring algorithms based on the temporal statistical features and spatial morphological features. To verify the feasibility of the model, a total of 103 days of ECG and PPG signals encompassing 21 participants were collected in this article. The BG levels of participants ranged between 2.2 and 21.8 mmol/L. The results obtained show that the proposed model has excellent BG monitoring performance with a root-mean-square error (RMSE) of 1.49 mmol/L, mean absolute relative difference (MARD) of 13.42%, and Zone A + B of 99.49% in tenfold cross-validation. Therefore, we conclude that the proposed fusion approach for BG monitoring has potentials in practical applications of diabetes management.

Kinematics Constraint Modeling for Flexible Robots based on Deep Learning1.

Application of flexible robotic systems and teleoperated control recently used in minimally invasive surgery have introduced paradigm shift in interventional surgery. While Prototypes of flexible robots have been proposed for surgical diagnostic and treatments, precise constraint control models are still needed for flexible pathway navigation. In this paper, a deep learning based kinematics model is proposed for motion control of flexible robots. Unlike previous approach, this study utilized the different layers of deep learning system for learning the best features to predict the damping value for each point in the robot's workspace. The method uses differential Jacobian to solve IK for given targets. Optimal damping factor that converges precisely around given target is rapidly predicted by a DNN. Simulation of the robot and implementation of the proposed control models are done in V-rep and Python. Validation with arbitrary points shows the deep-learning approach requires an average of 26.50 iterations, a mean error of 0.838, and an execution time of 3.6 ms for IK of single point; and converges faster than other existing methods.

Automatic tool segmentation and tracking during robotic intravascular catheterization for cardiac interventions.

BACKGROUND: Cardiovascular diseases resulting from aneurism, thrombosis, and atherosclerosis in the cardiovascular system are major causes of global mortality. Recent treatment methods have been based on catheterization of flexible endovascular tools with imaging guidance. While advances in robotic intravascular catheterization have led to modeling tool navigation approaches with data sensing and feedback, proper adaptation of image-based guidance for robotic navigation requires the development of sensitive segmentation and tracking models without specificity loss. Several methods have been developed to tackle non-uniform illumination, low contrast; however, presence of untargeted body organs commonly found in X-ray frames taken during angiography procedures still presents some major issues to be solved. METHODS: In this study, a segmentation method was developed for automatic detection and tracking of guidewire pixels in X-ray angiograms. Image frames were acquired during robotic intravascular catheterization for cardiac interventions. For segmentation, multiscale enhancement filtering was applied on preprocessed X-ray angiograms, while morphological operations and filters were applied to refine the frames for pixel intensity adjustment and vesselness measurement. Minima and maxima extrema of the pixels were obtained to detect guidewire pixels in the X-ray frames. Lastly, morphological operation was applied for guidewire pixel connectivity and tracking in segmented pixels. Method validation was performed on 12 X-ray angiogram sequences which were acquired during in vivo intravascular catheterization trials in rabbits. RESULTS: The study outcomes showed that an overall accuracy of 0.995±0.001 was achieved for segmentation. Tracking performance was characterized with displacement and orientation errors observed as 1.938±2.429 mm and 0.039±0.040°, respectively. Evaluation studies performed against 9 existing methods revealed that this proposed method provides more accurate segmentation with 0.753±0.074 area under curve. Simultaneously, high tracking accuracy of 0.995±0.001 with low displacement and orientation errors of 1.938±2.429 mm and 0.039±0.040°, respectively, were achieved. Also, the method demonstrated higher sensitivity and specificity values compared to the 9 existing methods, with a relatively faster exaction time. CONCLUSIONS: The proposed method has the capability to enhance robotic intravascular catheterization during percutaneous coronary interventions (PCIs). Thus, interventionists can be provided with better tool tracking and visualization systems while also reducing their exposure to operational hazards during intravascular catheterization for cardiac interventions.

High-Resolution and High-Sensitivity Flexible Capacitive Pressure Sensors Enhanced by a Transferable Electrode Array and a Micropillar-PVDF Film.

Flexible pressure sensors have attracted increasing attention because they can mimic human skin to sense external pressure; however, for mimicking human skin, the sensing of a pressure point is far from sufficient. To realize fully biomimetic skins, it is crucial for flexible sensors to have high resolution and high sensitivity. We conducted simulations and experiments to determine the relationship between the sensor sensitivity and physical parameters, such as the effective relative permittivity and air ratio of the dielectric layer. According to the results, a micropillar-poly(vinylidene fluoride) (PVDF) dielectric layer was designed to achieve high sensitivity (0.43 kPa-1) in the low-pressure regime (<1 kPa). An 8 × 8 pixel sensor matrix was prepared based on a micropillar-PVDF (MP) film and electrode array (MPEA) to detect the pressure distribution with high resolution (13 dpi). Each pixel could reflect the point of applied pressure through an obvious change in the relative capacitance; moreover, objects with various geometries could be mapped by the pixels of the flexible sensor. A counterweight, a plastic flag, and pine leaves were placed on the flexible sensor, and the shapes were successfully mapped; in particular, the mapping of the ∼0.005 g ultra-lightweight pine leaves with a length of 7 mm and a width of 0.6 mm shows the high sensitivity and high resolution of our flexible pressure sensor.

Facile Fabrication of 3D Porous Sponges Coated with Synergistic Carbon Black/Multiwalled Carbon Nanotubes for Tactile Sensing Applications.

Recently, flexible tactile sensors based on three-dimensional (3D) porous conductive composites, endowed with high sensitivity, a wide sensing range, fast response, and the capability to detect low pressures, have aroused considerable attention. These sensors have been employed in different practical domain areas such as artificial skin, healthcare systems, and human-machine interaction. In this study, a facile, cost-efficient method is proposed for fabricating a highly sensitive piezoresistive tactile sensor based on a 3D porous dielectric layer. The proposed sensor is designed with a simple dip-coating homogeneous synergetic conductive network of carbon black (CB) and multi-walled carbon nanotube (MWCNTs) composite on polydimethysiloxane (PDMS) sponge skeletons. The unique combination of a 3D porous structure, with hybrid conductive networks of CB/MWCNTs displayed a superior elasticity, with outstanding electrical characterization under external compression. The piezoresistive tactile sensor exhibited a high sensitivity of (15 kPa-1), with a rapid response time (100 ms), the capability of detecting both large and small compressive strains, as well as excellent mechanical deformability and stability over 1000 cycles. Benefiting from a long-term stability, fast response, and low-detection limit, the piezoresistive sensor was successfully utilized in monitoring human physiological signals, including finger heart rate, pulses, knee bending, respiration, and finger grabbing motions during the process of picking up an object. Furthermore, a comprehensive performance of the sensor was carried out, and the sensor's design fulfilled vital evaluation metrics, such as low-cost and simplicity in the fabrication process. Thus, 3D porous-based piezoresistive tactile sensors could rapidly promote the development of high-performance flexible sensors, and make them very attractive for an enormous range of potential applications in healthcare devices, wearable electronics, and intelligent robotic systems.

Printable, Highly Sensitive Flexible Temperature Sensors for Human Body Temperature Monitoring: A Review.

In recent years, the development and research of flexible sensors have gradually deepened, and the performance of wearable, flexible devices for monitoring body temperature has also improved. For the human body, body temperature changes reflect much information about human health, and abnormal body temperature changes usually indicate poor health. Although body temperature is independent of the environment, the body surface temperature is easily affected by the surrounding environment, bringing challenges to body temperature monitoring equipment. To achieve real-time and sensitive detection of various parts temperature of the human body, researchers have developed many different types of high-sensitivity flexible temperature sensors, perfecting the function of electronic skin, and also proposed many practical applications. This article reviews the current research status of highly sensitive patterned flexible temperature sensors used to monitor body temperature changes. First, commonly used substrates and active materials for flexible temperature sensors have been summarized. Second, patterned fabricating methods and processes of flexible temperature sensors are introduced. Then, flexible temperature sensing performance are comprehensively discussed, including temperature measurement range, sensitivity, response time, temperature resolution. Finally, the application of flexible temperature sensors based on highly delicate patterning are demonstrated, and the future challenges of flexible temperature sensors have prospected.

Motion and Trajectory Constraints Control Modeling for Flexible Surgical Robotic Systems.

Success of the da Vinci surgical robot in the last decade has motivated the development of flexible access robots to assist clinical experts during single-port interventions of core intrabody organs. Prototypes of flexible robots have been proposed to enhance surgical tasks, such as suturing, tumor resection, and radiosurgery in human abdominal areas; nonetheless, precise constraint control models are still needed for flexible pathway navigation. In this paper, the design of a flexible snake-like robot is presented, along with the constraints model that was proposed for kinematics and dynamics control, motion trajectory planning, and obstacle avoidance during motion. Simulation of the robot and implementation of the proposed control models were done in Matlab. Several points on different circular paths were used for evaluation, and the results obtained show the model had a mean kinematic error of 0.37 ± 0.36 mm with very fast kinematics and dynamics resolution times. Furthermore, the robot's movement was geometrically and parametrically continuous for three different trajectory cases on a circular pathway. In addition, procedures for dynamic constraint and obstacle collision detection were also proposed and validated. In the latter, a collision-avoidance scheme was kept optimal by keeping a safe distance between the robot's links and obstacles in the workspace. Analyses of the results showed the control system was optimal in determining the necessary joint angles to reach a given target point, and motion profiles with a smooth trajectory was guaranteed, while collision with obstacles were detected a priori and avoided in close to real-time. Furthermore, the complexity and computational effort of the algorithmic models were negligibly small. Thus, the model can be used to enhance the real-time control of flexible robotic systems.

Learning-based Parameter Estimation for Hysteresis Modeling in Robotic Catheterization.

In the last half decade, nearly 31% of annual global deaths are linked to cardiovascular diseases. Thus, robotic catheterizations are recently proposed for interventions of conditions such as aneurism or atherosclerosis formed along vascular paths leading to the heart. However, existence of mild to strong hysteresis while navigating unactuated catheters with the current robotic systems inhibits autonomous control for vascular surgery. Thus, immersion of surgeons remains high with most of their time spent on steering the catheter in-and- out of the vessels. In this study, an autoregressive nonlinear neural network model is adapted for parameterization of vital causal factors of hysteresis during robotic catheterization. Crucial for autonomous control, hysteretic behaviors of endovascular tool are modeled while suitable values are estimated and analyzed for five contributory factors. The network model is validated with hysteresis data we obtained from a two degree-of-freedom robotic system and an unactuated catheter. Result validation shows accurate description of the hysteresis profile recorded during catheterization trials with a vascular phantom model.

Signal Quality and Electrode-Skin Impedance Evaluation in the Context of Wearable Electroencephalographic Systems.

Recent advancement in technology has brought about increase in the application areas of wearable electroencephalographic devices. In that, new types of electrodes take place, and particular attention is needed to ensure the required quality of obtained signals. In this study, we evaluate electrode-skin impedance and signal quality for several kinds of electrodes when used in conditions typical for wearable devices. Results suggest that active dry electrode coated with gold alloy is superior while it was challenging to obtain appropriate signal quality when using passive dry electrodes. We also demonstrate electrode-skin impedance measurement using the analog frontend ADS1299, which is suitable for implementation in wearable devices.

A Novel Antibacterial Membrane Electrode Based on Bacterial Cellulose/Polyaniline/AgNO3 Composite for Bio-Potential Signal Monitoring.

We propose a flexible, dry, and antibacterial electrode with a low and stable skin electrode contact impedance for bio-potential signal monitoring. We fabricated a bacterial cellulose/polyaniline/AgNO3 nanocomposite membrane (BC/PANI/AgNO3) and used it for bio-potential signal monitoring. The bacterial cellulose (BC) provides a 3-D nanoporous network structure, and it was used as a substrate material in the BC/PANI/AgNO3 nanocomposite membrane. Polyaniline (PANI) and AgNO3, acting as conductive and antibacterial components, respectively, were polymerized and deposited on the surfaces of BC nanofibers to produce uniform thin film membrane with flexible, antibacterial, and conductive properties. Various measurements were conducted, in terms of antibacterial activity, skin electrode contact impedance, and qualitative analysis of ECG signal recordings. The BC/PANI/AgNO3 membrane revealed 100% antibacterial activities against both the Staphylococcus aureus and Escherichia coli bacteria. The skin electrode contact impedance of the proposed BC/PANI/AgNO3 electrode is lower than that of the Ag/AgCl gel electrode, with the same active area. In addition, the electrocardiogram (ECG) signals acquired with the proposed electrodes have stable characteristic waveforms, and they are not contaminated by noise. The waveform fidelity of the BC/PANI/AgNO3 membrane electrodes over 800 ECG cardiac cycles is 99.49%, and after the electrodes were worn for 24 hours, a fidelity of 98.40% was recorded over the same number of cardiac cycles. With the low and stable skin electrode contact impedance, the proposed dry BC/PANI/AgNO3 membrane electrode provided high fidelity for ECG signal recordings, thus offering a potential approach for bio-potential signal monitoring. With the above benefits, the novel flexible and dry BC/PANI/AgNO3 electrode has a significant antibacterial. Most of all, it is the first research to develop antibacterial in the electrode design.

Deeply-learnt damped least-squares (DL-DLS) method for inverse kinematics of snake-like robots.

Recently, snake-like robots are proposed to assist experts during medical procedures on internal organs via natural orifices. Despite their well-spelt advantages, applications in radiosurgery is still hindered by absence of suitable designs required for spatial navigations within clustered and confined parts of human body, and inexistence of precise and fast inverse kinematics (IK) models. In this study, a deeply-learnt damped least squares method is proposed for solving IK of spatial snake-like robot. The robot's model consists of several modules, and each module has a pair of serial-links connected with orthogonal twists. For precise control of the robot's end-effector, damped least-squares approach is used to minimize error magnitude in a function modeled over analytical Jacobian of the robot. This is iteratively done until an apt joint vector needed to converge the robot to desired positions is obtained. For fast control and singularity avoidance, a deep network is built for prediction of unique damping factor required for each target point in the robot's workspace. The deep network consists of 11 x 15 array of neurons at the hidden layer, and deeply-learnt with a huge dataset of 877,500 data points generated from workspace of the snake robot. Implementation results for both simulated and actual prototype of an eight-link model of the robot show the effectiveness of the proposed IK method. With error tolerance of 0.01 mm, the proposed method has a very high reachability measure of 91.59% and faster mean execution time of 9.20 (±16.92) ms for convergence. In addition, the method requires an average of 33.02 (±39.60) iterations to solve the IK problem. Hence, approximately 3.6 iterations can be executed in 1 ms. Evaluation against popularly used IK methods shows that the proposed method has very good performance in terms of accuracy and speed, simultaneously.

Towards Characterization and Adaptive Compensation of Backlash in a Novel Robotic Catheter System for Cardiovascular Interventions.

Despite the success and prospects of the robotic catheter system for the cardiovascular access, loss of vision, and haptics have limited its global adoption. A direct implication is the great difficulty posed when trying to eliminate the backlash in catheters during vascular cannulations. As a result, physicians and patients end up been exposed to high radiation for a long period of time. Existing control systems proposed for such interventional robots have not fully consider the hysteretic (backlash) behavior. In this study, a novel robotic catheter system is designed for accessing the human cardiac area through the radial vasculature, while single factor descriptive analysis is employed to characterize the backlash behavior during axial motions of the interventional robot. Based on the descriptive analysis, an adaptive system is proposed for the backlash compensation during the cardiovascular access. The adaptive system consists of a neuro-fuzzy module that predicts a backlash gap based on bounded motion signals, and contact force modulated from a modified error-based force control model. The proposed system is implemented in MATLAB and visual C++. Finally, an in vitro experiment with a human tubular model, shows that the proposed adaptive compensation system can minimize the backlash occurrence during cardiovascular access.

Co-contraction characteristics of lumbar muscles in patients with lumbar disc herniation during different types of movement.

BACKGROUND: Muscular performance is an important factor for the mechanical stability of lumbar spine in humans, in which, the co-contraction of lumbar muscles plays a key role. We hypothesized that when executing different daily living motions, the performance of the lumbar muscle co-contraction stabilization mechanism varies between patients with lumbar disc herniation (LDH) and healthy controls. Hence, in this study, co-contraction performance of lumbar muscles between patients with LDH and healthy subjects was explored to check if there are significant differences between the two groups when performing four representative movements. METHODS: Twenty-six LDH patients (15 females, 11 males) and a control group of twenty-eight subjects (16 females, 12 males) were recruited. Surface electromyography (EMG) signals were recorded from the external oblique, lumbar multifidus, and internal oblique/transversus abdominis muscles during the execution of four types of movement, namely: forward bending, backward bending, left lateral flexion and right lateral flexion. The acquired EMG signals were segmented, and wavelet decomposition was performed followed by reconstruction of the low-frequency components of the signal. Then, the reconstructed signals were used for further analysis. Co-contraction ratio was employed to assess muscle coordination and compare it between the LDH patients and healthy controls. The corresponding signals of the subjects in the two groups were compared to evaluate the differences in agonistic and antagonistic muscle performance during the different motions. Also, sample entropy was applied to evaluate complexity changes in lumbar muscle recruitment during the movements. RESULTS: Significant differences between the LDH and control groups were found in the studied situations (p < 0.05). During the four movements considered in this study, the participants of the LDH group exhibited a higher level of co-contraction ratio, lower agonistic, and higher antagonistic lumbar muscle activity (p < 0.01) than those of the control group. Furthermore, the co-contraction ratio of LDH patients was dominated by the antagonistic muscle activity during the movements, except for the forward bending motion. However, in the healthy control group, the agonistic muscle activity contributed more to the co-contraction ratio with an exception for the backward bending motion. Conversely, the sample entropy value was significantly lower for agonistic muscles of LDH group compared to the control group (p < 0.01) while the entropy value was significantly greater in antagonistic muscles (p < 0.01) during the four types of movement, respectively. CONCLUSIONS: Lumbar disc herniation patients exhibited numerous variations in the evaluated parameters that reflect the co-contraction of lumbar muscles, the agonistic and antagonistic muscle activities, and their respective sample entropy values when compared with the healthy control group. These variations could be due to the compensation mechanism that was required to stabilize the spine. The results of this study could facilitate the design of efficient rehabilitation methods for treatment of lumbar muscle dysfunctions.

A Master-Slave control system with workspaces isomerism for teleoperation of a snake robot.

Snake robots can be used to assist experts during surgical operations on internal organs via natural orifices. However, real-time control of such robot in Mater Slave (MS) teleoperation is a major challenge. Inverse kinematics solution of snake robots has being a key challenge towards real time control especially if the robot is hyper-redundant. This paper proposes a method that can achieve fast and precise inverse kinematics solution for real time control MS teleoperation. Monte Carlo method is applied to determine possible positions needed to reach a given target point, while best position is chosen based on the optimization algorithm. For workspace isomerism, the proposed method automatically determines appropriate kinematics mapping for the robots. Experimental results show that the method can achieve accurate position tracking in MS teleoperation.

Non-iterative geometric approach for inverse kinematics of redundant lead-module in a radiosurgical snake-like robot.

BACKGROUND: Snake-like robot is an emerging form of serial-link manipulator with the morphologic design of biological snakes. The redundant robot can be used to assist medical experts in accessing internal organs with minimal or no invasion. Several snake-like robotic designs have been proposed for minimal invasive surgery, however, the few that were developed are yet to be fully explored for clinical procedures. This is due to lack of capability for full-fledged spatial navigation. In rare cases where such snake-like designs are spatially flexible, there exists no inverse kinematics (IK) solution with both precise control and fast response. METHODS: In this study, we proposed a non-iterative geometric method for solving IK of lead-module of a snake-like robot designed for therapy or ablation of abdominal tumors. The proposed method is aimed at providing accurate and fast IK solution for given target points in the robot's workspace. n-1 virtual points (VPs) were geometrically computed and set as coordinates of intermediary joints in an n-link module. Suitable joint angles that can place the end-effector at given target points were then computed by vectorizing coordinates of the VPs, in addition to coordinates of the base point, target point, and tip of the first link in its default pose. The proposed method is applied to solve IK of two-link and redundant four-link modules. RESULTS: Both two-link and four-link modules were simulated with Robotics Toolbox in Matlab 8.3 (R2014a). Implementation result shows that the proposed method can solve IK of the spatially flexible robot with minimal error values. Furthermore, analyses of results from both modules show that the geometric method can reach 99.21 and 88.61% of points in their workspaces, respectively, with an error threshold of 1 mm. The proposed method is non-iterative and has a maximum execution time of 0.009 s. CONCLUSIONS: This paper focuses on solving IK problem of a spatially flexible robot which is part of a developmental project for abdominal surgery through minimal invasion or natural orifices. The study showed that the proposed geometric method can resolve IK of the snake-like robot with negligible error offset. Evaluation against well-known methods shows that the proposed method can reach several points in the robot's workspace with high accuracy and shorter computational time, simultaneously.