Unsupervised synthesis of realistic coronary artery X-ray angiogram.

PURPOSE: Medical image analysis suffers from a sparsity of annotated data necessary in learning-based models. Cardiorespiratory simulators have been developed to counter the lack of data. However, the resulting data often lack realism. Hence, the proposed method aims to synthesize realistic and fully customizable angiograms of coronary arteries for the training of learning-based biomedical tasks, for cardiologists performing interventions, and for cardiologist trainees. METHODS: 3D models of coronary arteries are generated with a fully customizable realistic cardiorespiratory simulator. The transfer of X-ray angiography style to simulator-generated images is performed using a new vessel-specific adaptation of the CycleGAN model. The CycleGAN model is paired with a vesselness-based loss function that is designed as a vessel-specific structural integrity constraint. RESULTS: Validation is performed both on the style and on the preservation of the shape of the arteries of the images. The results show a PSNR of 14.125, an SSIM of 0.898, and an overlapping of 89.5% using the Dice coefficient. CONCLUSION: We proposed a novel fluoroscopy-based style transfer method for the enhancement of the realism of simulated coronary artery angiograms. The results show that the proposed model is capable of accurately transferring the style of X-ray angiograms to the simulations while keeping the integrity of the structures of interest (i.e., the topology of the coronary arteries).

CycleGAN for style transfer in X-ray angiography.

PURPOSE: We aim to perform generation of angiograms for various vascular structures as a mean of data augmentation in learning tasks. The task is to enhance the realism of vessels images generated from an anatomically realistic cardiorespiratory simulator to make them look like real angiographies. METHODS: The enhancement is performed by applying the CycleGAN deep network for transferring the style of real angiograms acquired during percutaneous interventions into a data set composed of realistically simulated arteries. RESULTS: The cycle consistency was evaluated by comparing an input simulated image with the one obtained after two cycles of image translation. An average structural similarity (SSIM) of 0.948 on our data sets has been obtained. The vessel preservation was measured by comparing segmentations of an input image and its corresponding enhanced image using Dice coefficient. CONCLUSIONS: We proposed an application of the CycleGAN deep network for enhancing the artificial data as an alternative to classical data augmentation techniques for medical applications, particularly focused on angiogram generation. We discussed success and failure cases, explaining conditions for the realistic data augmentation which respects both the complex physiology of arteries and the various patterns and textures generated by X-ray angiography.

Epistemic Uncertainty Modeling for Vessel Segmentation.

X-ray angiograms are currently the gold-standard in percutaneous guidance during cardiovascular interventions. However, due to lack of contrast, to overlapping artifacts and to the rapid dilution of the contrast agent, they remain difficult to analyze either by cardiologists, or automatically by computers. Providing, a general yet accurate multi-arteries segmentation method along with the uncertainty linked to those segmentations would not only ease the analysis of medical imaging by cardiologists, but also provide a required pre-processing of the data for tasks ranging from 3D reconstruction to motion tracking of arteries. The proposed method has been validated on clinical data providing an average accuracy of 94.9%. Additionally, results show good transposition of learning from one type of artery to another. Epistemic uncertainty maps provide areas where the segmentation should be validated by an expert before being used, and could provide identification of regions of interest for data augmentation purposes.

Model-free Cardiorespiratory Motion Prediction from X-ray Angiography Sequence with LSTM Network.

We present a novel model-free approach for cardiorespiratory motion prediction from X-ray angiography time series based on Long Short-Term Memory Recurrent Neural Networks (LSTM-RNN). Cardiorespiratory motion prediction is defined as a problem of estimating the future displacement of the coronary vessels in the next image frame in an X-ray angiography sequence. The displacement of the vessels is represented as a sequence of 2D affine transformation matrices allowing 2D X-ray registrations in a sequence. The new displacement parameters from a sequence of transformation matrices are predicted using an LSTM model. LSTM is a particular form of Recurrent Neural Network (RNN) architecture suitable for learning sequential data and predicting time series. The method was developed and validated by simulated data using a realistic cardiorespiratory motion simulator (XCAT). The results show that this method converges quickly and can predict the complex motion in the angiography sequences with irregularities. The mean values of prediction error over all the patients are approximately 0.29 mm (2 pixels) difference for the combination of both motions, 0.51 mm (3.5 pixels) difference for cardiac motion and 0.44 mm (3 pixels) difference for respiratory motion.