The Contribution of Thoracic Radiation Dose Volumes to Subsequent Development of Cardiovascular Disease in Cancer Survivors.

BACKGROUND: Although radiation therapy (RT) has been recognized for contributing to cardiovascular disease (CVD), it is unknown whether specific doses received by cardiovascular tissues influence development. OBJECTIVE: In this pilot study, we examined the contribution of RT dose distribution on the development of CVD events in patients with cancer within 5 years of RT. METHODS: A retrospective case-controlled design was used matching 28 cases receiving thoracic RT who subsequently developed an adverse CVD event with 28 controls based upon age, gender, and cancer type. Dose volume histograms of nongated computed tomography scans received during RT characterized the dose delivered to the heart. Heart chambers were segmented using an atlas approach, and radiomics features for the segmentation as well as planning dose in each chamber were tabulated for analysis. RESULT: No significant differences were observed in the RT dose statistics between groups, preexisting CVD, nor significant differences of RT doses delivered to distinct chambers of the heart. Cases were found to have greater CVD risk factors at the time of cancer diagnosis. Morphological significant differences for perimeter on border ( P = .043), equivalent spherical radius ( P = .050), and elongation ( P = .038) were observed, with preexisting CVD having the highest values (ie, larger hearts). CONCLUSION: Traditional CVD risk factors were more prevalent in the cases who developed CVD. No differences were observed in doses of RT. Of note, we observed significant differences in heart morphology and mass in known diseased hearts on the pretreatment scans. These new metrics may have implications for the measurement and quantification of CVD.

Semi-automated three-dimensional segmentation for cardiac CT images using deep learning and randomly distributed points.

Given the prevalence of cardiovascular diseases (CVDs), the segmentation of the heart on cardiac computed tomography (CT) remains of great importance. Manual segmentation is time-consuming and intra-and inter-observer variabilities yield inconsistent and inaccurate results. Computer-assisted, and in particular, deep learning approaches to segmentation continue to potentially offer an accurate, efficient alternative to manual segmentation. However, fully automated methods for cardiac segmentation have yet to achieve accurate enough results to compete with expert segmentation. Thus, we focus on a semi-automated deep learning approach to cardiac segmentation that bridges the divide between a higher accuracy from manual segmentation and higher efficiency from fully automated methods. In this approach, we selected a fixed number of points along the surface of the cardiac region to mimic user interaction. Points-distance maps were then generated from these points selections, and a three-dimensional (3D) fully convolutional neural network (FCNN) was trained using points-distance maps to provide a segmentation prediction. Testing our method with different numbers of selected points, we achieved a Dice score from 0.742 to 0.917 across the four chambers. Specifically. Dice scores averaged 0.846 ± 0.059, 0.857 ± 0.052, 0.826 ± 0.062, and 0.824 ± 0.062 for the left atrium, left ventricle, right atrium, and right ventricle, respectively across all points selections. This point-guided, image-independent, deep learning segmentation approach illustrated a promising performance for chamber-by-chamber delineation of the heart in CT images.

A low-cost PVC-based dual-modality kidney phantom.

Phantoms are invaluable tools broadly used for research and training purposes designed to mimic tissues and structures in the body. In this paper, polyvinyl chloride (PVC)-plasticizer and silicone rubbers were explored as economical materials to reliably create long-lasting, realistic kidney phantoms with contrast under both ultrasound (US) and X-ray imaging. The radiodensity properties of varying formulations of soft PVC-based gels were characterized to allow adjustable image intensity and contrast. Using this data, a phantom creation workflow was established which can be easily adapted to match radiodensity values of other organs and soft tissues in the body. Internal kidney structures such as the medulla and ureter were created using a two-part molding process to allow greater phantom customization. The kidney phantoms were imaged under US and X-ray scanners to compare the contrast enhancement of a PVC-based medulla versus a silicone-based medulla. Silicone was found to have higher attenuation than plastic under X-ray imaging, but poor quality under US imaging. PVC was found to exhibit good contrast under X-ray imaging and excellent performance for US imaging. Finally, the durability and shelf life of our PVC-based phantoms were observed to be vastly superior to that of common agar-based phantoms. The work presented here allows extended periods of usage and storage for each kidney phantom while simultaneously preserving anatomical detail, contrast under dual-modality imaging, and low cost of materials.

An Augmented Reality-Assisted Visualization System for Potential Applications in Prostate Biopsy.

Ultrasound-guided biopsy is widely used for disease detection and diagnosis. We plan to register preoperative imaging, such as positron emission tomography / computed tomography (PET/CT) and/or magnetic resonance imaging (MRI), with real-time intraoperative ultrasound imaging for improved localization of suspicious lesions that may not be seen on ultrasound but visible on other imaging modalities. Once the image registration is completed, we will combine the images from two or more imaging modalities and use Microsoft HoloLens 2 augmented reality (AR) headset to display three-dimensional (3D) segmented lesions and organs from previously acquired images and real-time ultrasound images. In this work, we are developing a multi-modal, 3D augmented reality system for the potential use in ultrasound-guided prostate biopsy. Preliminary results demonstrate the feasibility of combining images from multiple modalities into an AR-guided system.

Identifying Unique Acoustic Signatures from Chemically-Crosslinked Microbubble Clusters Using Deep Learning.

Ultrasound contrast agents (UCA) are gas encapsulated microspheres that oscillate volumetrically when exposed to an ultrasound field producing a backscattered signal which can be used for improved ultrasound imaging and drug delivery. UCA's are being used widely for contrast-enhanced ultrasound imaging, but there is a need for improved UCAs to develop faster and more accurate contrast agent detection algorithms. Recently, we introduced a new class of lipid based UCAs called Chemically Cross-linked Microbubble Clusters (CCMCs). CCMCs are formed by the physical tethering of individual lipid microbubbles into a larger aggregate cluster. The advantages of these novel CCMCs are their ability to fuse together when exposed to low intensity pulsed ultrasound (US), potentially generating unique acoustic signatures that can enable better contrast agent detection. In this study, our main objective is to demonstrate that the acoustic response of CCMCs is unique and distinct when compared to individual UCAs using deep learning algorithms. Acoustic characterization of CCMCs and individual bubbles was performed using a broadband hydrophone or a clinical transducer attached to a Verasonics Vantage 256. A simple artificial neural network (ANN) was trained and used to classify raw 1D RF ultrasound data as either from CCMC or non-tethered individual bubble populations of UCAs. The ANN was able to classify CCMCs at an accuracy of 93.8% for data collected from broadband hydrophone and 90% for data collected using Verasonics with a clinical transducer. The results obtained suggest the acoustic response of CCMCs is unique and has the potential to be used in developing a novel contrast agent detection technique.

Automatic Segmentation of Uterine Cavity and Placenta on MR Images Using Deep Learning.

Magnetic resonance imaging (MRI) is useful for the detection of abnormalities affecting maternal and fetal health. In this study, we used a fully convolutional neural network for simultaneous segmentation of the uterine cavity and placenta on MR images. We trained the network with MR images of 181 patients, with 157 for training and 24 for validation. The segmentation performance of the algorithm was evaluated using MR images of 60 additional patients that were not involved in training. The average Dice similarity coefficients achieved for the uterine cavity and placenta were 92% and 80%, respectively. The algorithm could estimate the volume of the uterine cavity and placenta with average errors of less than 1.1% compared to manual estimations. Automated segmentation, when incorporated into clinical use, has the potential to quantify, standardize, and improve placental assessment, resulting in improved outcomes for mothers and fetuses.

CascadeNet for hysterectomy prediction in pregnant women due to placenta accreta spectrum.

In severe cases, placenta accreta spectrum (PAS) requires emergency hysterectomy, endangering the life of both mother and fetus. Early prediction may reduce complications and aid in management decisions in these high-risk pregnancies. In this work, we developed a novel convolutional network architecture to combine MRI volumes, radiomic features, and custom feature maps to predict PAS severe enough to result in hysterectomy after fetal delivery in pregnant women. We trained, optimized, and evaluated the networks using data from 241 patients, in groups of 157, 24, and 60 for training, validation, and testing, respectively. We found the network using all three paths produced the best performance, with an AUC of 87.8, accuracy 83.3%, sensitivity of 85.0, and specificity of 82.5. This deep learning algorithm, deployed in clinical settings, may identify women at risk before birth, resulting in improved patient outcomes.

Placenta Accreta Spectrum and Hysterectomy Prediction Using MRI Radiomic Features.

In women with placenta accreta spectrum (PAS), patient management may involve cesarean hysterectomy at delivery. Magnetic resonance imaging (MRI) has been used for further evaluation of PAS and surgical planning. This work tackles two prediction problems: predicting presence of PAS and predicting hysterectomy using MR images of pregnant patients. First, we extracted approximately 2,500 radiomic features from MR images with two regions of interest: the placenta and the uterus. In addition to analyzing two regions of interest, we dilated the placenta and uterus masks by 5, 10, 15, and 20 mm to gain insights from the myometrium, where the uterus and placenta overlap in the case of PAS. This study cohort includes 241 pregnant women. Of these women, 89 underwent hysterectomy while 152 did not; 141 with suspected PAS, and 100 without suspected PAS. We obtained an accuracy of 0.88 for predicting hysterectomy and an accuracy of 0.92 for classifying suspected PAS. The radiomic analysis tool is further validated, it can be useful for aiding clinicians in decision making on the care of pregnant women.

Technical note: The effect of image annotation with minimal manual interaction for semiautomatic prostate segmentation in CT images using fully convolutional neural networks.

PURPOSE: The goal is to study the performance improvement of a deep learning algorithm in three-dimensional (3D) image segmentation through incorporating minimal user interaction into a fully convolutional neural network (CNN). METHODS: A U-Net CNN was trained and tested for 3D prostate segmentation in computed tomography (CT) images. To improve the segmentation accuracy, the CNN's input images were annotated with a set of border landmarks to supervise the network for segmenting the prostate. The network was trained and tested again with annotated images after 5, 10, 15, 20, or 30 landmark points were used. RESULTS: Compared to fully automatic segmentation, the Dice similarity coefficient increased up to 9% when 5-30 sparse landmark points were involved, with the segmentation accuracy improving as more border landmarks were used. CONCLUSIONS: When a limited number of sparse border landmarks are used on the input image, the CNN performance approaches the interexpert observer difference observed in manual segmentation.

Deep learning-based segmentation of the placenta and uterus on MR images.

Purpose: Magnetic resonance imaging has been recently used to examine the abnormalities of the placenta during pregnancy. Segmentation of the placenta and uterine cavity allows quantitative measures and further analyses of the organs. The objective of this study is to develop a segmentation method with minimal user interaction. Approach: We developed a fully convolutional neural network (CNN) for simultaneous segmentation of the uterine cavity and placenta in three dimensions (3D) while a minimal operator interaction was incorporated for training and testing of the network. The user interaction guided the network to localize the placenta more accurately. In the experiments, we trained two CNNs, one using 70 normal training cases and the other using 129 training cases including normal cases as well as cases with suspected placenta accreta spectrum (PAS). We evaluated the performance of the segmentation algorithms on two test sets: one with 20 normal cases and the other with 50 images from both normal women and women with suspected PAS. Results: For the normal test data, the average Dice similarity coefficient (DSC) was 92% and 82% for the uterine cavity and placenta, respectively. For the combination of normal and abnormal cases, the DSC was 88% and 83% for the uterine cavity and placenta, respectively. The 3D segmentation algorithm estimated the volume of the normal and abnormal uterine cavity and placenta with average volume estimation errors of 4% and 9%, respectively. Conclusions: The deep learning-based segmentation method provides a useful tool for volume estimation and analysis of the placenta and uterus cavity in human placental imaging.

Virtual Reality Assisted Cardiac Catheterization.

Cardiac catheterization is a delicate strategy often used during various heart procedures. However, the procedure carries a myriad of risks associated with it, including damage to the vessel or heart itself, blood clots, and arrhythmias. Many of these risks increase in probability as the length of the operation increases, creating a demand for a more accurate procedure while reducing the overall time required. To this end, we developed an adaptable virtual reality simulation and visualization method to provide essential information to the physician ahead of time with the goal of reducing potential risks, decreasing operation time, and improving the accuracy of cardiac catheterization procedures. We additionally conducted a phantom study to evaluate the impact of using our virtual reality system prior to a procedure.

Augmented Reality-Assisted Biopsy of Soft Tissue Lesions.

Guided biopsy of soft tissue lesions can be challenging in the presence of sensitive organs or when the lesion itself is small. Computed tomography (CT) is the most frequently used modality to target soft tissue lesions. In order to aid physicians, small field of view (FOV) low dose non-contrast CT volumes are acquired prior to intervention while the patient is on the procedure table to localize the lesion and plan the best approach. However, patient motion between the end of the scan and the start of the biopsy procedure can make it difficult for a physician to translate the lesion location from the CT onto the patient body, especially for a deep-seated lesion. In addition, the needle should be managed well in three-dimensional trajectories in order to reach the lesion and avoid vital structures. This is especially challenging for less experienced interventionists. These usually result in multiple additional image acquisitions during the course of procedure to ensure accurate needle placement, especially when multiple core biopsies are required. In this work, we present an augmented reality (AR)-guided biopsy system and procedure for soft tissue and lung lesions and quantify the results using a phantom study. We found an average error of 0.75 cm from the center of the lesion when AR guidance was used, compared to an error of 1.52 cm from the center of the lesion during unguided biopsy for soft tissue lesions while upon testing the system on lung lesions, an average error of 0.62 cm from the center of the tumor while using AR guidance versus a 1.12 cm error while relying on unguided biopsies. The AR-guided system is able to improve the accuracy and could be useful in the clinical application.

Image Guided Mitral Valve Replacement: Registration of 3D Ultrasound and 2D X-ray Images.

Mitral valve repair or replacement is important in the treatment of mitral regurgitation. For valve replacement, a transcatheter approach had the possibility of decrease the invasiveness of the procedure while retaining the benefit of replacement over repair. However, fluoroscopy images acquired during the procedure provide no anatomical information regarding the placement of the probe tip once the catheter has entered a cardiac chamber. By using 3D ultrasound and registering the 3D ultrasound images to the fluoroscopy images, a physician can gain a greater understanding of the mitral valve region during transcatheter mitral valve replacement surgery. In this work, we present a graphical user interface which allows the registration of two co-planar X-ray images with 3D ultrasound during mitral valve replacement surgery.

Incorporating minimal user input into deep learning based image segmentation.

Computer-assisted image segmentation techniques could help clinicians to perform the border delineation task faster with lower inter-observer variability. Recently, convolutional neural networks (CNNs) are widely used for automatic image segmentation. In this study, we used a technique to involve observer inputs for supervising CNNs to improve the accuracy of the segmentation performance. We added a set of sparse surface points as an additional input to supervise the CNNs for more accurate image segmentation. We tested our technique by applying minimal interactions to supervise the networks for segmentation of the prostate on magnetic resonance images. We used U-Net and a new network architecture that was based on U-Net (dual-input path [DIP] U-Net), and showed that our supervising technique could significantly increase the segmentation accuracy of both networks as compared to fully automatic segmentation using U-Net. We also showed DIP U-Net outperformed U-Net for supervised image segmentation. We compared our results to the measured inter-expert observer difference in manual segmentation. This comparison suggests that applying about 15 to 20 selected surface points can achieve a performance comparable to manual segmentation.

Segmentation of uterus and placenta in MR images using a fully convolutional neural network.

Segmentation of the uterine cavity and placenta in fetal magnetic resonance (MR) imaging is useful for the detection of abnormalities that affect maternal and fetal health. In this study, we used a fully convolutional neural network for 3D segmentation of the uterine cavity and placenta while a minimal operator interaction was incorporated for training and testing the network. The user interaction guided the network to localize the placenta more accurately. We trained the network with 70 training and 10 validation MRI cases and evaluated the algorithm segmentation performance using 20 cases. The average Dice similarity coefficient was 92% and 82% for the uterine cavity and placenta, respectively. The algorithm could estimate the volume of the uterine cavity and placenta with average errors of 2% and 9%, respectively. The results demonstrate that the deep learning-based segmentation and volume estimation is possible and can potentially be useful for clinical applications of human placental imaging.

Fully automated segmentation of the right ventricle in patients with repaired Tetralogy of Fallot using U-Net.

Cardiac magnetic resonance (CMR) imaging is considered the standard imaging modality for volumetric analysis of the right ventricle (RV), an especially important practice in the evaluation of heart structure and function in patients with repaired Tetralogy of Fallot (rTOF). In clinical practice, however, this requires time-consuming manual delineation of the RV endocardium in multiple 2-dimensional (2D) slices at multiple phases of the cardiac cycle. In this work, we employed a U-Net based 2D convolutional neural network (CNN) classifier in the fully automatic segmentation of the RV blood pool. Our dataset was comprised of 5,729 short-axis cine CMR slices taken from 100 individuals with rTOF. Training of our CNN model was performed on images from 50 individuals while validation was performed on images from 10 individuals. Segmentation results were evaluated by Dice similarity coefficient (DSC) and Hausdorff distance (HD). Use of the CNN model on our testing group of 40 individuals yielded a median DSC of 90% and a median 95th percentile HD of 5.1 mm, demonstrating good performance in these metrics when compared to literature results. Our preliminary results suggest that our deep learning-based method can be effective in automating RV segmentation.

A complex dual-modality kidney phantom for renal biopsy studies.

We developed a reliable and repeatable process to create hyper-realistic, kidney phantoms with tunable image visibility under ultrasound (US) and CT imaging modalities. A methodology was defined to create phantoms that could be produced for renal biopsy evaluation. The final complex kidney phantom was devised containing critical structures of a kidney: kidney cortex, medulla, and ureter. Simultaneously, some lesions were integrated into the phantom to mimic the presence of tumors during biopsy. The phantoms were created and scanned by ultrasound and CT scanners to verify the visibility of the complex internal structures and to observe the interactions between material properties. The result was a successful advancement in knowledge of materials with ideal acoustic and impedance properties to replicate human organs for the field of image-guided interventions.

Tumor detection of the thyroid and salivary glands using hyperspectral imaging and deep learning.

The performance of hyperspectral imaging (HSI) for tumor detection is investigated in ex-vivo specimens from the thyroid (N = 200) and salivary glands (N = 16) from 82 patients. Tissues were imaged with HSI in broadband reflectance and autofluorescence modes. For comparison, the tissues were imaged with two fluorescent dyes. Additionally, HSI was used to synthesize three-band RGB multiplex images to represent the human-eye response and Gaussian RGBs, which are referred to as HSI-synthesized RGB images. Using histological ground truths, deep learning algorithms were developed for tumor detection. For the classification of thyroid tumors, HSI-synthesized RGB images achieved the best performance with an AUC score of 0.90. In salivary glands, HSI had the best performance with 0.92 AUC score. This study demonstrates that HSI could aid surgeons and pathologists in detecting tumors of the thyroid and salivary glands.

Hyperspectral Imaging of Head and Neck Squamous Cell Carcinoma for Cancer Margin Detection in Surgical Specimens from 102 Patients Using Deep Learning.

Surgical resection of head and neck (H and N) squamous cell carcinoma (SCC) may yield inadequate surgical cancer margins in 10 to 20% of cases. This study investigates the performance of label-free, reflectance-based hyperspectral imaging (HSI) and autofluorescence imaging for SCC detection at the cancer margin in excised tissue specimens from 102 patients and uses fluorescent dyes for comparison. Fresh surgical specimens (n = 293) were collected during H and N SCC resections (n = 102). The tissue specimens were imaged with reflectance-based HSI and autofluorescence imaging and afterwards with two fluorescent dyes for comparison. A histopathological ground truth was made. Deep learning tools were developed to detect SCC with new patient samples (inter-patient) and machine learning for intra-patient tissue samples. Area under the curve (AUC) of the receiver-operator characteristic was used as the main evaluation metric. Additionally, the performance was estimated in mm increments circumferentially from the tumor-normal margin. In intra-patient experiments, HSI classified conventional SCC with an AUC of 0.82 up to 3 mm from the cancer margin, which was more accurate than proflavin dye and autofluorescence (both p < 0.05). Intra-patient autofluorescence imaging detected human papilloma virus positive (HPV+) SCC with an AUC of 0.99 at 3 mm and greater accuracy than proflavin dye (p < 0.05). The inter-patient results showed that reflectance-based HSI and autofluorescence imaging outperformed proflavin dye and standard red, green, and blue (RGB) images (p < 0.05). In new patients, HSI detected conventional SCC in the larynx, oropharynx, and nasal cavity with 0.85-0.95 AUC score, and autofluorescence imaging detected HPV+ SCC in tonsillar tissue with 0.91 AUC score. This study demonstrates that label-free, reflectance-based HSI and autofluorescence imaging methods can accurately detect the cancer margin in ex-vivo specimens within minutes. This non-ionizing optical imaging modality could aid surgeons and reduce inadequate surgical margins during SCC resections.

Deep learning-based three-dimensional segmentation of the prostate on computed tomography images.

Segmentation of the prostate in computed tomography (CT) is used for planning and guidance of prostate treatment procedures. However, due to the low soft-tissue contrast of the images, manual delineation of the prostate on CT is a time-consuming task with high interobserver variability. We developed an automatic, three-dimensional (3-D) prostate segmentation algorithm based on a customized U-Net architecture. Our dataset contained 92 3-D abdominal CT scans from 92 patients, of which 69 images were used for training and validation and the remaining for testing the convolutional neural network model. Compared to manual segmentation by an expert radiologist, our method achieved 83 % ± 6 % for Dice similarity coefficient (DSC), 2.3 ± 0.6 mm for mean absolute distance (MAD), and 1.9 ± 4.0 cm 3 for signed volume difference ( Δ V ). The average recorded interexpert difference measured on the same test dataset was 92% (DSC), 1.1 mm (MAD), and 2.1 cm 3 ( Δ V ). The proposed algorithm is fast, accurate, and robust for 3-D segmentation of the prostate on CT images.