Deep learning-based ultrasound auto-segmentation of the prostate with brachytherapy implanted needles.

BACKGROUND: Accurate segmentation of the clinical target volume (CTV) corresponding to the prostate with or without proximal seminal vesicles is required on transrectal ultrasound (TRUS) images during prostate brachytherapy procedures. Implanted needles cause artifacts that may make this task difficult and time-consuming. Thus, previous studies have focused on the simpler problem of segmentation in the absence of needles at the cost of reduced clinical utility. PURPOSE: To use a convolutional neural network (CNN) algorithm for segmentation of the prostatic CTV in TRUS images post-needle insertion obtained from prostate brachytherapy procedures to better meet the demands of the clinical procedure. METHODS: A dataset consisting of 144 3-dimensional (3D) TRUS images with implanted metal brachytherapy needles and associated manual CTV segmentations was used for training a 2-dimensional (2D) U-Net CNN using a Dice Similarity Coefficient (DSC) loss function. These were split by patient, with 119 used for training and 25 reserved for testing. The 3D TRUS training images were resliced at radial (around the axis normal to the coronal plane) and oblique angles through the center of the 3D image, as well as axial, coronal, and sagittal planes to obtain 3689 2D TRUS images and masks for training. The network generated boundary predictions on 300 2D TRUS images obtained from reslicing each of the 25 3D TRUS images used for testing into 12 radial slices (15° apart), which were then reconstructed into 3D surfaces. Performance metrics included DSC, recall, precision, unsigned and signed volume percentage differences (VPD/sVPD), mean surface distance (MSD), and Hausdorff distance (HD). In addition, we studied whether providing algorithm-predicted boundaries to the physicians and allowing modifications increased the agreement between physicians. This was performed by providing a subset of 3D TRUS images of five patients to five physicians who segmented the CTV using clinical software and repeated this at least 1 week apart. The five physicians were given the algorithm boundary predictions and allowed to modify them, and the resulting inter- and intra-physician variability was evaluated. RESULTS: Median DSC, recall, precision, VPD, sVPD, MSD, and HD of the 3D-reconstructed algorithm segmentations were 87.2 [84.1, 88.8]%, 89.0 [86.3, 92.4]%, 86.6 [78.5, 90.8]%, 10.3 [4.5, 18.4]%, 2.0 [-4.5, 18.4]%, 1.6 [1.2, 2.0] mm, and 6.0 [5.3, 8.0] mm, respectively. Segmentation time for a set of 12 2D radial images was 2.46 [2.44, 2.48] s. With and without U-Net starting points, the intra-physician median DSCs were 97.0 [96.3, 97.8]%, and 94.4 [92.5, 95.4]% (p < 0.0001), respectively, while the inter-physician median DSCs were 94.8 [93.3, 96.8]% and 90.2 [88.7, 92.1]%, respectively (p < 0.0001). The median segmentation time for physicians, with and without U-Net-generated CTV boundaries, were 257.5 [211.8, 300.0] s and 288.0 [232.0, 333.5] s, respectively (p = 0.1034). CONCLUSIONS: Our algorithm performed at a level similar to physicians in a fraction of the time. The use of algorithm-generated boundaries as a starting point and allowing modifications reduced physician variability, although it did not significantly reduce the time compared to manual segmentations.

Validation of a surface-based deformable MRI-3D ultrasound image registration algorithm toward clinical implementation for interstitial prostate brachytherapy.

PURPOSE: The purpose of this study was to evaluate and clinically implement a deformable surface-based magnetic resonance imaging (MRI) to three-dimensional ultrasound (US) image registration algorithm for prostate brachytherapy (BT) with the aim to reduce operator dependence and facilitate dose escalation to an MRI-defined target. METHODS AND MATERIALS: Our surface-based deformable image registration (DIR) algorithm first translates and scales to align the US- and MR-defined prostate surfaces, followed by deformation of the MR-defined prostate surface to match the US-defined prostate surface. The algorithm performance was assessed in a phantom using three deformation levels, followed by validation in three retrospective high-dose-rate BT clinical cases. For comparison, manual rigid registration and cognitive fusion by physician were also employed. Registration accuracy was assessed using the Dice similarity coefficient (DSC) and target registration error (TRE) for embedded spherical landmarks. The algorithm was then implemented intraoperatively in a prospective clinical case. RESULTS: In the phantom, our DIR algorithm demonstrated a mean DSC and TRE of 0.74 ± 0.08 and 0.94 ± 0.49 mm, respectively, significantly improving the performance compared to manual rigid registration with 0.64 ± 0.16 and 1.88 ± 1.24 mm, respectively. Clinical results demonstrated reduced variability compared to the current standard of cognitive fusion by physicians. CONCLUSIONS: We successfully validated a DIR algorithm allowing for translation of MR-defined target and organ-at-risk contours into the intraoperative environment. Prospective clinical implementation demonstrated the intraoperative feasibility of our algorithm, facilitating targeted biopsies and dose escalation to the MR-defined lesion. This method provides the potential to standardize the registration procedure between physicians, reducing operator dependence.

Three-dimensional therapy needle applicator segmentation for ultrasound-guided focal liver ablation.

PURPOSE: Minimally invasive procedures, such as microwave ablation, are becoming first-line treatment options for early-stage liver cancer due to lower complication rates and shorter recovery times than conventional surgical techniques. Although these procedures are promising, one reason preventing widespread adoption is inadequate local tumor ablation leading to observations of higher local cancer recurrence compared to conventional procedures. Poor ablation coverage has been associated with two-dimensional (2D) ultrasound (US) guidance of the therapy needle applicators and has stimulated investigation into the use of three-dimensional (3D) US imaging for these procedures. We have developed a supervised 3D US needle applicator segmentation algorithm using a single user input to augment the addition of 3D US to the current focal liver tumor ablation workflow with the goals of identifying and improving needle applicator localization efficiency. METHODS: The algorithm is initialized by creating a spherical search space of line segments around a manually chosen seed point that is selected by a user on the needle applicator visualized in a 3D US image. The most probable trajectory is chosen by maximizing the count and intensity of threshold voxels along a line segment and is filtered using the Otsu method to determine the tip location. Homogeneous tissue mimicking phantom images containing needle applicators were used to optimize the parameters of the algorithm prior to a four-user investigation on retrospective 3D US images of patients who underwent microwave ablation for liver cancer. Trajectory, axis localization, and tip errors were computed based on comparisons to manual segmentations in 3D US images. RESULTS: Segmentation of needle applicators in ten phantom 3D US images was optimized to median (Q1, Q3) trajectory, axis, and tip errors of 2.1 (1.1, 3.6)°, 1.3 (0.8, 2.1) mm, and 1.3 (0.7, 2.5) mm, respectively, with a mean ± SD segmentation computation time of 0.246 ± 0.007 s. Use of the segmentation method with a 16 in vivo 3D US patient dataset resulted in median (Q1, Q3) trajectory, axis, and tip errors of 4.5 (2.4, 5.2)°, 1.9 (1.7, 2.1) mm, and 5.1 (2.2, 5.9) mm based on all users. CONCLUSIONS: Segmentation of needle applicators in 3D US images during minimally invasive liver cancer therapeutic procedures could provide a utility that enables enhanced needle applicator guidance, placement verification, and improved clinical workflow. A semi-automated 3D US needle applicator segmentation algorithm used in vivo demonstrated localization of the visualized trajectory and tip with less than 5° and 5.2 mm errors, respectively, in less than 0.31 s. This offers the ability to assess and adjust needle applicator placements intraoperatively to potentially decrease the observed liver cancer recurrence rates associated with current ablation procedures. Although optimized for deep and oblique angle needle applicator insertions, this proposed workflow has the potential to be altered for a variety of image-guided minimally invasive procedures to improve localization and verification of therapy needle applicators intraoperatively.

Accuracy and variability of high-dose-rate prostate brachytherapy needle tip localization using live two-dimensional and sagittally reconstructed three-dimensional ultrasound.

PURPOSE: To measure the accuracy and variability of manual high-dose-rate (HDR) prostate brachytherapy (BT) needle tip localization using sagittally reconstructed three-dimensional (3D) transrectal ultrasound (TRUS) augmented with live two-dimensional (2D) sagittal TRUS. METHODS AND MATERIALS: Ten prostate cancer patients underwent HDR-BT during which the sagittally assisted sagittally reconstructed (SASR) segmentation technique was completed in parallel with commercially available sagittally assisted axially reconstructed (SAAR) TRUS for comparison. The SASR technique makes use of live 2D ultrasound intraoperatively and allows needle tip updates using the final 3D image in the absence of image artifacts. These updates were repeated offline twice by two separate users. Needle end-length measurements were used to calculate insertion depth errors (IDEs) for each technique. RESULTS: Images of 147 needles were analyzed. For the SASR technique, both users were confident in tip positions on the final 3D image within 3 mm for 52% of needles, so these tip positions were updated. For the remaining 48% of needles, the tip positions from the live 2D images were used. This SASR technique enabled the localization of all needles with IDEs within ±3 mm for 84% of needles and IDE range of [-6.2 mm, 5.9 mm], compared with 57% and [-8.1 mm, 7.7 mm] when using the commercially available SAAR technique. CONCLUSIONS: The SASR technique mitigates the impact of 3D TRUS image artifacts on HDR-BT needle tip localization by incorporating live 2D sagittal TRUS intraoperatively and provides a statistically significant reduction in IDE variance compared with the routine SAAR technique.

Simultaneous automatic segmentation of multiple needles using 3D ultrasound for high-dose-rate prostate brachytherapy.

PURPOSE: Sagittally reconstructed 3D (SR3D) ultrasound imaging shows promise for improved needle localization for high-dose-rate prostate brachytherapy (HDR-BT); however, needles must be manually segmented intraoperatively while the patient is anesthetized to create a treatment plan. The purpose of this article was to describe and validate an automatic needle segmentation algorithm designed for HDR-BT, specifically capable of simultaneously segmenting all needles in an HDR-BT implant using a single SR3D image with ~5 mm interneedle spacing. MATERIALS AND METHODS: The segmentation algorithm involves regularized feature point classification and line trajectory identification based on the randomized 3D Hough transform modified to handle multiple straight needles in a single image simultaneously. Needle tips are identified based on peaks in the derivative of the signal intensity profile along the needle trajectory. For algorithm validation, 12 prostate cancer patients underwent HDR-BT during which SR3D images were acquired with all needles in place. Needles present in each of the 12 images were segmented manually, providing a gold standard for comparison, and using the algorithm. Tip errors were assessed in terms of the 3D Euclidean distance between needle tips, and trajectory error was assessed in terms of 2D distance in the axial plane and angular deviation between trajectories. RESULTS: In total, 190 needles were investigated. Mean execution time of the algorithm was 11.0 s per patient, or 0.7 s per needle. The algorithm identified 82% and 85% of needle tips with 3D errors ≤3 mm and ≤5 mm, respectively, 91% of needle trajectories with 2D errors in the axial plane ≤3 mm, and 83% of needle trajectories with angular errors ≤3°. The largest tip error component was in the needle insertion direction. CONCLUSIONS: Previous work has indicated HDR-BT needles may be manually segmented using SR3D images with insertion depth errors ≤3 mm and ≤5 mm for 83% and 92% of needles, respectively. The algorithm shows promise for reducing the time required for the segmentation of straight HDR-BT needles, and future work involves improving needle tip localization performance through improved image quality and modeling curvilinear trajectories.

Three-dimensional transrectal ultrasound guided high-dose-rate prostate brachytherapy: A comparison of needle segmentation accuracy with two-dimensional image guidance.

PURPOSE: Conventional transrectal ultrasound guided high-dose-rate prostate brachytherapy (HDR-BT) uses an axially acquired image set for organ segmentation and 2D sagittal images for needle segmentation. Sagittally reconstructed 3D (SR3D) transrectal ultrasound enables both organ and needle segmentation and has the potential to reduce organ-needle alignment uncertainty. This study compares the accuracy of needle tip localization between the conventional 2D sagittally assisted axially reconstructed (SAAR) and SR3D approaches. METHODS AND MATERIALS: Twelve patients underwent SAAR-guided HDR-BT, during which SR3D images were acquired for subsequent segmentation and analysis. A total of 183 needles were investigated. Needle end-length measurements were taken, providing a gold standard for insertion depths. Dosimetric impact of insertion depth errors (IDEs) on clinical treatment plans was assessed. RESULTS: SR3D guidance provided statistically significantly smaller IDEs than SAAR guidance with a mean ± SD of -0.6 ± 3.2 mm and 2.8 ± 3.2 mm, respectively (p < 0.001). Shadow artifacts were found to obstruct the view of some needle tips in SR3D images either partially (12%) or fully (10%); however, SR3D IDEs had a statistically significantly smaller impact on prostate V100% than SAAR IDEs with mean ± SD decreases of -1.2 ± 1.3% and -6.5 ± 6.7%, respectively (p < 0.05). CONCLUSIONS: SR3D-guided HDR-BT eliminates a source of systematic uncertainty from the SAAR-guided approach, providing decreased IDEs for most needles, leading to a significant decrease in dosimetric uncertainty. Although imaging artifacts can limit the accuracy of tip localization in a subset of needles, we identified a method to mitigate these artifacts for clinical implementation.

A compact mechatronic system for 3D ultrasound guided prostate interventions.

PURPOSE: Ultrasound imaging has improved the treatment of prostate cancer by producing increasingly higher quality images and influencing sophisticated targeting procedures for the insertion of radioactive seeds during brachytherapy. However, it is critical that the needles be placed accurately within the prostate to deliver the therapy to the planned location and avoid complications of damaging surrounding tissues. METHODS: The authors have developed a compact mechatronic system, as well as an effective method for guiding and controlling the insertion of transperineal needles into the prostate. This system has been designed to allow guidance of a needle obliquely in 3D space into the prostate, thereby reducing pubic arch interference. The choice of needle trajectory and location in the prostate can be adjusted manually or with computer control. RESULTS: To validate the system, a series of experiments were performed on phantoms. The 3D scan of the string phantom produced minimal geometric error, which was less than 0.4 mm. Needle guidance accuracy tests in agar prostate phantoms showed that the mean error of bead placement was less then 1.6 mm along parallel needle paths that were within 1.2 mm of the intended target and 1 degree from the preplanned trajectory. At oblique angles of up to 15 degrees relative to the probe axis, beads were placed to within 3.0 mm along a trajectory that were within 2.0 mm of the target with an angular error less than 2 degrees. CONCLUSIONS: By combining 3D TRUS imaging system to a needle tracking linkage, this system should improve the physician's ability to target and accurately guide a needle to selected targets without the need for the computer to directly manipulate and insert the needle. This would be beneficial as the physician has complete control of the system and can safely maneuver the needle guide around obstacles such as previously placed needles.