Nested block self-attention multiple resolution residual network for multiorgan segmentation from CT.

BACKGROUND: Fast and accurate multiorgans segmentation from computed tomography (CT) scans is essential for radiation treatment planning. Self-attention(SA)-based deep learning methodologies provide higher accuracies than standard methods but require memory and computationally intensive calculations, which restricts their use to relatively shallow networks. PURPOSE: Our goal was to develop and test a new computationally fast and memory-efficient bidirectional SA method called nested block self-attention (NBSA), which is applicable to shallow and deep multiorgan segmentation networks. METHODS: A new multiorgan segmentation method combining a deep multiple resolution residual network with computationally efficient SA called nested block SA (MRRN-NBSA) was developed and evaluated to segment 18 different organs from head and neck (HN) and abdomen organs. MRRN-NBSA combines features from multiple image resolutions and feature levels with SA to extract organ-specific contextual features. Computational efficiency is achieved by using memory blocks of fixed spatial extent for SA calculation combined with bidirectional attention flow. Separate models were trained for HN (n = 238) and abdomen (n = 30) and tested on set aside open-source grand challenge data sets for HN (n = 10) using a public domain database of computational anatomy and blinded testing on 20 cases from Beyond the Cranial Vault data set with overall accuracy provided by the grand challenge website for abdominal organs. Robustness to two-rater segmentations was also evaluated for HN cases using the open-source data set. Statistical comparison of MRRN-NBSA against Unet, convolutional network-based SA using criss-cross attention (CCA), dual SA, and transformer-based (UNETR) methods was done by measuring the differences in the average Dice similarity coefficient (DSC) accuracy for all HN organs using the Kruskall-Wallis test, followed by individual method comparisons using paired, two-sided Wilcoxon-signed rank tests at 95% confidence level with Bonferroni correction used for multiple comparisons. RESULTS: MRRN-NBSA produced an average high DSC of 0.88 for HN and 0.86 for the abdomen that exceeded current methods. MRRN-NBSA was more accurate than the computationally most efficient CCA (average DSC of 0.845 for HN, 0.727 for abdomen). Kruskal-Wallis test showed significant difference between evaluated methods (p=0.00025). Pair-wise comparisons showed significant differences between MRRN-NBSA than Unet (p=0.0003), CCA (p=0.030), dual (p=0.038), and UNETR methods (p=0.012) after Bonferroni correction. MRRN-NBSA produced less variable segmentations for submandibular glands (0.82 ± 0.06) compared to two raters (0.75 ± 0.31). CONCLUSIONS: MRRN-NBSA produced more accurate multiorgan segmentations than current methods on two different public data sets. Testing on larger institutional cohorts is required to establish feasibility for clinical use.

Prospectively-validated deep learning model for segmenting swallowing and chewing structures in CT.

Objective.Delineating swallowing and chewing structures aids in radiotherapy (RT) treatment planning to limit dysphagia, trismus, and speech dysfunction. We aim to develop an accurate and efficient method to automate this process.Approach.CT scans of 242 head and neck (H&N) cancer patients acquired from 2004 to 2009 at our institution were used to develop auto-segmentation models for the masseters, medial pterygoids, larynx, and pharyngeal constrictor muscle using DeepLabV3+. A cascaded framework was used, wherein models were trained sequentially to spatially constrain each structure group based on prior segmentations. Additionally, an ensemble of models, combining contextual information from axial, coronal, and sagittal views was used to improve segmentation accuracy. Prospective evaluation was conducted by measuring the amount of manual editing required in 91 H&N CT scans acquired February-May 2021.Main results. Medians and inter-quartile ranges of Dice similarity coefficients (DSC) computed on the retrospective testing set (N = 24) were 0.87 (0.85-0.89) for the masseters, 0.80 (0.79-0.81) for the medial pterygoids, 0.81 (0.79-0.84) for the larynx, and 0.69 (0.67-0.71) for the constrictor. Auto-segmentations, when compared to two sets of manual segmentations in 10 randomly selected scans, showed better agreement (DSC) with each observer than inter-observer DSC. Prospective analysis showed most manual modifications needed for clinical use were minor, suggesting auto-contouring could increase clinical efficiency. Trained segmentation models are available for research use upon request viahttps://github.com/cerr/CERR/wiki/Auto-Segmentation-models.Significance.We developed deep learning-based auto-segmentation models for swallowing and chewing structures in CT and demonstrated its potential for use in treatment planning to limit complications post-RT. To the best of our knowledge, this is the only prospectively-validated deep learning-based model for segmenting chewing and swallowing structures in CT. Segmentation models have been made open-source to facilitate reproducibility and multi-institutional research.

Deep learning auto-segmentation and automated treatment planning for trismus risk reduction in head and neck cancer radiotherapy.

BACKGROUND AND PURPOSE: Reducing trismus in radiotherapy for head and neck cancer (HNC) is important. Automated deep learning (DL) segmentation and automated planning was used to introduce new and rarely segmented masticatory structures to study if trismus risk could be decreased. MATERIALS AND METHODS: Auto-segmentation was based on purpose-built DL, and automated planning used our in-house system, ECHO. Treatment plans for ten HNC patients, treated with 2 Gy × 35 fractions, were optimized (ECHO0). Six manually segmented OARs were replaced with DL auto-segmentations and the plans re-optimized (ECHO1). In a third set of plans, mean doses for auto-segmented ipsilateral masseter and medial pterygoid (MIMean, MPIMean), derived from a trismus risk model, were implemented as dose-volume objectives (ECHO2). Clinical dose-volume criteria were compared between the two scenarios (ECHO0 vs. ECHO1; ECHO1 vs. ECHO2; Wilcoxon signed-rank test; significance: p < 0.01). RESULTS: Small systematic differences were observed between the doses to the six auto-segmented OARs and their manual counterparts (median: ECHO1 = 6.2 (range: 0.4, 21) Gy vs. ECHO0 = 6.6 (range: 0.3, 22) Gy; p = 0.007), and the ECHO1 plans provided improved normal tissue sparing across a larger dose-volume range. Only in the ECHO2 plans, all patients fulfilled both MIMean and MPIMean criteria. The population median MIMean and MPIMean were considerably lower than those suggested by the trismus model (ECHO0: MIMean = 13 Gy vs. ≤42 Gy; MPIMean = 29 Gy vs. ≤68 Gy). CONCLUSIONS: Automated treatment planning can efficiently incorporate new structures from DL auto-segmentation, which results in trismus risk sparing without deteriorating treatment plan quality. Auto-planning and deep learning auto-segmentation together provide a powerful platform to further improve treatment planning.

Metrics to evaluate the performance of auto-segmentation for radiation treatment planning: A critical review.

Advances in artificial intelligence-based methods have led to the development and publication of numerous systems for auto-segmentation in radiotherapy. These systems have the potential to decrease contour variability, which has been associated with poor clinical outcomes and increased efficiency in the treatment planning workflow. However, there are no uniform standards for evaluating auto-segmentation platforms to assess their efficacy at meeting these goals. Here, we review the most frequently used evaluation techniques which include geometric overlap, dosimetric parameters, time spent contouring, and clinical rating scales. These data suggest that many of the most commonly used geometric indices, such as the Dice Similarity Coefficient, are not well correlated with clinically meaningful endpoints. As such, a multi-domain evaluation, including composite geometric and/or dosimetric metrics with physician-reported assessment, is necessary to gauge the clinical readiness of auto-segmentation for radiation treatment planning.

Clinical implementation of deep learning contour autosegmentation for prostate radiotherapy.

BACKGROUND AND PURPOSE: Artificial intelligence advances have stimulated a new generation of autosegmentation, however clinical evaluations of these algorithms are lacking. This study assesses the clinical utility of deep learning-based autosegmentation for MR-based prostate radiotherapy planning. MATERIALS AND METHODS: Data was collected prospectively for patients undergoing prostate-only radiation at our institution from June to December 2019. Geometric indices (volumetric Dice-Sørensen Coefficient, VDSC; surface Dice-Sørensen Coefficient, SDSC; added path length, APL) compared automated to final contours. Physicians reported contouring time and rated autocontours on 3-point protocol deviation scales. Descriptive statistics and univariable analyses evaluated relationships between the aforementioned metrics. RESULTS: Among 173 patients, 85% received SBRT. The CTV was available for 167 (97%) with median VDSC, SDSC, and APL for CTV (prostate and SV) 0.89 (IQR 0.83-0.95), 0.91 (IQR 0.75-0.96), and 1801 mm (IQR 1140-2703), respectively. Physicians completed surveys for 43/55 patients (RR 78%). 33% of autocontours (14/43) required major "clinically significant" edits. Physicians spent a median of 28 min contouring (IQR 20-30), representing a 12-minute (30%) time savings compared to historic controls (median 40, IQR 25-68, n = 21, p < 0.01). Geometric indices correlated weakly with contouring time, and had no relationship with quality scores. CONCLUSION: Deep learning-based autosegmentation was implemented successfully and improved efficiency. Major "clinically significant" edits are uncommon and do not correlate with geometric indices. APL was supported as a clinically meaningful quantitative metric. Efforts are needed to educate and generate consensus among physicians, and develop mechanisms to flag cases for quality assurance.

Library of deep-learning image segmentation and outcomes model-implementations.

An open-source library of implementations for deep-learning-based image segmentation and outcomes models based on radiotherapy and radiomics is presented. As oncology treatment planning becomes increasingly driven by automation, such a library of model implementations is crucial to (i) validate existing models on datasets collected at different institutions, (ii) automate segmentation, (iii) create ensembles for improving performance and (iv) incorporate validated models in the clinical workflow. Inclusion of deep-learning-based image segmentation and outcomes models in the same library provides a fully automated and reproduceable pipeline to estimate prognosis. The library was developed with the Computational Environment for Radiological Research (CERR) software platform. Centralizing model implementations in CERR builds upon its rich set of radiotherapy and radiomics tools and caters to the world-wide user base. CERR provides well-validated feature extraction pipelines for radiotherapy dosimetry and radiomics with fine control over the calculation settings, allowing users to select appropriate parameters used in model derivation. Models for automatic image segmentation are distributed via containers, allowing them to be deployed with a variety of scientific computing architectures. The library includes implementations of popular DVH-based models outlined in the Quantitative Analysis of Normal Tissue Effects in the Clinic effort and recently published literature. Radiomics models include features from the Image Biomarker Standardization Initiative and application-specific features found to be relevant across multiple sites and image modalities. The library is distributed as a module within CERR at https://www.github.com/cerr/CERR under the GNU-GPL copyleft with additional restrictions on clinical and commercial use and provision to dual license in future.

Efficiency and safety increases after the implementation of a multi-institutional automated plan check tool at our institution.

PURPOSE: The plan check tool (PCT) is the result of a multi-institutional collaboration to jointly develop a flexible automated plan checking framework designed with the versatility to be shared across collaborating facilities while supporting the individual differences between practices. We analyze the effect that PCT has had on the efficiency and effectiveness of initial chart checks at our institution. METHODS AND MATERIALS: Data on errors identified during initial chart checks were acquired during two time periods: before the introduction of PCT in the clinic (6/24/2015 to 7/31/2015, 187 checks) and post-clinical release (4/14/2016 to 5/2/2016, 186 checks). During each time period, human plan checkers were asked to record all issues that they either manually detected or that were detected by PCT as well as the amount of time, less breaks, or interruptions, it took to check each plan. RESULTS: After the clinical release of PCT, there was a statistically significant decrease in the number of issues recorded by the human plan checkers both related to checks explicitly performed by PCT (13 vs 50, P < 0.001) and in issues identified overall (127 vs 200, P < 0.001). The mean and medium time for a plan check decreased by 20%. CONCLUSIONS: The use of a multi-institutional, configurable, automated plan checking tool has resulted in both substantial gains in efficiency and moving error detection to earlier points in the planning process, decreasing their likelihood that they reach the patient. The sizeable startup effort needed to create this tool from scratch was mitigated by the sharing, and subsequent co-development, of software code from a peer institution.

Deep learning-based auto-segmentation of targets and organs-at-risk for magnetic resonance imaging only planning of prostate radiotherapy.

BACKGROUND AND PURPOSE: Magnetic resonance (MR) only radiation therapy for prostate treatment provides superior contrast for defining targets and organs-at-risk (OARs). This study aims to develop a deep learning model to leverage this advantage to automate the contouring process. MATERIALS AND METHODS: Six structures (bladder, rectum, urethra, penile bulb, rectal spacer, prostate and seminal vesicles) were contoured and reviewed by a radiation oncologist on axial T2-weighted MR image sets from 50 patients, which constituted expert delineations. The data was split into a 40/10 training and validation set to train a two-dimensional fully convolutional neural network, DeepLabV3+, using transfer learning. The T2-weighted image sets were pre-processed to 2D false color images to leverage pre-trained (from natural images) convolutional layers' weights. Independent testing was performed on an additional 50 patient's MR scans. Performance comparison was done against a U-Net deep learning method. Algorithms were evaluated using volumetric Dice similarity coefficient (VDSC) and surface Dice similarity coefficient (SDSC). RESULTS: When comparing VDSC, DeepLabV3+ significantly outperformed U-Net for all structures except urethra (P < 0.001). Average VDSC was 0.93 ± 0.04 (bladder), 0.83 ± 0.06 (prostate and seminal vesicles [CTV]), 0.74 ± 0.13 (penile bulb), 0.82 ± 0.05 (rectum), 0.69 ± 0.10 (urethra), and 0.81 ± 0.1 (rectal spacer). Average SDSC was 0.92 ± 0.1 (bladder), 0.85 ± 0.11 (prostate and seminal vesicles [CTV]), 0.80 ± 0.22 (penile bulb), 0.87 ± 0.07 (rectum), 0.85 ± 0.25 (urethra), and 0.83 ± 0.26 (rectal spacer). CONCLUSION: A deep learning-based model produced contours that show promise to streamline an MR-only planning workflow in treating prostate cancer.

Five years' experience with a customized electronic checklist for radiation therapy planning quality assurance in a multicampus institution.

INTRODUCTION: An electronic checklist has been designed with the intention of reducing errors while minimizing user effort in completing the checklist. We analyze the clinical use and evolution of the checklist over the past 5 years and review data in an incident learning system (ILS) to investigate whether it has contributed to an improvement in patient safety. METHODS AND MATERIALS: The checklist is written as a standalone HTML application using VBScript. User selection of pertinent demographic details limits the display of checklist items only to those necessary for the particular clinical scenario. Ten common clinical scenarios were used to illustrate the difference between the maximum possible number of checklist items available in the code versus the number displayed to the user at any one time. An ILS database of errors and near misses was reviewed to evaluate whether the checklist influenced the occurrence of reported events. RESULTS: Over 5 years, the number of checklist items available in the code nearly doubled, whereas the number displayed to the user at any one time stayed constant. Events reported in our ILS related to the beam energy used with pacemakers, projection of anatomy on digitally reconstructed radiographs, orthogonality of setup fields, and field extension beyond match lines, did not recur after the items were added to the checklist. Other events related to bolus documentation and breakpoints continued to be reported. CONCLUSION: Our checklist is adaptable to the introduction of new technologies, transitions between planning systems, and to errors and near misses recorded in the ILS. The electronic format allows us to restrict user display to a small, relevant, subset of possible checklist items, limiting the planner effort needed to review and complete the checklist.