Deformation driven Seq2Seq longitudinal tumor and organs-at-risk prediction for radiotherapy.

PURPOSE: Radiotherapy presents unique challenges and clinical requirements for longitudinal tumor and organ-at-risk (OAR) prediction during treatment. The challenges include tumor inflammation/edema and radiation-induced changes in organ geometry, whereas the clinical requirements demand flexibility in input/output sequence timepoints to update the predictions on rolling basis and the grounding of all predictions in relationship to the pre-treatment imaging information for response and toxicity assessment in adaptive radiotherapy. METHODS: To deal with the aforementioned challenges and to comply with the clinical requirements, we present a novel 3D sequence-to-sequence model based on Convolution Long Short-Term Memory (ConvLSTM) that makes use of series of deformation vector fields (DVFs) between individual timepoints and reference pre-treatment/planning CTs to predict future anatomical deformations and changes in gross tumor volume as well as critical OARs. High-quality DVF training data are created by employing hyper-parameter optimization on the subset of the training data with DICE coefficient and mutual information metric. We validated our model on two radiotherapy datasets: a publicly available head-and-neck dataset (28 patients with manually contoured pre-, mid-, and post-treatment CTs), and an internal non-small cell lung cancer dataset (63 patients with manually contoured planning CT and 6 weekly CBCTs). RESULTS: The use of DVF representation and skip connections overcomes the blurring issue of ConvLSTM prediction with the traditional image representation. The mean and standard deviation of DICE for predictions of lung GTV at weeks 4, 5, and 6 were 0.83 ± 0.09, 0.82 ± 0.08, and 0.81 ± 0.10, respectively, and for post-treatment ipsilateral and contralateral parotids, were 0.81 ± 0.06 and 0.85 ± 0.02. CONCLUSION: We presented a novel DVF-based Seq2Seq model for medical images, leveraging the complete 3D imaging information of a relatively large longitudinal clinical dataset, to carry out longitudinal GTV/OAR predictions for anatomical changes in HN and lung radiotherapy patients, which has potential to improve RT outcomes.

Multitask 3D CBCT-to-CT translation and organs-at-risk segmentation using physics-based data augmentation.

PURPOSE: In current clinical practice, noisy and artifact-ridden weekly cone beam computed tomography (CBCT) images are only used for patient setup during radiotherapy. Treatment planning is performed once at the beginning of the treatment using high-quality planning CT (pCT) images and manual contours for organs-at-risk (OARs) structures. If the quality of the weekly CBCT images can be improved while simultaneously segmenting OAR structures, this can provide critical information for adapting radiotherapy mid-treatment as well as for deriving biomarkers for treatment response. METHODS: Using a novel physics-based data augmentation strategy, we synthesize a large dataset of perfectly/inherently registered pCT and synthetic-CBCT pairs for locally advanced lung cancer patient cohort, which are then used in a multitask three-dimensional (3D) deep learning framework to simultaneously segment and translate real weekly CBCT images to high-quality pCT-like images. RESULTS: We compared the synthetic CT and OAR segmentations generated by the model to real pCT and manual OAR segmentations and showed promising results. The real week 1 (baseline) CBCT images which had an average mean absolute error (MAE) of 162.77 HU compared to pCT images are translated to synthetic CT images that exhibit a drastically improved average MAE of 29.31 HU and average structural similarity of 92% with the pCT images. The average DICE scores of the 3D OARs segmentations are: lungs 0.96, heart 0.88, spinal cord 0.83, and esophagus 0.66. CONCLUSIONS: We demonstrate an approach to translate artifact-ridden CBCT images to high-quality synthetic CT images, while simultaneously generating good quality segmentation masks for different OARs. This approach could allow clinicians to adjust treatment plans using only the routine low-quality CBCT images, potentially improving patient outcomes. Our code, data, and pre-trained models will be made available via our physics-based data augmentation library, Physics-ArX, at https://github.com/nadeemlab/Physics-ArX.

Generalizable cone beam CT esophagus segmentation using physics-based data augmentation.

Automated segmentation of the esophagus is critical in image-guided/adaptive radiotherapy of lung cancer to minimize radiation-induced toxicities such as acute esophagitis. We have developed a semantic physics-based data augmentation method for segmenting the esophagus in both planning CT (pCT) and cone beam CT (CBCT) using 3D convolutional neural networks. One hundred and ninety-one cases with their pCTs and CBCTs from four independent datasets were used to train a modified 3D U-Net architecture and a multi-objective loss function specifically designed for soft-tissue organs such as the esophagus. Scatter artifacts and noises were extracted from week-1 CBCTs using a power-law adaptive histogram equalization method and induced to the corresponding pCT were reconstructed using CBCT reconstruction parameters. Moreover, we leveraged physics-based artifact induction in pCTs to drive the esophagus segmentation in real weekly CBCTs. Segmentations were evaluated using the geometric Dice coefficient and Hausdorff distance as well as dosimetrically using mean esophagus dose and D 5cc. Due to the physics-based data augmentation, our model trained just on the synthetic CBCTs was robust and generalizable enough to also produce state-of-the-art results on the pCTs and CBCTs, achieving Dice overlaps of 0.81 and 0.74, respectively. It is concluded that our physics-based data augmentation spans the realistic noise/artifact spectrum across patient CBCT/pCT data and can generalize well across modalities, eventually improving the accuracy of treatment setup and response analysis.

Early Prediction of Acute Esophagitis for Adaptive Radiation Therapy.

PURPOSE: Acute esophagitis (AE) is a common dose-limiting toxicity in radiation therapy of locally advanced non-small cell lung cancer (LA-NSCLC). We developed an early AE prediction model from weekly accumulated esophagus dose and its associated local volumetric change. METHODS AND MATERIALS: Fifty-one patients with LA-NSCLC underwent treatment with intensity modulated radiation therapy to 60 Gy in 2-Gy fractions with concurrent chemotherapy and weekly cone beam computed tomography (CBCT). Twenty-eight patients (55%) developed grade ≥2 AE (≥AE2) at a median of 4 weeks after the start of radiation therapy. For early ≥AE2 prediction, the esophagus on CBCT of the first 2 weeks was deformably registered to the planning computed tomography images, and weekly esophagus dose was accumulated. Week 1-to-week 2 (w1→w2) esophagus volume changes including maximum esophagus expansion (MEex%) and volumes with ≥x% local expansions (VEx%; x = 5, 10, 15) were calculated from the Jacobian map of deformation vector field gradients. Logistic regression model with 5-fold cross-validation was built using combinations of the accumulated mean esophagus doses (MED) and the esophagus change parameters with the lowest P value in univariate analysis. The model was validated on an additional 18 and 11 patients with weekly CBCT and magnetic resonance imaging (MRI), respectively, and compared with models using only planned mean dose (MEDPlan). Performance was assessed using area under the curve (AUC) and Hosmer-Lemeshow test (PHL). RESULTS: Univariately, w1→w2 VE10% (P = .004), VE5% (P = .01) and MEex% (P = .02) significantly predicted ≥AE2. A model combining MEDW2 and w1→w2 VE10% had the best performance (AUC = 0.80; PHL = 0.43), whereas the MEDPlan model had a lower accuracy (AUC = 0.67; PHL = 0.26). The combined model also showed high accuracy in the CBCT (AUC = 0.78) and MRI validations (AUC = 0.75). CONCLUSIONS: A CBCT-based, cross-validated, and internally validated model on MRI with a combination of accumulated esophagus dose and local volume change from the first 2 weeks of chemotherapy significantly improved AE prediction compared with conventional models using only the planned dose. This model could inform plan adaptation early to lower the risk of esophagitis.

Reproducible and Interpretable Spiculation Quantification for Lung Cancer Screening.

Spiculations are important predictors of lung cancer malignancy, which are spikes on the surface of the pulmonary nodules. In this study, we proposed an interpretable and parameter-free technique to quantify the spiculation using area distortion metric obtained by the conformal (angle-preserving) spherical parameterization. We exploit the insight that for an angle-preserved spherical mapping of a given nodule, the corresponding negative area distortion precisely characterizes the spiculations on that nodule. We introduced novel spiculation scores based on the area distortion metric and spiculation measures. We also semi-automatically segment lung nodule (for reproducibility) as well as vessel and wall attachment to differentiate the real spiculations from lobulation and attachment. A simple pathological malignancy prediction model is also introduced. We used the publicly-available LIDC-IDRI dataset pathologists (strong-label) and radiologists (weak-label) ratings to train and test radiomics models containing this feature, and then externally validate the models. We achieved AUC = 0.80 and 0.76, respectively, with the models trained on the 811 weakly-labeled LIDC datasets and tested on the 72 strongly-labeled LIDC and 73 LUNGx datasets; the previous best model for LUNGx had AUC = 0.68. The number-of-spiculations feature was found to be highly correlated (Spearman's rank correlation coefficient ρ=0.44) with the radiologists' spiculation score. We developed a reproducible and interpretable, parameter-free technique for quantifying spiculations on nodules. The spiculation quantification measures was then applied to the radiomics framework for pathological malignancy prediction with reproducible semi-automatic segmentation of nodule. Using our interpretable features (size, attachment, spiculation, lobulation), we were able to achieve higher performance than previous models. In the future, we will exhaustively test our model for lung cancer screening in the clinic.