Can input reconstruction be used to directly estimate uncertainty of a dose prediction U-Net model?

BACKGROUND: The reliable and efficient estimation of uncertainty in artificial intelligence (AI) models poses an ongoing challenge in many fields such as radiation therapy. AI models are intended to automate manual steps involved in the treatment planning workflow. We focus in this study on dose prediction models that predict an optimal dose trade-off for each new patient for a specific treatment modality. They can guide physicians in the optimization, be part of automatic treatment plan generation or support decision in treatment indication. Most common uncertainty estimation methods are based on Bayesian approximations, like Monte Carlo dropout (MCDO) or Deep ensembling (DE). These two techniques, however, have a high inference time (i.e., require multiple inference passes) and might not work for detecting out-of-distribution (OOD) data (i.e., overlapping uncertainty estimate for in-distribution (ID) and OOD). PURPOSE: In this study, we present a direct uncertainty estimation method and apply it for a dose prediction U-Net architecture. It can be used to flag OOD data and give information on the quality of the dose prediction. METHODS: Our method consists in the addition of a branch decoding from the bottleneck which reconstructs the CT scan given as input. The input reconstruction error can be used as a surrogate of the model uncertainty. For the proof-of-concept, our method is applied to proton therapy dose prediction in head and neck cancer patients. A dataset of 60 oropharyngeal patients was used to train the network using a nested cross-validation approach with 11 folds (training: 50 patients, validation: 5 patients, test: 5 patients). For the OOD experiment, we used 10 extra patients with a different head and neck sub-location. Accuracy, time-gain, and OOD detection are analyzed for our method in this particular application and compared with the popular MCDO and DE. RESULTS: The additional branch did not reduce the accuracy of the dose prediction model. The median absolute error is close to zero for the target volumes and less than 1% of the dose prescription for organs at risk. Our input reconstruction method showed a higher Pearson correlation coefficient with the prediction error (0.620) than DE (0.447) and MCDO (between 0.599 and 0.612). Moreover, our method allows an easier identification of OOD (no overlap for ID and OOD data and a Z-score of 34.05). The uncertainty is estimated simultaneously to the regression task, therefore requires less time and computational resources. CONCLUSIONS: This study shows that the error in the CT scan reconstruction can be used as a surrogate of the uncertainty of the model. The Pearson correlation coefficient with the dose prediction error is slightly higher than state-of-the-art techniques. OOD data can be more easily detected and the uncertainty metric is computed simultaneously to the regression task, therefore faster than MCDO or DE. The code and pretrained model are available on the gitlab repository: https://gitlab.com/ai4miro/ct-reconstruction-for-uncertainty-quatification-of-hdunet.

Patient selection for proton therapy using Normal Tissue Complication Probability with deep learning dose prediction for oropharyngeal cancer.

BACKGROUND: In cancer care, determining the most beneficial treatment technique is a key decision affecting the patient's survival and quality of life. Patient selection for proton therapy (PT) over conventional radiotherapy (XT) currently entails comparing manually generated treatment plans, which requires time and expertise. PURPOSE: We developed an automatic and fast tool, AI-PROTIPP (Artificial Intelligence Predictive Radiation Oncology Treatment Indication to Photons/Protons), that assesses quantitatively the benefits of each therapeutic option. Our method uses deep learning (DL) models to directly predict the dose distributions for a given patient for both XT and PT. By using models that estimate the Normal Tissue Complication Probability (NTCP), namely the likelihood of side effects to occur for a specific patient, AI-PROTIPP can propose a treatment selection quickly and automatically. METHODS: A database of 60 patients presenting oropharyngeal cancer, obtained from the Cliniques Universitaires Saint Luc in Belgium, was used in this study. For every patient, a PT plan and an XT plan were generated. The dose distributions were used to train the two dose DL prediction models (one for each modality). The model is based on U-Net architecture, a type of convolutional neural network currently considered as the state of the art for dose prediction models. A NTCP protocol used in the Dutch model-based approach, including grades II and III xerostomia and grades II and III dysphagia, was later applied in order to perform automatic treatment selection for each patient. The networks were trained using a nested cross-validation approach with 11-folds. We set aside three patients in an outer set and each fold consists of 47 patients in training, five in validation and five for testing. This method allowed us to assess our method on 55 patients (five patients per test times the number of folds). RESULTS: The treatment selection based on the DL-predicted doses reached an accuracy of 87.4% for the threshold parameters set by the Health Council of the Netherlands. The selected treatment is directly linked with these threshold parameters as they express the minimal gain brought by the PT treatment for a patient to be indicated to PT. To validate the performance of AI-PROTIPP in other conditions, we modulated these thresholds, and the accuracy was above 81% for all the considered cases. The difference in average cumulative NTCP per patient of predicted and clinical dose distributions is very similar (less than 1% difference). CONCLUSIONS: AI-PROTIPP shows that using DL dose prediction in combination with NTCP models to select PT for patients is feasible and can help to save time by avoiding the generation of treatment plans only used for the comparison. Moreover, DL models are transferable, allowing, in the future, experience to be shared with centers that would not have PT planning expertise.

Machine learning-based automatic proton therapy planning: Impact of post-processing and dose-mimicking in plan robustness.

PURPOSE: Automated treatment planning strategies are being widely implemented in clinical routines to reduce inter-planner variability, speed up the optimization process, and improve plan quality. This study aims to evaluate the feasibility and quality of intensity-modulated proton therapy (IMPT) plans generated with four different knowledge-based planning (KBP) pipelines fully integrated into a commercial treatment planning system (TPS). MATERIALS/METHODS: A data set containing 60 oropharyngeal cancer patients was split into 11 folds, each containing 47 patients for training, five patients for validation, and five patients for testing. A dose prediction model was trained on each of the folds, resulting in a total of 11 models. Three patients were left out in order to assess if the differences introduced between models were significant. From voxel-based dose predictions, we analyze the two steps that follow the dose prediction: post-processing of the predicted dose and dose mimicking (DM). We focused on the effect of post-processing (PP) or no post-processing (NPP) combined with two different DM algorithms for optimization: the one available in the commercial TPS RayStation (RSM) and a simpler isodose-based mimicking (IBM). Using 55 test patients (five test patients for each model), we evaluated the quality and robustness of the plans generated by the four proposed KBP pipelines (PP-RSM, PP-IBM, NPP-RSM, NPP-IBM). After robust evaluation, dose-volume histogram (DVH) metrics in nominal and worst-case scenarios were compared to those of the manually generated plans. RESULTS: Nominal doses from the four KBP pipelines showed promising results achieving comparable target coverage and improved dose to organs at risk (OARs) compared to the manual plans. However, too optimistic post-processing applied to the dose prediction (i.e. important decrease of the dose to the organs) compromised the robustness of the plans. Even though RSM seemed to partially compensate for the lack of robustness in the PP plans, still 65% of the patients did not achieve the expected robustness levels. NPP-RSM plans seemed to achieve the best trade-off between robustness and OAR sparing. DISCUSSION/CONCLUSIONS: PP and DM strategies are crucial steps to generate acceptable robust and deliverable IMPT plans from ML-predicted doses. Before the clinical implementation of any KBP pipeline, the PP and DM parameters predefined by the commercial TPS need to be modified accordingly with a comprehensive feedback loop in which the robustness of the final dose calculations is evaluated. With the right choice of PP and DM parameters, KBP strategies have the potential to generate IMPT plans within clinically acceptable levels comparable to plans manually generated by dosimetrists.