A distributed feature selection pipeline for survival analysis using radiomics in non-small cell lung cancer patients.

Predictive modelling of cancer outcomes using radiomics faces dimensionality problems and data limitations, as radiomics features often number in the hundreds, and multi-institutional data sharing is ()often unfeasible. Federated learning (FL) and feature selection (FS) techniques combined can help overcome these issues, as one provides the means of training models without exchanging sensitive data, while the other identifies the most informative features, reduces overfitting, and improves model interpretability. Our proposed FS pipeline based on FL principles targets data-driven radiomics FS in a multivariate survival study of non-small cell lung cancer patients. The pipeline was run across datasets from three institutions without patient-level data exchange. It includes two FS techniques, Correlation-based Feature Selection and LASSO regularization, and Cox Proportional-Hazard regression with Overall Survival as endpoint. Trained and validated on 828 patients overall, our pipeline yielded a radiomic signature comprising "intensity-based energy" and "mean discretised intensity". Validation resulted in a mean Harrell C-index of 0.59, showcasing fair efficacy in risk stratification. In conclusion, we suggest a distributed radiomics approach that incorporates preliminary feature selection to systematically decrease the feature set based on data-driven considerations. This aims to address dimensionality challenges beyond those associated with data constraints and interpretability concerns.

Estimating how contouring differences affect normal tissue complication probability modelling.

Background and purpose: Normal tissue complication probability (NTCP) models are developed from large retrospective datasets where automatic contouring is often used to contour the organs at risk. This study proposes a methodology to estimate how discrepancies between two sets of contours are reflected on NTCP model performance. We apply this methodology to heart contours within a dataset of non-small cell lung cancer (NSCLC) patients. Materials and methods: One of the contour sets is designated the ground truth and a dosimetric parameter derived from it is used to simulate outcomes via a predefined NTCP relationship. For each simulated outcome, the selected dosimetric parameters associated with each contour set are individually used to fit a toxicity model and their performance is compared. Our dataset comprised 605 stage IIA-IIIB NSCLC patients. Manual, deep learning, and atlas-based heart contours were available. Results: How contour differences were reflected in NTCP model performance depended on the slope of the predefined model, the dosimetric parameter utilized, and the size of the cohort. The impact of contour differences on NTCP model performance increased with steeper NTCP curves. In our dataset, parameters on the low range of the dose-volume histogram were more robust to contour differences. Conclusions: Our methodology can be used to estimate whether a given contouring model is fit for NTCP model development. For the heart in comparable datasets, average Dice should be at least as high as between our manual and deep learning contours for shallow NTCP relationships (88.5 ± 4.5 %) and higher for steep relationships.

Multicenter comparison of measures for quantitative evaluation of contouring in radiotherapy.

Background and Purpose: A wide range of quantitative measures are available to facilitate clinical implementation of auto-contouring software, on-going Quality Assurance (QA) and interobserver contouring variation studies. This study aimed to assess the variation in output when applying different implementations of the measures to the same data in order to investigate how consistently such measures are defined and implemented in radiation oncology. Materials and Methods: A survey was conducted to assess if there were any differences in definitions of contouring measures or their implementations that would lead to variation in reported results between institutions. This took two forms: a set of computed tomography (CT) image data with "Test" and "Reference" contours was distributed for participants to process using their preferred tools and report results, and a questionnaire regarding the definition of measures and their implementation was completed by the participants. Results: Thirteen participants completed the survey and submitted results, with one commercial and twelve in-house solutions represented. Excluding outliers, variations of up to 50% in Dice Similarity Coefficient (DSC), 50% in 3D Hausdorff Distance (HD), and 200% in Average Distance (AD) were observed between the participant submitted results. Collaborative investigation with participants revealed a large number of bugs in implementation, confounding the understanding of intentional implementation choices. Conclusion: Care must be taken when comparing quantitative results between different studies. There is a need for a dataset with clearly defined measures and ground truth for validation of such tools prior to their use.

Automated detection and segmentation of non-small cell lung cancer computed tomography images.

Detection and segmentation of abnormalities on medical images is highly important for patient management including diagnosis, radiotherapy, response evaluation, as well as for quantitative image research. We present a fully automated pipeline for the detection and volumetric segmentation of non-small cell lung cancer (NSCLC) developed and validated on 1328 thoracic CT scans from 8 institutions. Along with quantitative performance detailed by image slice thickness, tumor size, image interpretation difficulty, and tumor location, we report an in-silico prospective clinical trial, where we show that the proposed method is faster and more reproducible compared to the experts. Moreover, we demonstrate that on average, radiologists & radiation oncologists preferred automatic segmentations in 56% of the cases. Additionally, we evaluate the prognostic power of the automatic contours by applying RECIST criteria and measuring the tumor volumes. Segmentations by our method stratified patients into low and high survival groups with higher significance compared to those methods based on manual contours.

Segmentation Uncertainty Estimation as a Sanity Check for Image Biomarker Studies.

Problem. Image biomarker analysis, also known as radiomics, is a tool for tissue characterization and treatment prognosis that relies on routinely acquired clinical images and delineations. Due to the uncertainty in image acquisition, processing, and segmentation (delineation) protocols, radiomics often lack reproducibility. Radiomics harmonization techniques have been proposed as a solution to reduce these sources of uncertainty and/or their influence on the prognostic model performance. A relevant question is how to estimate the protocol-induced uncertainty of a specific image biomarker, what the effect is on the model performance, and how to optimize the model given the uncertainty. Methods. Two non-small cell lung cancer (NSCLC) cohorts, composed of 421 and 240 patients, respectively, were used for training and testing. Per patient, a Monte Carlo algorithm was used to generate three hundred synthetic contours with a surface dice tolerance measure of less than 1.18 mm with respect to the original GTV. These contours were subsequently used to derive 104 radiomic features, which were ranked on their relative sensitivity to contour perturbation, expressed in the parameter η. The top four (low η) and the bottom four (high η) features were selected for two models based on the Cox proportional hazards model. To investigate the influence of segmentation uncertainty on the prognostic model, we trained and tested the setup in 5000 augmented realizations (using a Monte Carlo sampling method); the log-rank test was used to assess the stratification performance and stability of segmentation uncertainty. Results. Although both low and high η setup showed significant testing set log-rank p-values (p = 0.01) in the original GTV delineations (without segmentation uncertainty introduced), in the model with high uncertainty, to effect ratio, only around 30% of the augmented realizations resulted in model performance with p < 0.05 in the test set. In contrast, the low η setup performed with a log-rank p < 0.05 in 90% of the augmented realizations. Moreover, the high η setup classification was uncertain in its predictions for 50% of the subjects in the testing set (for 80% agreement rate), whereas the low η setup was uncertain only in 10% of the cases. Discussion. Estimating image biomarker model performance based only on the original GTV segmentation, without considering segmentation, uncertainty may be deceiving. The model might result in a significant stratification performance, but can be unstable for delineation variations, which are inherent to manual segmentation. Simulating segmentation uncertainty using the method described allows for more stable image biomarker estimation, selection, and model development. The segmentation uncertainty estimation method described here is universal and can be extended to estimate other protocol uncertainties (such as image acquisition and pre-processing).

Evaluation of two independent dose prediction methods to personalize the automated radiotherapy planning process for prostate cancer.

BACKGROUND AND PURPOSE: Currently, automatic approaches for radiotherapy planning are widely used, however creation of high quality treatment plans is still challenging. In this study, two independent dose prediction methods were used to personalize the initial settings for the automated planning template for optimizing prostate cancer treatment plans. This study evaluated the dose metrics of these plans comparing both methods with the current clinical automated prostate cancer treatment plans. MATERIAL AND METHODS: Datasets of 20 high-risk prostate cancer treatment plans were taken from our clinical database. The prescription dose for these plans was 70 Gy given in fractions of 2.5 Gy. Plans were replanned using the current clinical automated treatment and compared with two personalized automated planning methods. The feasibility dose volume histogram (FDVH) and modified filter back projection (mFBP) methods were used to calculate independent dose predictions. Parameters for the initial objective values of the planning template were extracted from these predictions and used to personalize the optimization of the automated planning process. RESULTS: The current automated replanned clinical plans and the automated plans optimized with the personalized template methods fulfilled the clinical dose criteria. For both methods a reduction in the average mean dose of the rectal wall was found, from 22.5 to 20.1 Gy for the FDVH and from 22.5 to 19.6 Gy for the mFBP method. CONCLUSIONS: With both dose-prediction methods the initial settings of the template could be personalized. Hereby, the average dose to the rectal wall was reduced compared to the standard template method.

A systematic review and quality of reporting checklist for repeatability and reproducibility of radiomic features.

PURPOSE: Although quantitative image biomarkers (radiomics) show promising value for cancer diagnosis, prognosis, and treatment assessment, these biomarkers still lack reproducibility. In this systematic review, we aimed to assess the progress in radiomics reproducibility and repeatability in the recent years. METHODS AND MATERIALS: Four hundred fifty-one abstracts were retrieved according to the original PubMed search pattern with the publication dates ranging from 2017/05/01 to 2020/12/01. Each abstract including the keywords was independently screened by four observers. Forty-two full-text articles were selected for further analysis. Patient population data, radiomic feature classes, feature extraction software, image preprocessing, and reproducibility results were extracted from each article. To support the community with a standardized reporting strategy, we propose a specific reporting checklist to evaluate the feasibility to reproduce each study. RESULTS: Many studies continue to under-report essential reproducibility information: all but one clinical and all but two phantom studies missed to report at least one important item reporting image acquisition. The studies included in this review indicate that all radiomic features are sensitive to image acquisition, reconstruction, tumor segmentation, and interpolation. However, the amount of sensitivity is feature dependent, for instance, textural features were, in general, less robust than statistical features. CONCLUSIONS: Radiomics repeatability, reproducibility, and reporting quality can substantially be improved regarding feature extraction software and settings, image preprocessing and acquisition, cutoff values for stable feature selection. Our proposed radiomics reporting checklist can serve to simplify and improve the reporting and, eventually, guarantee the possibility to fully replicate and validate radiomic studies.

Radiomics integration into a picture archiving and communication system.

Radiomics is referred to as quantitative imaging of biomarkers used for clinical outcome prognosis or tumor characterization. In order to bridge radiomics and its clinical application, we aimed to build an integrated solution of radiomics extraction with an open-source Picture Archiving and Communication System (PACS). The integrated SQLite4Radiomics software was tested in three different imaging modalities and its performance was benchmarked in lung cancer open datasets RIDER and MMD with median extraction time of 10.7 (percentiles 25-75: 8.9-18.7) seconds per ROI in three different configurations.

Deep learning model for automatic contouring of cardiovascular substructures on radiotherapy planning CT images: Dosimetric validation and reader study based clinical acceptability testing.

BACKGROUND AND PURPOSE: Large radiotherapy (RT) planning imaging datasets with consistently contoured cardiovascular structures are essential for robust cardiac radiotoxicity research in thoracic cancers. This study aims to develop and validate a highly accurate automatic contouring model for the heart, cardiac chambers, and great vessels for RT planning computed tomography (CT) images that can be used for dose-volume parameter estimation. MATERIALS AND METHODS: A neural network model was trained using a dataset of 127 expertly contoured planning CT images from RT treatment of locally advanced non-small-cell lung cancer (NSCLC) patients. Evaluation of geometric accuracy and quality of dosimetric parameter estimation was performed on 50 independent scans with contrast and without contrast enhancement. The model was further evaluated regarding the clinical acceptability of the contours in 99 scans randomly sampled from the RTOG-0617 dataset by three experienced radiation oncologists. RESULTS: Median surface dice at 3 mm tolerance for all dedicated thoracic structures was 90% in the test set. Median absolute difference between mean dose computed with model contours and expert contours was 0.45 Gy averaged over all structures. The mean clinical acceptability rate by majority vote in the RTOG-0617 scans was 91%. CONCLUSION: This model can be used to contour the heart, cardiac chambers, and great vessels in large datasets of RT planning thoracic CT images accurately, quickly, and consistently. Additionally, the model can be used as a time-saving tool for contouring in clinic practice.

Deciphering the glioblastoma phenotype by computed tomography radiomics.

INTRODUCTION: Glioblastoma (GBM) is the most common malignant primary brain tumour which has, despite extensive treatment, a median overall survival of 15 months. Radiomics is the high-throughput extraction of large amounts of image features from radiographic images, which allows capturing the tumour phenotype in 3D and in a non-invasive way. In this study we assess the prognostic value of CT radiomics for overall survival in patients with a GBM. MATERIALS AND METHODS: Clinical data and pre-treatment CT images were obtained from 218 patients diagnosed with a GBM via biopsy who underwent radiotherapy +/- temozolomide between 2004 and 2015 treated at three independent institutes (n = 93, 62 and 63). A clinical prognostic score (CPS), a simple radiomics model consisting of volume based score (VPS), a complex radiomics prognostic score (RPS) and a combined clinical and radiomics (C + R)PS model were developed. The population was divided into three risk groups for each prognostic score and respective Kaplan-Meier curves were generated. RESULTS: Patient characteristics were broadly comparable. Clinically significant differences were observed with regards to radiation dose, tumour volume and performance status between datasets. Image acquisition parameters differed between institutes. The cross-validated c-indices were moderately discriminative and for the CPS ranged from 0.63 to 0.65; the VPS c-indices ranged between 0.52 and 0.61; the RPS c-indices ranged from 0.57 to 0.64 and the combined clinical and radiomics model resulted in c-indices of 0.59-0.71. CONCLUSION: In this study clinical and CT radiomics features were used to predict OS in GBM. Discrimination between low-, middle- and high-risk patients based on the combined clinical and radiomics model was comparable to previous MRI-based models.

FAIR-compliant clinical, radiomics and DICOM metadata of RIDER, interobserver, Lung1 and head-Neck1 TCIA collections.

PURPOSE: One of the most frequently cited radiomics investigations showed that features automatically extracted from routine clinical images could be used in prognostic modeling. These images have been made publicly accessible via The Cancer Imaging Archive (TCIA). There have been numerous requests for additional explanatory metadata on the following datasets - RIDER, Interobserver, Lung1, and Head-Neck1. To support repeatability, reproducibility, generalizability, and transparency in radiomics research, we publish the subjects' clinical data, extracted radiomics features, and digital imaging and communications in medicine (DICOM) headers of these four datasets with descriptive metadata, in order to be more compliant with findable, accessible, interoperable, and reusable (FAIR) data management principles. ACQUISITION AND VALIDATION METHODS: Overall survival time intervals were updated using a national citizens registry after internal ethics board approval. Spatial offsets of the primary gross tumor volume (GTV) regions of interest (ROIs) associated with the Lung1 CT series were improved on the TCIA. GTV radiomics features were extracted using the open-source Ontology-Guided Radiomics Analysis Workflow (O-RAW). We reshaped the output of O-RAW to map features and extraction settings to the latest version of Radiomics Ontology, so as to be consistent with the Image Biomarker Standardization Initiative (IBSI). Digital imaging and communications in medicine metadata was extracted using a research version of Semantic DICOM (SOHARD, GmbH, Fuerth; Germany). Subjects' clinical data were described with metadata using the Radiation Oncology Ontology. All of the above were published in Resource Descriptor Format (RDF), that is, triples. Example SPARQL queries are shared with the reader to use on the online triples archive, which are intended to illustrate how to exploit this data submission. DATA FORMAT: The accumulated RDF data are publicly accessible through a SPARQL endpoint where the triples are archived. The endpoint is remotely queried through a graph database web application at http://sparql.cancerdata.org. SPARQL queries are intrinsically federated, such that we can efficiently cross-reference clinical, DICOM, and radiomics data within a single query, while being agnostic to the original data format and coding system. The federated queries work in the same way even if the RDF data were partitioned across multiple servers and dispersed physical locations. POTENTIAL APPLICATIONS: The public availability of these data resources is intended to support radiomics features replication, repeatability, and reproducibility studies by the academic community. The example SPARQL queries may be freely used and modified by readers depending on their research question. Data interoperability and reusability are supported by referencing existing public ontologies. The RDF data are readily findable and accessible through the aforementioned link. Scripts used to create the RDF are made available at a code repository linked to this submission: https://gitlab.com/UM-CDS/FAIR-compliant_clinical_radiomics_and_DICOM_metadata.

Distributed learning on 20 000+ lung cancer patients - The Personal Health Train.

BACKGROUND AND PURPOSE: Access to healthcare data is indispensable for scientific progress and innovation. Sharing healthcare data is time-consuming and notoriously difficult due to privacy and regulatory concerns. The Personal Health Train (PHT) provides a privacy-by-design infrastructure connecting FAIR (Findable, Accessible, Interoperable, Reusable) data sources and allows distributed data analysis and machine learning. Patient data never leaves a healthcare institute. MATERIALS AND METHODS: Lung cancer patient-specific databases (tumor staging and post-treatment survival information) of oncology departments were translated according to a FAIR data model and stored locally in a graph database. Software was installed locally to enable deployment of distributed machine learning algorithms via a central server. Algorithms (MATLAB, code and documentation publicly available) are patient privacy-preserving as only summary statistics and regression coefficients are exchanged with the central server. A logistic regression model to predict post-treatment two-year survival was trained and evaluated by receiver operating characteristic curves (ROC), root mean square prediction error (RMSE) and calibration plots. RESULTS: In 4 months, we connected databases with 23 203 patient cases across 8 healthcare institutes in 5 countries (Amsterdam, Cardiff, Maastricht, Manchester, Nijmegen, Rome, Rotterdam, Shanghai) using the PHT. Summary statistics were computed across databases. A distributed logistic regression model predicting post-treatment two-year survival was trained on 14 810 patients treated between 1978 and 2011 and validated on 8 393 patients treated between 2012 and 2015. CONCLUSION: The PHT infrastructure demonstrably overcomes patient privacy barriers to healthcare data sharing and enables fast data analyses across multiple institutes from different countries with different regulatory regimens. This infrastructure promotes global evidence-based medicine while prioritizing patient privacy.

External validation of deep learning-based contouring of head and neck organs at risk.

BACKGROUND AND PURPOSE: Head and neck (HN) radiotherapy can benefit from automatic delineation of tumor and surrounding organs because of the complex anatomy and the regular need for adaptation. The aim of this study was to assess the performance of a commercially available deep learning contouring (DLC) model on an external validation set. MATERIALS AND METHODS: The CT-based DLC model, trained at the University Medical Center Groningen (UMCG), was applied to an independent set of 58 patients from the Radboud University Medical Center (RUMC). DLC results were compared to the RUMC manual reference using the Dice similarity coefficient (DSC) and 95th percentile of Hausdorff distance (HD95). Craniocaudal spatial information was added by calculating binned measures. In addition, a qualitative evaluation compared the acceptance of manual and DLC contours in both groups of observers. RESULTS: Good correspondence was shown for the mandible (DSC 0.90; HD95 3.6 mm). Performance was reasonable for the glandular OARs, brainstem and oral cavity (DSC 0.78-0.85, HD95 3.7-7.3 mm). The other aerodigestive tract OARs showed only moderate agreement (DSC 0.53-0.65, HD95 around 9 mm). The binned measures displayed the largest deviations caudally and/or cranially. CONCLUSIONS: This study demonstrates that the DLC model can provide a reasonable starting point for delineation when applied to an independent patient cohort. The qualitative evaluation did not reveal large differences in the interpretation of contouring guidelines between RUMC and UMCG observers.

Distributed radiomics as a signature validation study using the Personal Health Train infrastructure.

Prediction modelling with radiomics is a rapidly developing research topic that requires access to vast amounts of imaging data. Methods that work on decentralized data are urgently needed, because of concerns about patient privacy. Previously published computed tomography medical image sets with gross tumour volume (GTV) outlines for non-small cell lung cancer have been updated with extended follow-up. In a previous study, these were referred to as Lung1 (n = 421) and Lung2 (n = 221). The Lung1 dataset is made publicly accessible via The Cancer Imaging Archive (TCIA; https://www.cancerimagingarchive.net ). We performed a decentralized multi-centre study to develop a radiomic signature (hereafter "ZS2019") in one institution and validated the performance in an independent institution, without the need for data exchange and compared this to an analysis where all data was centralized. The performance of ZS2019 for 2-year overall survival validated in distributed radiomics was not statistically different from the centralized validation (AUC 0.61 vs 0.61; p = 0.52). Although slightly different in terms of data and methods, no statistically significant difference in performance was observed between the new signature and previous work (c-index 0.58 vs 0.65; p = 0.37). Our objective was not the development of a new signature with the best performance, but to suggest an approach for distributed radiomics. Therefore, we used a similar method as an earlier study. We foresee that the Lung1 dataset can be further re-used for testing radiomic models and investigating feature reproducibility.

Learning from scanners: Bias reduction and feature correction in radiomics.

PURPOSE: Radiomics are quantitative features extracted from medical images. Many radiomic features depend not only on tumor properties, but also on non-tumor related factors such as scanner signal-to-noise ratio (SNR), reconstruction kernel and other image acquisition settings. This causes undesirable value variations in the features and reduces the performance of prediction models. In this paper, we investigate whether we can use phantom measurements to characterize and correct for the scanner SNR dependence. METHODS: We used a phantom with 17 regions of interest (ROI) to investigate the influence of different SNR values. CT scans were acquired with 9 different exposure settings. We developed an additive correction model to reduce scanner SNR influence. RESULTS: Sixty-two of 92 radiomic features showed high variance due to the scanner SNR. Of these 62 features, 47 showed at least a factor 2 significant standard deviation reduction by using the additive correction model. We assessed the clinical relevance of radiomics instability by using a 221 NSCLC patient cohort measured with the same scanner. CONCLUSIONS: Phantom measurements show that roughly two third of the radiomic features depend on the exposure setting of the scanner. The dependence can be modeled and corrected significantly reducing the variation in feature values with at least a factor of 2. More complex models will likely increase the correctability. Scanner SNR correction will result in more reliable radiomics predictions in NSCLC.

Longitudinal radiomics of cone-beam CT images from non-small cell lung cancer patients: Evaluation of the added prognostic value for overall survival and locoregional recurrence.

BACKGROUND AND PURPOSE: The prognostic value of radiomics for non-small cell lung cancer (NSCLC) patients has been investigated for images acquired prior to treatment, but no prognostic model has been developed that includes the change of radiomic features during treatment. Therefore, the aim of this study was to investigate the potential added prognostic value of a longitudinal radiomics approach using cone-beam computed tomography (CBCT) for NSCLC patients. MATERIALS AND METHODS: This retrospective study includes a training dataset of 141 stage I-IV NSCLC patients and three external validation datasets of 94, 61 and 41 patients, all treated with curative intended (chemo)radiotherapy. The change of radiomic features extracted from CBCT images was summarized as the slope of a linear regression. The CBCT slope-features and CT-extracted features were used as input for a Cox proportional hazards model. Moreover, prognostic performance of clinical parameters was investigated for overall survival and locoregional recurrence. Model performances were assessed using the Kaplan-Meier curves and c-index. RESULTS: The radiomics model contained only CT-derived features and reached a c-index of 0.63 for overall survival and could be validated on the first validation dataset. No model for locoregional recurrence could be developed that validated on the validation datasets. The clinical parameters model could not be validated for either overall survival or locoregional recurrence. CONCLUSION: In this study we could not confirm our hypothesis that longitudinal CBCT-extracted radiomic features contribute to improved prognostic information. Moreover, performance of baseline radiomic features or clinical parameters was poor, probably affected by heterogeneity within and between datasets.

Multicenter CT phantoms public dataset for radiomics reproducibility tests.

PURPOSE: The aim of this paper is to describe a public, open-access, computed tomography (CT) phantom image set acquired at three centers and collected especially for radiomics reproducibility research. The dataset is useful to test radiomic features reproducibility with respect to various parameters, such as acquisition settings, scanners, and reconstruction algorithms. ACQUISITION AND VALIDATION METHODS: Three phantoms were scanned in three independent institutions. Images of the following phantoms were acquired: Catphan 700 and COPDGene Phantom II (Phantom Laboratory, Greenwich, NY, USA), and the Triple modality 3D Abdominal Phantom (CIRS, Norfolk, VA, USA). Data were collected at three Dutch medical centers: MAASTRO Clinic (Maastricht, NL), Radboud University Medical Center (Nijmegen, NL), and University Medical Center Groningen (Groningen, NL) with scanners from two different manufacturers Siemens Healthcare and Philips Healthcare. The following acquisition parameter were varied in the phantom scans: slice thickness, reconstruction kernels, and tube current. DATA FORMAT AND USAGE NOTES: We made the dataset publically available on the Dutch instance of "Extensible Neuroimaging Archive Toolkit-XNAT" (https://xnat.bmia.nl). The dataset is freely available and reusable with attribution (Creative Commons 3.0 license). POTENTIAL APPLICATIONS: Our goal was to provide a findable, open-access, annotated, and reusable CT phantom dataset for radiomics reproducibility studies. Reproducibility testing and harmonization are fundamental requirements for wide generalizability of radiomics-based clinical prediction models. It is highly desirable to include only reproducible features into models, to be more assured of external validity across hitherto unseen contexts. In this view, phantom data from different centers represent a valuable source of information to exclude CT radiomic features that may already be unstable with respect to simplified structures and tightly controlled scan settings. The intended extension of our shared dataset is to include other modalities and phantoms with more realistic lesion simulations.

Independent knowledge-based treatment planning QA to audit Pinnacle autoplanning.

BACKGROUND AND PURPOSE: With the advent of automatic treatment planning options like Pinnacle's Autoplanning (PAP), the challenge arises how to assess the quality of a plan that no dosimetrist did work on. The aim of this study was to assess plan quality consistency of PAP prostate cancer patients in clinical practice. MATERIALS AND METHODS: 100 prostate cancer patients were included from NKI and 129 from RadboudUMC (RUMC). Per institute a previously developed [1] treatment planning QA model, based on overlap volume histograms, was trained on PAP plans to predict achievable dose metrics which were then compared to the clinical PAP plans. A threshold of 3 Gy (DVH dose parameters)/3% (DVH volume parameters) was used to detect outliers. For the outlier plans, the PAP technique was adjusted with the aim of meeting the threshold. RESULTS: The average difference between the prediction and the clinically achieved value was <0.5 Gy (mean dose parameters) and <1.2% (volume parameters), with standard deviation of 1.9 Gy/1.5% respectively. We found 8% (NKI)/25% (RUMC) of patients to exceed the 3 Gy/3% threshold, with deviations up to 6.7 Gy (mean dose rectum) and 6% (rectal wall V64Gy). In all cases the plans could be improved to fall within the thresholds, without compromising the other dose metrics. CONCLUSION: Independent treatment planning QA was used successfully to assess the quality of clinical PAP in a multi-institutional setting. Respectively 8% and 25% suboptimal clinical PAP plans were detected that all could be improved with replanning. Therefore we recommend the use of independent treatment plan QA in combination with PAP for prostate cancer patients.

External validation of an NTCP model for acute esophageal toxicity in locally advanced NSCLC patients treated with intensity-modulated (chemo-)radiotherapy.

BACKGROUND AND PURPOSE: We externally validated a previously established multivariable normal-tissue complication probability (NTCP) model for Grade ≥2 acute esophageal toxicity (AET) after intensity-modulated (chemo-)radiotherapy or volumetric-modulated arc therapy for locally advanced non-small cell lung cancer. MATERIALS AND METHODS: A total of 603 patients from five cohorts (A-E) within four different Dutch institutes were included. Using the NTCP model, containing predictors concurrent chemoradiotherapy, mean esophageal dose, gender and clinical tumor stage, the risk of Grade ≥2 AET was estimated per patient and model discrimination and (re)calibration performance were evaluated. RESULTS: Four validation cohorts (A, B, D, E) experienced higher incidence of Grade ≥2 AET compared to the training cohort (49.3-70.2% vs 35.6%; borderline significant for one cohort, highly significant for three cohorts). Cohort C experienced lower Grade ≥2 AET incidence (21.7%, p < 0.001). For three cohorts (A-C), discriminative performance was similar to the training cohort (area under the curve (AUC) 0.81-0.89 vs 0.84). In the two remaining cohorts (D-E) the model showed poor discriminative power (AUC 0.64 and 0.63). Reasonable calibration performance was observed in two cohorts (A-B), and recalibration further improved performance in all three cohorts with good discrimination (A-C). Recalibration for the two poorly discriminating cohorts (D-E) did not improve performance. CONCLUSIONS: The NTCP model for AET prediction was successfully validated in three out of five patient cohorts (AUC ≥0.80). The model did not perform well in two cohorts, which included patients receiving substantially different treatment. Before applying the model in clinical practice, validation of discrimination and (re)calibration performance in a local cohort is recommended.