Associations Between Radiation Oncologist Demographic Factors and Segmentation Similarity Benchmarks: Insights From a Crowd-Sourced Challenge Using Bayesian Estimation.

PURPOSE: The quality of radiotherapy auto-segmentation training data, primarily derived from clinician observers, is of utmost importance. However, the factors influencing the quality of clinician-derived segmentations are poorly understood; our study aims to quantify these factors. METHODS: Organ at risk (OAR) and tumor-related segmentations provided by radiation oncologists from the Contouring Collaborative for Consensus in Radiation Oncology data set were used. Segmentations were derived from five disease sites: breast, sarcoma, head and neck (H&N), gynecologic (GYN), and GI. Segmentation quality was determined on a structure-by-structure basis by comparing the observer segmentations with an expert-derived consensus, which served as a reference standard benchmark. The Dice similarity coefficient (DSC) was primarily used as a metric for the comparisons. DSC was stratified into binary groups on the basis of structure-specific expert-derived interobserver variability (IOV) cutoffs. Generalized linear mixed-effects models using Bayesian estimation were used to investigate the association between demographic variables and the binarized DSC for each disease site. Variables with a highest density interval excluding zero were considered to substantially affect the outcome measure. RESULTS: Five hundred seventy-four, 110, 452, 112, and 48 segmentations were used for the breast, sarcoma, H&N, GYN, and GI cases, respectively. The median percentage of segmentations that crossed the expert DSC IOV cutoff when stratified by structure type was 55% and 31% for OARs and tumors, respectively. Regression analysis revealed that the structure being tumor-related had a substantial negative impact on binarized DSC for the breast, sarcoma, H&N, and GI cases. There were no recurring relationships between segmentation quality and demographic variables across the cases, with most variables demonstrating large standard deviations. CONCLUSION: Our study highlights substantial uncertainty surrounding conventionally presumed factors influencing segmentation quality relative to benchmarks.

Determining The Role Of Radiation Oncologist Demographic Factors On Segmentation Quality: Insights From A Crowd-Sourced Challenge Using Bayesian Estimation.

BACKGROUND: Medical image auto-segmentation is poised to revolutionize radiotherapy workflows. The quality of auto-segmentation training data, primarily derived from clinician observers, is of utmost importance. However, the factors influencing the quality of these clinician-derived segmentations have yet to be fully understood or quantified. Therefore, the purpose of this study was to determine the role of common observer demographic variables on quantitative segmentation performance. METHODS: Organ at risk (OAR) and tumor volume segmentations provided by radiation oncologist observers from the Contouring Collaborative for Consensus in Radiation Oncology public dataset were utilized for this study. Segmentations were derived from five separate disease sites comprised of one patient case each: breast, sarcoma, head and neck (H&N), gynecologic (GYN), and gastrointestinal (GI). Segmentation quality was determined on a structure-by-structure basis by comparing the observer segmentations with an expert-derived consensus gold standard primarily using the Dice Similarity Coefficient (DSC); surface DSC was investigated as a secondary metric. Metrics were stratified into binary groups based on previously established structure-specific expert-derived interobserver variability (IOV) cutoffs. Generalized linear mixed-effects models using Markov chain Monte Carlo Bayesian estimation were used to investigate the association between demographic variables and the binarized segmentation quality for each disease site separately. Variables with a highest density interval excluding zero - loosely analogous to frequentist significance - were considered to substantially impact the outcome measure. RESULTS: After filtering by practicing radiation oncologists, 574, 110, 452, 112, and 48 structure observations remained for the breast, sarcoma, H&N, GYN, and GI cases, respectively. The median percentage of observations that crossed the expert DSC IOV cutoff when stratified by structure type was 55% and 31% for OARs and tumor volumes, respectively. Bayesian regression analysis revealed tumor category had a substantial negative impact on binarized DSC for the breast (coefficient mean ± standard deviation: -0.97 ± 0.20), sarcoma (-1.04 ± 0.54), H&N (-1.00 ± 0.24), and GI (-2.95 ± 0.98) cases. There were no clear recurring relationships between segmentation quality and demographic variables across the cases, with most variables demonstrating large standard deviations and wide highest density intervals. CONCLUSION: Our study highlights substantial uncertainty surrounding conventionally presumed factors influencing segmentation quality. Future studies should investigate additional demographic variables, more patients and imaging modalities, and alternative metrics of segmentation acceptability.

Large scale crowdsourced radiotherapy segmentations across a variety of cancer anatomic sites.

Clinician generated segmentation of tumor and healthy tissue regions of interest (ROIs) on medical images is crucial for radiotherapy. However, interobserver segmentation variability has long been considered a significant detriment to the implementation of high-quality and consistent radiotherapy dose delivery. This has prompted the increasing development of automated segmentation approaches. However, extant segmentation datasets typically only provide segmentations generated by a limited number of annotators with varying, and often unspecified, levels of expertise. In this data descriptor, numerous clinician annotators manually generated segmentations for ROIs on computed tomography images across a variety of cancer sites (breast, sarcoma, head and neck, gynecologic, gastrointestinal; one patient per cancer site) for the Contouring Collaborative for Consensus in Radiation Oncology challenge. In total, over 200 annotators (experts and non-experts) contributed using a standardized annotation platform (ProKnow). Subsequently, we converted Digital Imaging and Communications in Medicine data into Neuroimaging Informatics Technology Initiative format with standardized nomenclature for ease of use. In addition, we generated consensus segmentations for experts and non-experts using the Simultaneous Truth and Performance Level Estimation method. These standardized, structured, and easily accessible data are a valuable resource for systematically studying variability in segmentation applications.

E pluribus unum: prospective acceptability benchmarking from the Contouring Collaborative for Consensus in Radiation Oncology crowdsourced initiative for multiobserver segmentation.

Purpose: Contouring Collaborative for Consensus in Radiation Oncology (C3RO) is a crowdsourced challenge engaging radiation oncologists across various expertise levels in segmentation. An obstacle to artificial intelligence (AI) development is the paucity of multiexpert datasets; consequently, we sought to characterize whether aggregate segmentations generated from multiple nonexperts could meet or exceed recognized expert agreement. Approach: Participants who contoured ≥ 1 region of interest (ROI) for the breast, sarcoma, head and neck (H&N), gynecologic (GYN), or gastrointestinal (GI) cases were identified as a nonexpert or recognized expert. Cohort-specific ROIs were combined into single simultaneous truth and performance level estimation (STAPLE) consensus segmentations. STAPLE nonexpert ROIs were evaluated against STAPLE expert contours using Dice similarity coefficient (DSC). The expert interobserver DSC ( IODSC expert ) was calculated as an acceptability threshold between STAPLE nonexpert and STAPLE expert . To determine the number of nonexperts required to match the IODSC expert for each ROI, a single consensus contour was generated using variable numbers of nonexperts and then compared to the IODSC expert . Results: For all cases, the DSC values for STAPLE nonexpert versus STAPLE expert were higher than comparator expert IODSC expert for most ROIs. The minimum number of nonexpert segmentations needed for a consensus ROI to achieve IODSC expert acceptability criteria ranged between 2 and 4 for breast, 3 and 5 for sarcoma, 3 and 5 for H&N, 3 and 5 for GYN, and 3 for GI. Conclusions: Multiple nonexpert-generated consensus ROIs met or exceeded expert-derived acceptability thresholds. Five nonexperts could potentially generate consensus segmentations for most ROIs with performance approximating experts, suggesting nonexpert segmentations as feasible cost-effective AI inputs.

The challenge of planning vertebral body SBRT: Optimizing target volume coverage.

Stereotactic ablative body radiotherapy for vertebral metastases has been shown to be safe and effective to achieve tumor and pain control. To raise awareness of and build familiarity with vertebral stereotactic ablative body radiation therapy (SBRT) for a multicenter clinical trial including SBRT to vertebral metastases, Trans Tasman Radiation Oncology Cancer Research performed an international planning challenge. A single vertebral case was selected and the computed tomography image and contours were made available. Participants performed a treatment plan according to the NIVORAD clinical trial protocol and uploaded the treatment plan and dose grid Digital Imaging and Communications in Medicine (DICOM) files. A progressive scoring matrix was applied which gave each plan a score based on target and organ at risk dosimetry. The plans were compared based on achieved score and treatment technique details. A total of 149 plans were submitted from 26 countries; the treatment geometry for four plans was deemed to result in collision with the couch and these were removed from analysis. Only one plan exceeded spinal cord constraints; all other plans met protocol constraints. The largest variation in plan quality was observed with the target coverage; the highest scoring plans were able to achieve higher target coverage whilst respecting adjacent organ at risk (OAR) constraints. Consequently, plan score was correlated with the dose gradient at the target-cord interface. We have conducted a large multicenter, international vertebral SBRT planning challenge. The results showed consistent ability to meet protocol constraints, however a large variation in the ability to cover the target volume was observed.

Assessing the Variability and Quality of Lung Stereotactic Radiation Therapy Treatment Plans Using a Web-Based Crowdsourcing Platform.

PURPOSE: The quality of stereotactic body radiation therapy (SBRT) treatment plans for early stage lung cancer are unknown outside of peer-reviewed publications. Thus, a study was conducted to crowdsource and analyze a variety of lung SBRT treatment plans from around the world. METHODS AND MATERIALS: This study had 2 parts, planning and contouring, and each was facilitated by a web-based technology platform. For planning, lung SBRT planners were invited to design, score, and submit their treatment plans (prescription of 11 Gy × 5) for a centralized stage I lung cancer case using standardized images and predefined contours. Each plan was scored with 20 weighted metrics adapted from currently recruiting phase 3 lung SBRT trials. For contouring, a separate image set was used to evaluate organ-at-risk contour accuracy using Dice coefficients and a StructSure score. RESULTS: For planning 227 plans were submitted in total with 7 different treatment planning systems and 7 different delivery methods represented. Variability was primarily user driven and not associated with the treatment planning system, delivery modality, total monitor units, or estimated beam-on time. Many of the highest-quality plans required the shortest amount of time to deliver, independent of the delivery technique. For contouring, organ-at-risk contours were frequently over- or undercontoured and often included only the luminal air of the trachea, proximal bronchial tree, and esophagus, even when the mucosal linings were within a few centimeters of the target tumor. CONCLUSIONS: These findings demonstrate the importance of quality assurance to help improve planning and contouring and the value of peer review and comparison. More readily accessible quality evaluation software solutions, such as the one used herein, may help meet this growing need.

A method for a priori estimation of best feasible DVH for organs-at-risk: Validation for head and neck VMAT planning.

PURPOSE: Despite improvements in optimization and automation algorithms, the quality of radiation treatment plans still varies dramatically. A tool that allows a priori estimation of the best possible sparing (Feasibility DVH, or FDVH) of an organ at risk (OAR) in high-energy photon planning may help reduce plan quality variability by deriving patient-specific OAR goals prior to optimization. Such a tool may be useful for (a) meaningfully evaluating patient-specific plan quality and (b) supplying best theoretically achievable DVH goals, thus pushing the solution toward automatic Pareto optimality. This work introduces such a tool and validates it for clinical Head and Neck (HN) datasets. METHODS: To compute FDVH, first the targets are assigned uniform prescription doses, with no reference to any particular beam arrangement. A benchmark 3D dose built outside the targets is estimated using a series of energy-specific dose spread calculations reflecting observed properties of radiation distribution in media. For the patient, the calculation is performed on the heterogeneous dataset, taking into account the high- (penumbra driven) and low- (PDD and scatter-driven) gradient dose spreading. The former is driven mostly by target dose and surface shape, while the latter adds the dependence on target volume. This benchmark dose is used to produce the "best possible sparing" FDVH for an OAR, and based on it, progressively more easily achievable FDVH curves can be estimated. Validation was performed using test cylindrical geometries as well as 10 clinical HN datasets. For HN, VMAT plans were prepared with objectives of covering the primary and the secondary (bilateral elective neck) PTVs while addressing only one OAR at a time, with the goal of maximum sparing. The OARs were each parotid, the larynx, and the inferior pharyngeal constrictor. The difference in mean OAR doses was computed for the achieved vs. FDVHs, and the shapes of those DVHs were compared by means of the Dice similarity coefficient (DSC). RESULTS: For all individually optimized HN OARs (N = 38), the average DSC between the planned DVHs and the FDVHs was 0.961 ± 0.018 (95% CI 0.955-0.967), with the corresponding average of mean OAR dose differences of 1.8 ± 5.8% (CI -0.1-3.6%). For realistic plans the achieved DVHs run no lower than the FDVHs, except when target coverage is compromised at the target/OAR interface. CONCLUSIONS: For the validation of VMAT plans, the OAR DVHs optimized one-at-a-time were similar in shape to and bound on the low side by the FDVHs, within the confines of planner's ability to precisely cover the target(s) with the prescription dose(s). The method is best suited for the OARs close to the target. This approach is fundamentally different from "knowledge-based planning" because it is (a) independent of the treatment plan and prior experience, and (b) it approximates, from nearly first principles, the lowest possible boundary of the OAR DVH, but not necessarily its actual shape in the presence of competing OAR sparing and target dose homogeneity objectives.

Validation of an improved helical diode array and dose reconstruction software using TG-244 datasets and stringent dose comparison criteria.

The original helical ArcCHECK (AC) diode array and associated software for 3D measurement-guided dose reconstruction were characterized and validated; however, recent design changes to the AC required that the subject be revisited. The most important AC change starting in 2014 was a significant reduction in the overresponse of diodes to scattered radiation outside of the direct beam, accom-plished by reducing the amount of high-Z materials adjacent to the diodes. This change improved the diode measurement accuracy, but in the process invalidated the dose reconstruction models that were assembled based on measured data acquired with the older version of the AC. A correction mechanism was intro-duced in the reconstruction software (3DVH) to accommodate this and potential future design changes without requiring updating model parameters. For each permutation of AC serial number and beam model, the user can define in 3DVH a single correction factor which will be used to compensate for the difference in the out-of-field response between the new and original AC designs. The exact value can be determined by minimizing the dose-difference with an ionization chamber or another independent dosimeter. A single value of 1.17, corresponding to the maximum measured out-of-field response difference between the new and old AC, provided satisfactory results for all studied energies (6X, 15X, and flatten-ing filter-free 10XFFF). A library of standard cases recommended by the AAPM TG-244 Report was used for reconstructed dose verification. The overall difference between reconstructed dose and an ion chamber in a water-equivalent phantom in the targets was 0.0% ± 1.4% (1 SD). The reconstructed dose on a homogeneous phantom was also compared to a biplanar diode dosimeter (Delta4) using gamma analysis with 2% (local dose-error normalization) / 2 mm / 10% cutoff criteria. The mean agreement rate was 96.7% ± 3.7%. For the plans common with the previous comparison, the mean agreement rate was 98.3% ± 0.8%, essentially unchanged. We conclude that the proposed software modification adequately addresses the change in the dosimeter response.

Anatomical contouring variability in thoracic organs at risk.

The purpose of this study was to determine whether contouring thoracic organs at risk was consistent among medical dosimetrists and to identify how trends in dosimetrist׳s education and experience affected contouring accuracy. Qualitative and quantitative methods were used to contextualize the raw data that were obtained. A total of 3 different computed tomography (CT) data sets were provided to medical dosimetrists (N = 13) across 5 different institutions. The medical dosimetrists were directed to contour the lungs, heart, spinal cord, and esophagus. The medical dosimetrists were instructed to contour in line with their institutional standards and were allowed to use any contouring tool or technique that they would traditionally use. The contours from each medical dosimetrist were evaluated against "gold standard" contours drawn and validated by 2 radiation oncology physicians. The dosimetrist-derived contours were evaluated against the gold standard using both a Dice coefficient method and a penalty-based metric scoring system. A short survey was also completed by each medical dosimetrist to evaluate their individual contouring experience. There was no significant variation in the contouring consistency of the lungs and spinal cord. Intradosimetrist contouring was consistent for those who contoured the esophagus and heart correctly; however, medical dosimetrists with a poor metric score showed erratic and inconsistent methods of contouring.

Initial evaluation of automated treatment planning software.

Even with advanced inverse-planning techniques, radiation treatment plan opti-mization remains a very time-consuming task with great output variability, which prompted the development of more automated approaches. One commercially available technique mimics the actions of experienced human operators to pro-gressively guide the traditional optimization process with automatically created regions of interest and associated dose-volume objectives. We report on the initial evaluation of this algorithm on 10 challenging cases of locoreginally advanced head and neck cancer. All patients were treated with VMAT to 70 Gy to the gross disease and 56 Gy to the elective bilateral nodes. The results of post-treatment autoplanning (AP) were compared to the original human-driven plans (HDP). We used an objective scoring system based on defining a collection of specific dosimetric metrics and corresponding numeric score functions for each. Five AP techniques with different input dose goals were applied to all patients. The best of them averaged the composite score 8% lower than the HDP, across the patient population. The difference in median values was statistically significant at the 95% confidence level (Wilcoxon paired signed-rank test p = 0.027). This result reflects the premium the institution places on dose homogeneity, which was consistently higher with the HDPs. The OAR sparing was consistently better with the APs, the differences reaching statistical significance for the mean doses to the parotid glands (p < 0.001) and the inferior pharyngeal constrictor (p = 0.016), as well as for the maximum doses to the spinal cord (p = 0.018) and brainstem (p = 0.040). If one is prepared to accept less stringent dose homogeneity criteria from the RTOG 1016 protocol, nine APs would comply with the protocol, while providing lower OAR doses than the HDPs. Overall, AP is a promising clinical tool, but it could benefit from a better process for shifting the balance between the target dose coverage/homogeneity and OAR sparing.

Technical Note: Motion-perturbation method applied to dosimetry of dynamic MLC target tracking--A proof-of-concept.

PURPOSE: Previous studies show that dose to a moving target can be estimated using 4D measurement-guided dose reconstruction based on a process called virtual motion simulation, or VMS. A potential extension of VMS is to estimate dose during dynamic multileaf collimator (MLC)-tracking treatments. The authors introduce a modified VMS method and quantify its performance as proof-of-concept for tracking applications. METHODS: Direct measurements with a moving biplanar diode array were used to verify accuracy of the VMS dose estimates. A tracking environment for variably sized circular MLC apertures was simulated by sending preprogrammed control points to the MLC while simultaneously moving the accelerator treatment table. Sensitivity of the method to simulated tracking latency (0-700 ms) was also studied. Potential applicability of VMS to fast changing beam apertures was evaluated by modeling, based on the demonstrated dependence of the cumulative dose on the temporal dose gradient. RESULTS: When physical and virtual latencies were matched, the agreement rates (2% global/2 mm gamma) between the VMS and the biplanar dosimeter were above 96%. When compared to their own reference dose (0 induced latency), the agreement rates for VMS and biplanar array track closely up to 200 ms of induced latency with 10% low-dose cutoff threshold and 300 ms with 50% cutoff. Time-resolved measurements suggest that even in the modulated beams, the error in the cumulative dose introduced by the 200 ms VMS time resolution is not likely to exceed 0.5%. CONCLUSIONS: Based on current results and prior benchmarks of VMS accuracy, the authors postulate that this approach should be applicable to any MLC-tracking treatments where leaf speeds do not exceed those of the current Varian accelerators.

Methods, software and datasets to verify DVH calculations against analytical values: Twenty years late(r).

PURPOSE: The authors designed data, methods, and metrics that can serve as a standard, independent of any software package, to evaluate dose-volume histogram (DVH) calculation accuracy and detect limitations. The authors use simple geometrical objects at different orientations combined with dose grids of varying spatial resolution with linear 1D dose gradients; when combined, ground truth DVH curves can be calculated analytically in closed form to serve as the absolute standards. METHODS: dicom RT structure sets containing a small sphere, cylinder, and cone were created programmatically with axial plane spacing varying from 0.2 to 3 mm. Cylinders and cones were modeled in two different orientations with respect to the IEC 1217 Y axis. The contours were designed to stringently but methodically test voxelation methods required for DVH. Synthetic RT dose files were generated with 1D linear dose gradient and with grid resolution varying from 0.4 to 3 mm. Two commercial DVH algorithms-pinnacle (Philips Radiation Oncology Systems) and PlanIQ (Sun Nuclear Corp.)-were tested against analytical values using custom, noncommercial analysis software. In Test 1, axial contour spacing was constant at 0.2 mm while dose grid resolution varied. In Tests 2 and 3, the dose grid resolution was matched to varying subsampled axial contours with spacing of 1, 2, and 3 mm, and difference analysis and metrics were employed: (1) histograms of the accuracy of various DVH parameters (total volume, Dmax, Dmin, and doses to % volume: D99, D95, D5, D1, D0.03 cm(3)) and (2) volume errors extracted along the DVH curves were generated and summarized in tabular and graphical forms. RESULTS: In Test 1, pinnacle produced 52 deviations (15%) while PlanIQ produced 5 (1.5%). In Test 2, pinnacle and PlanIQ differed from analytical by >3% in 93 (36%) and 18 (7%) times, respectively. Excluding Dmin and Dmax as least clinically relevant would result in 32 (15%) vs 5 (2%) scored deviations for pinnacle vs PlanIQ in Test 1, while Test 2 would yield 53 (25%) vs 17 (8%). In Test 3, statistical analyses of volume errors extracted continuously along the curves show pinnacle to have more errors and higher variability (relative to PlanIQ), primarily due to pinnacle's lack of sufficient 3D grid supersampling. Another major driver for pinnacle errors is an inconsistency in implementation of the "end-capping"; the additional volume resulting from expanding superior and inferior contours halfway to the next slice is included in the total volume calculation, but dose voxels in this expanded volume are excluded from the DVH. PlanIQ had fewer deviations, and most were associated with a rotated cylinder modeled by rectangular axial contours; for coarser axial spacing, the limited number of cross-sectional rectangles hinders the ability to render the true structure volume. CONCLUSIONS: The method is applicable to any DVH-calculating software capable of importing dicom RT structure set and dose objects (the authors' examples are available for download). It includes a collection of tests that probe the design of the DVH algorithm, measure its accuracy, and identify failure modes. Merits and applicability of each test are discussed.

Measurement-guided volumetric dose reconstruction for helical tomotherapy.

It was previously demonstrated that dose delivered by a conventional linear accelerator using IMRT or VMAT can be reconstructed - on patient or phantom datasets - using helical diode array measurements and a technique called planned dose perturbation (PDP). This allows meaningful and intuitive analysis of the agreement between the planned and delivered dose, including direct comparison of the dose-volume histograms. While conceptually similar to modulated arc techniques, helical tomotherapy introduces significant challenges to the PDP formalism, arising primarily from TomoTherapy delivery dynamics. The temporal characteristics of the delivery are of the same order or shorter than the dosimeter's update interval (50 ms). Additionally, the prevalence of often small and complex segments, particularly with the 1 cm Y jaw setting, lead to challenges related to detector spacing. Here, we present and test a novel method of tomotherapy-PDP (TPDP) designed to meet these challenges. One of the novel techniques introduced for TPDP is organization of the subbeams into larger subunits called sectors, which assures more robust synchronization of the measurement and delivery dynamics. Another important change is the optional application of a correction based on ion chamber (IC) measurements in the phantom. The TPDP method was validated by direct comparisons to the IC and an independent, biplanar diode array dosimeter previously evaluated for tomotherapy delivery quality assurance. Nineteen plans with varying complexity were analyzed for the 2.5 cm tomotherapy jaw setting and 18 for the 1 cm opening. The dose differences between the TPDP and IC were 1.0% ± 1.1% and 1.1% ± 1.1%, for 2.5 and 1.0 cm jaw plans, respectively. Gamma analysis agreement rates between TPDP and the independent array were: 99.1%± 1.8% (using 3% global normalization/3 mm criteria) and 93.4% ± 7.1% (using 2% global/2 mm) for the 2.5 cm jaw plans; for 1 cm plans, they were 95.2% ± 6.7% (3% G/3) and 83.8% ± 12% (2% G/2). We conclude that TPDP is capable of volumetric dose reconstruction with acceptable accuracy. However, the challenges of fast tomotherapy delivery dynamics make TPDP less precise than the IMRT/VMAT PDP version, particularly for the 1 cm jaw setting.

Motion as perturbation. II. Development of the method for dosimetric analysis of motion effects with fixed-gantry IMRT.

PURPOSE: In this work, the feasibility of implementing a motion-perturbation approach to accurately estimate volumetric dose in the presence of organ motion--previously demonstrated for VMAT--is studied for static gantry IMRT. The method's accuracy is improved for the voxels that have very low planned dose but acquire appreciable dose due to motion. The study describes the modified algorithm and its experimental validation and provides an example of a clinical application. METHODS: A contoured region-of-interest is propagated according to the predefined motion kernel throughout time-resolved 4D phantom dose grids. This timed series of 3D dose grids is produced by the measurement-guided dose reconstruction algorithm, based on an irradiation of a static ARCCHECK (AC) helical dosimeter array (Sun Nuclear Corp., Melbourne, FL). Each moving voxel collects dose over the dynamic simulation. The difference in dose-to-moving voxel vs dose-to-static voxel in-phantom forms the basis of a motion perturbation correction that is applied to the corresponding voxel in the patient dataset. A new method to synchronize the accelerator and dosimeter clocks, applicable to fixed-gantry IMRT, was developed. Refinements to the algorithm account for the excursion of low dose voxels into high dose regions, causing appreciable dose increase due to motion (LDVE correction). For experimental validation, four plans using TG-119 structure sets and objectives were produced using segmented IMRT direct machine parameters optimization in Pinnacle treatment planning system (v. 9.6, Philips Radiation Oncology Systems, Fitchburg, WI). All beams were delivered with the gantry angle of 0°. Each beam was delivered three times: (1) to the static AC centered on the room lasers; (2) to a static phantom containing a MAPCHECK2 (MC2) planar diode array dosimeter (Sun Nuclear); and (3) to the moving MC2 phantom. The motion trajectory was an ellipse in the IEC XY plane, with 3 and 1.5 cm axes. The period was 5 s, with the resulting average motion speed of 1.45 cm/s. The motion-perturbed high resolution (2 mm voxel) volumetric dose grids on the MC2 phantom were generated for each beam. From each grid, a coronal dose plane at the detector level was extracted and compared to the corresponding moving MC2 measurement, using gamma analysis with both global (G) and local (L) dose-error normalization. RESULTS: Using the TG-119 criteria of (3%G/3 mm), per beam average gamma analysis passing rates exceeded 95% in all cases. No individual beam had a passing rate below 91%. LDVE correction eliminated systematic disagreement patterns at the beams' aperture edges. In a representative example, application of LDVE correction improved (2%L/2 mm) gamma analysis passing rate for an IMRT beam from 74% to 98%. CONCLUSIONS: The effect of motion on the moving region-of-interest IMRT dose can be estimated with a standard, static phantom QA measurement, provided the motion characteristics are independently known from 4D CT or otherwise. The motion-perturbed absolute dose estimates were validated by the direct planar diode array measurements, and were found to reliably agree with them in a homogeneous phantom.

Cross-validation of two commercial methods for volumetric high-resolution dose reconstruction on a phantom for non-coplanar VMAT beams.

BACKGROUND AND PURPOSE: Delta(4) (ScandiDos AB, Uppsala, Sweden) and ArcCHECK with 3DVH software (Sun Nuclear Corp., Melbourne, FL, USA) are commercial quasi-three-dimensional diode dosimetry arrays capable of volumetric measurement-guided dose reconstruction. A method to reconstruct dose for non-coplanar VMAT beams with 3DVH is described. The Delta(4) 3D dose reconstruction on its own phantom for VMAT delivery has not been thoroughly evaluated previously, and we do so by comparison with 3DVH. MATERIALS AND METHODS: Reconstructed volumetric doses for VMAT plans delivered with different table angles were compared between the Delta(4) and 3DVH using gamma analysis. RESULTS: The average γ (2% local dose-error normalization/2mm) passing rate comparing the directly measured Delta(4) diode dose with 3DVH was 98.2 ± 1.6% (1SD). The average passing rate for the full volumetric comparison of the reconstructed doses on a homogeneous cylindrical phantom was 95.6 ± 1.5%. No dependence on the table angle was observed. CONCLUSIONS: Modified 3DVH algorithm is capable of 3D VMAT dose reconstruction on an arbitrary volume for the full range of table angles. Our comparison results between different dosimeters make a compelling case for the use of electronic arrays with high-resolution 3D dose reconstruction as primary means of evaluating spatial dose distributions during IMRT/VMAT verification.

Evaluating IMRT and VMAT dose accuracy: practical examples of failure to detect systematic errors when applying a commonly used metric and action levels.

PURPOSE: This study (1) examines a variety of real-world cases where systematic errors were not detected by widely accepted methods for IMRT/VMAT dosimetric accuracy evaluation, and (2) drills-down to identify failure modes and their corresponding means for detection, diagnosis, and mitigation. The primary goal of detailing these case studies is to explore different, more sensitive methods and metrics that could be used more effectively for evaluating accuracy of dose algorithms, delivery systems, and QA devices. METHODS: The authors present seven real-world case studies representing a variety of combinations of the treatment planning system (TPS), linac, delivery modality, and systematic error type. These case studies are typical to what might be used as part of an IMRT or VMAT commissioning test suite, varying in complexity. Each case study is analyzed according to TG-119 instructions for gamma passing rates and action levels for per-beam and/or composite plan dosimetric QA. Then, each case study is analyzed in-depth with advanced diagnostic methods (dose profile examination, EPID-based measurements, dose difference pattern analysis, 3D measurement-guided dose reconstruction, and dose grid inspection) and more sensitive metrics (2% local normalization/2 mm DTA and estimated DVH comparisons). RESULTS: For these case studies, the conventional 3%/3 mm gamma passing rates exceeded 99% for IMRT per-beam analyses and ranged from 93.9% to 100% for composite plan dose analysis, well above the TG-119 action levels of 90% and 88%, respectively. However, all cases had systematic errors that were detected only by using advanced diagnostic techniques and more sensitive metrics. The systematic errors caused variable but noteworthy impact, including estimated target dose coverage loss of up to 5.5% and local dose deviations up to 31.5%. Types of errors included TPS model settings, algorithm limitations, and modeling and alignment of QA phantoms in the TPS. Most of the errors were correctable after detection and diagnosis, and the uncorrectable errors provided useful information about system limitations, which is another key element of system commissioning. CONCLUSIONS: Many forms of relevant systematic errors can go undetected when the currently prevalent metrics for IMRT∕VMAT commissioning are used. If alternative methods and metrics are used instead of (or in addition to) the conventional metrics, these errors are more likely to be detected, and only once they are detected can they be properly diagnosed and rooted out of the system. Removing systematic errors should be a goal not only of commissioning by the end users but also product validation by the manufacturers. For any systematic errors that cannot be removed, detecting and quantifying them is important as it will help the physicist understand the limits of the system and work with the manufacturer on improvements. In summary, IMRT and VMAT commissioning, along with product validation, would benefit from the retirement of the 3%/3 mm passing rates as a primary metric of performance, and the adoption instead of tighter tolerances, more diligent diagnostics, and more thorough analysis.

Experimentally studied dynamic dose interplay does not meaningfully affect target dose in VMAT SBRT lung treatments.

PURPOSE: The effects of respiratory motion on the tumor dose can be divided into the gradient and interplay effects. While the interplay effect is likely to average out over a large number of fractions, it may play a role in hypofractionated [stereotactic body radiation therapy (SBRT)] treatments. This subject has been extensively studied for intensity modulated radiation therapy but less so for volumetric modulated arc therapy (VMAT), particularly in application to hypofractionated regimens. Also, no experimental study has provided full four-dimensional (4D) dose reconstruction in this scenario. The authors demonstrate how a recently described motion perturbation method, with full 4D dose reconstruction, is applied to describe the gradient and interplay effects during VMAT lung SBRT treatments. METHODS: VMAT dose delivered to a moving target in a patient can be reconstructed by applying perturbations to the treatment planning system-calculated static 3D dose. Ten SBRT patients treated with 6 MV VMAT beams in five fractions were selected. The target motion (motion kernel) was approximated by 3D rigid body translation, with the tumor centroids defined on the ten phases of the 4DCT. The motion was assumed to be periodic, with the period T being an average from the empirical 4DCT respiratory trace. The real observed tumor motion (total displacement ≤ 8 mm) was evaluated first. Then, the motion range was artificially increased to 2 or 3 cm. Finally, T was increased to 60 s. While not realistic, making T comparable to the delivery time elucidates if the interplay effect can be observed. For a single fraction, the authors quantified the interplay effect as the maximum difference in the target dosimetric indices, most importantly the near-minimum dose (D99%), between all possible starting phases. For the three- and five-fractions, statistical simulations were performed when substantial interplay was found. RESULTS: For the motion amplitudes and periods obtained from the 4DCT, the interplay effect is negligible (<0.2%). It is also small (0.9% average, 2.2% maximum) when the target excursion increased to 2-3 cm. Only with large motion and increased period (60 s) was a significant interplay effect observed, with D99% ranging from 16% low to 17% high. The interplay effect was statistically significantly lower for the three- and five-fraction statistical simulations. Overall, the gradient effect dominates the clinical situation. CONCLUSIONS: A novel method was used to reconstruct the volumetric dose to a moving tumor during lung SBRT VMAT deliveries. With the studied planning and treatment technique for realistic motion periods, regardless of the amplitude, the interplay has nearly no impact on the near-minimum dose. The interplay effect was observed, for study purposes only, with the period comparable to the VMAT delivery time.

Validation of measurement-guided 3D VMAT dose reconstruction on a heterogeneous anthropomorphic phantom.

3DVH software (Sun Nuclear Corp., Melbourne, FL) is capable of generating a volumetric patient VMAT dose by applying a volumetric perturbation algorithm based on comparing measurement-guided dose reconstruction and TPS-calculated dose to a cylindrical phantom. The primary purpose of this paper is to validate this dose reconstruction on an anthropomorphic heterogeneous thoracic phantom by direct comparison to independent measurements. The dosimetric insert to the phantom is novel, and thus the secondary goal is to demonstrate how it can be used for the hidden target end-to-end testing of VMAT treatments in lung. A dosimetric insert contains a 4 cm diameter unit-density spherical target located inside the right lung (0.21 g/cm(3) density). It has 26 slots arranged in two orthogonal directions, milled to hold optically stimulated luminescent dosimeters (OSLDs). Dose profiles in three cardinal orthogonal directions were obtained for five VMAT plans with varying degrees of modulation. After appropriate OSLD corrections were applied, 3DVH measurement-guided VMAT dose reconstruction agreed 100% with the measurements in the unit density target sphere at 3%/3 mm level (composite analysis) for all profile points for the four less-modulated VMAT plans, and for 96% of the points in the highly modulated C-shape plan (from TG-119). For this latter plan, while 3DVH shows acceptable agreement with independent measurements in the unit density target, in the lung disagreement with experiment is relatively high for both the TPS calculation and 3DVH reconstruction. For the four plans excluding the C-shape, 3%/3 mm overall composite analysis passing rates for 3DVH against independent measurement ranged from 93% to 100%. The C-shape plan was deliberately chosen as a stress test of the algorithm. The dosimetric spatial alignment hidden target test demonstrated the average distance to agreement between the measured and TPS profiles in the steep dose gradient area at the edge of the 2 cm target to be 1.0 ± 0.7, 0.3 ± 0.3, and 0.3 ± 0.3 mm for the IEC X, Y, and Z directions, respectively.

On the use of biomathematical models in patient-specific IMRT dose QA.

PURPOSE: To investigate the use of biomathematical models such as tumor control probability (TCP) and normal tissue complication probability (NTCP) as new quality assurance (QA) metrics. METHODS: Five different types of error (MLC transmission, MLC penumbra, MLC tongue and groove, machine output, and MLC position) were intentionally induced to 40 clinical intensity modulated radiation therapy (IMRT) patient plans (20 H&N cases and 20 prostate cases) to simulate both treatment planning system errors and machine delivery errors in the IMRT QA process. The changes in TCP and NTCP for eight different anatomic structures (H&N: CTV, GTV, both parotids, spinal cord, larynx; prostate: CTV, rectal wall) were calculated as the new QA metrics to quantify the clinical impact on patients. The correlation between the change in TCP∕NTCP and the change in selected DVH values was also evaluated. The relation between TCP∕NTCP change and the characteristics of the TCP∕NTCP curves is discussed. RESULTS: ΔTCP and ΔNTCP were summarized for each type of induced error and each structure. The changes/degradations in TCP and NTCP caused by the errors vary widely depending on dose patterns unique to each plan, and are good indicators of each plan's "robustness" to that type of error. CONCLUSIONS: In this in silico QA study the authors have demonstrated the possibility of using biomathematical models not only as patient-specific QA metrics but also as objective indicators that quantify, pretreatment, a plan's robustness with respect to possible error types.

Motion as a perturbation: measurement-guided dose estimates to moving patient voxels during modulated arc deliveries.

PURPOSE: To present a framework for measurement-guided VMAT dose reconstruction to moving patient voxels from a known motion kernel and the static phantom data, and to validate this perturbation-based approach with the proof-of-principle experiments. METHODS: As described previously, the VMAT 3D dose to a static patient can be estimated by applying a phantom measurement-guided perturbation to the treatment planning system (TPS)-calculated dose grid. The fraction dose to any voxel in the presence of motion, assuming the motion kernel is known, can be derived in a similar fashion by applying a measurement-guided motion perturbation. The dose to the diodes in a helical phantom is recorded at 50 ms intervals and is transformed into a series of time-resolved high-density volumetric dose grids. A moving voxel is propagated through this 4D dose space and the fraction dose to that voxel in the phantom is accumulated. The ratio of this motion-perturbed, reconstructed dose to the TPS dose in the phantom serves as a perturbation factor, applied to the TPS fraction dose to the similarly situated voxel in the patient. This approach was validated by the ion chamber and film measurements on four phantoms of different shape and structure: homogeneous and inhomogeneous cylinders, a homogeneous cube, and an anthropomorphic thoracic phantom. A 2D motion stage was used to simulate the motion. The stage position was synchronized with the beam start time with the respiratory gating simulator. The motion patterns were designed such that the motion speed was in the upper range of the expected tumor motion (1-1.4 cm∕s) and the range exceeded the normally observed limits (up to 5.7 cm). The conformal arc plans for X or Y motion (in the IEC 61217 coordinate system) consisted of manually created narrow (3 cm) rectangular strips moving in-phase (tracking) or phase-shifted by 90° (crossing) with respect to the phantom motion. The XY motion was tested with the computer-derived VMAT MLC sequences. For all phantoms and plans, time-resolved (10 Hz) ion chamber dose was collected. In addition, coronal (XY) films were exposed in the cube phantom to a VMAT beam with two different starting phases, and compared to the reconstructed motion-perturbed dose planes. RESULTS: For the X or Y motions with the moving strip and geometrical phantoms, the maximum difference between perturbation-reconstructed and ion chamber doses did not exceed 1.9%, and the average for any motion pattern∕starting phase did not exceed 1.3%. For the VMAT plans on the cubic and thoracic phantoms, one point exhibited a 3.5% error, while the remaining five were all within 1.1%. Across all the measurements (N = 22), the average disagreement was 0.5 ± 1.3% (1 SD). The films exhibited γ(3%∕3 mm) passing rates ≥90%. CONCLUSIONS: The dose to an arbitrary moving voxel in a patient can be estimated with acceptable accuracy for a VMAT delivery, by performing a single QA measurement with a cylindrical phantom and applying two consecutive perturbations to the TPS-calculated patient dose. The first one accounts for the differences between the planned and delivered static doses, while the second one corrects for the motion.