Outcomes and potential impact of a virtual hands-on training program on MRI staging confidence and performance in rectal cancer.

OBJECTIVES: To explore the potential impact of a dedicated virtual training course on MRI staging confidence and performance in rectal cancer. METHODS: Forty-two radiologists completed a stepwise virtual training course on rectal cancer MRI staging composed of a pre-course (baseline) test with 7 test cases (5 staging, 2 restaging), a 1-day online workshop, 1 month of individual case readings (n = 70 cases with online feedback), a live online feedback session supervised by two expert faculty members, and a post-course test. The ESGAR structured reporting templates for (re)staging were used throughout the course. Results of the pre-course and post-course test were compared in terms of group interobserver agreement (Krippendorf's alpha), staging confidence (perceived staging difficulty), and diagnostic accuracy (using an expert reference standard). RESULTS: Though results were largely not statistically significant, the majority of staging variables showed a mild increase in diagnostic accuracy after the course, ranging between + 2% and + 17%. A similar trend was observed for IOA which improved for nearly all variables when comparing the pre- and post-course. There was a significant decrease in the perceived difficulty level (p = 0.03), indicating an improved diagnostic confidence after completion of the course. CONCLUSIONS: Though exploratory in nature, our study results suggest that use of a dedicated virtual training course and web platform has potential to enhance staging performance, confidence, and interobserver agreement to assess rectal cancer on MRI virtual training and could thus be a good alternative (or addition) to in-person training. CLINICAL RELEVANCE STATEMENT: Rectal cancer MRI reporting quality is highly dependent on radiologists' expertise, stressing the need for dedicated training/teaching. This study shows promising results for a virtual web-based training program, which could be a good alternative (or addition) to in-person training. KEY POINTS: • Rectal cancer MRI reporting quality is highly dependent on radiologists' expertise, stressing the need for dedicated training and teaching. • Using a dedicated virtual training course and web-based platform, encouraging first results were achieved to improve staging accuracy, diagnostic confidence, and interobserver agreement. • These exploratory results suggest that virtual training could thus be a good alternative (or addition) to in-person training.

Sense and nonsense of yT-staging on MRI after chemoradiotherapy in rectal cancer.

AIM: The aim of this work was to investigate the value of rectal cancer T-staging on MRI after chemoradiotherapy (ymrT-staging) in relation to the degree of fibrotic transformation of the tumour bed as assessed using the pathological tumour regression grade (pTRG) of Mandard as a standard of reference. METHOD: Twenty two radiologists, including five rectal MRI experts and 17 'nonexperts' (general/abdominal radiologists), evaluated the ymrT stage on the restaging MRIs of 90 rectal cancer patients after chemoradiotherapy. The ymrT stage was compared with the final ypT stage at histopathology; the percentages of correct staging (ymrT = ypT), understaging (ymrT < ypT) and overstaging (ymrT > ypT) were calculated and compared between patients with predominant tumour at histopathology (pTRG4-5) and patients with predominant fibrosis (pTRG1-3). Interobserver agreement (IOA) was computed using Krippendorff's alpha. RESULTS: Average ymrT/ypT stage concordance was 48% for the experts and 43% for the nonexperts; ymrT/ypT stage concordance was significantly higher in the pTRG4-5 subgroup (58% vs. 41% for the pTRG1-3 group; p = 0.01), with the best results for the MRI experts. Overstaging was the main source of error, especially in the pTRG1-3 subgroup (average overstaging rate 38%-44% vs. 13%-55% in the pTRG4-5 subgroup). IOA was higher for the expert versus nonexpert readers (α = 0.67 vs. α = 0.39). CONCLUSIONS: ymrT-staging is moderately accurate; accuracy is higher in poorly responding patients with predominant tumour but low in good responders with predominant fibrosis, resulting in significant overstaging. Radiologists should shift their focus from ymrT-staging to detecting gross residual (and progressive) disease, and identifying potential candidates for organ preservation who would benefit from further clinical and endoscopic evaluation to guide final treatment planning.

Development and multicenter validation of a multiparametric imaging model to predict treatment response in rectal cancer.

OBJECTIVES: To develop and validate a multiparametric model to predict neoadjuvant treatment response in rectal cancer at baseline using a heterogeneous multicenter MRI dataset. METHODS: Baseline staging MRIs (T2W (T2-weighted)-MRI, diffusion-weighted imaging (DWI) / apparent diffusion coefficient (ADC)) of 509 patients (9 centres) treated with neoadjuvant chemoradiotherapy (CRT) were collected. Response was defined as (1) complete versus incomplete response, or (2) good (Mandard tumor regression grade (TRG) 1-2) versus poor response (TRG3-5). Prediction models were developed using combinations of the following variable groups: (1) Non-imaging: age/sex/tumor-location/tumor-morphology/CRT-surgery interval (2) Basic staging: cT-stage/cN-stage/mesorectal fascia involvement, derived from (2a) original staging reports, or (2b) expert re-evaluation (3) Advanced staging: variables from 2b combined with cTN-substaging/invasion depth/extramural vascular invasion/tumor length (4) Quantitative imaging: tumour volume + first-order histogram features (from T2W-MRI and DWI/ADC) Models were developed with data from 6 centers (n = 412) using logistic regression with the Least Absolute Shrinkage and Selector Operator (LASSO) feature selection, internally validated using repeated (n = 100) random hold-out validation, and externally validated using data from 3 centers (n = 97). RESULTS: After external validation, the best model (including non-imaging and advanced staging variables) achieved an area under the curve of 0.60 (95%CI=0.48-0.72) to predict complete response and 0.65 (95%CI=0.53-0.76) to predict a good response. Quantitative variables did not improve model performance. Basic staging variables consistently achieved lower performance compared to advanced staging variables. CONCLUSIONS: Overall model performance was moderate. Best results were obtained using advanced staging variables, highlighting the importance of good-quality staging according to current guidelines. Quantitative imaging features had no added value (in this heterogeneous dataset). CLINICAL RELEVANCE STATEMENT: Predicting tumour response at baseline could aid in tailoring neoadjuvant therapies for rectal cancer. This study shows that image-based prediction models are promising, though are negatively affected by variations in staging quality and MRI acquisition, urging the need for harmonization. KEY POINTS: This multicenter study combining clinical information and features derived from MRI rendered disappointing performance to predict response to neoadjuvant treatment in rectal cancer. Best results were obtained with the combination of clinical baseline information and state-of-the-art image-based staging variables, highlighting the importance of good quality staging according to current guidelines and staging templates. No added value was found for quantitative imaging features in this multicenter retrospective study. This is likely related to acquisition variations, which is a major problem for feature reproducibility and thus model generalizability.

Predicting response to chemoradiotherapy in rectal cancer via visual morphologic assessment and staging on baseline MRI: a multicenter and multireader study.

PURPOSE: Pre-treatment knowledge of the anticipated response of rectal tumors to neoadjuvant chemoradiotherapy (CRT) could help to further optimize the treatment. Van Griethuysen et al. proposed a visual 5-point confidence score to predict the likelihood of response on baseline MRI. Aim was to evaluate this score in a multicenter and multireader study setting and compare it to two simplified (4-point and 2-point) adaptations in terms of diagnostic performance, interobserver agreement (IOA), and reader preference. METHODS: Twenty-two radiologists from 14 countries (5 MRI-experts,17 general/abdominal radiologists) retrospectively reviewed 90 baseline MRIs to estimate if patients would likely achieve a (near-)complete response (nCR); first using the 5-point score by van Griethuysen (1=highly unlikely to 5=highly likely to achieve nCR), second using a 4-point adaptation (with 1-point each for high-risk T-stage, obvious mesorectal fascia invasion, nodal involvement, and extramural vascular invasion), and third using a 2-point score (unlikely/likely to achieve nCR). Diagnostic performance was calculated using ROC curves and IOA using Krippendorf's alpha (α). RESULTS: Areas under the ROC curve to predict the likelihood of a nCR were similar for the three methods (0.71-0.74). IOA was higher for the 5- and 4-point scores (α=0.55 and 0.57 versus 0.46 for the 2-point score) with best results for the MRI-experts (α=0.64-0.65). Most readers (55%) favored the 4-point score. CONCLUSIONS: Visual morphologic assessment and staging methods can predict neoadjuvant treatment response with moderate-good performance. Compared to a previously published confidence-based scoring system, study readers preferred a simplified 4-point risk score based on high-risk T-stage, MRF involvement, nodal involvement, and EMVI.

Pelvic CT in addition to MRI to differentiate between rectal and sigmoid cancer on imaging using the sigmoid take-off as a landmark.

BACKGROUND: The sigmoid take-off (STO) is a recently established landmark to discern rectal from sigmoid cancer on imaging. STO-assessment can be challenging on magnetic resonance imaging (MRI) due to varying axial planes. PURPOSE: To establish the benefit of using computed tomography (CT; with consistent axial planes), in addition to MRI, to anatomically classify rectal versus sigmoid cancer using the STO. MATERIAL AND METHODS: A senior and junior radiologist retrospectively classified 40 patients with rectal/rectosigmoid cancers using the STO, first on MRI-only (sagittal and oblique-axial views) and then using a combination of MRI and axial CT. Tumors were classified as rectal/rectosigmoid/sigmoid (according to published STO definitions) and then dichotomized into rectal versus sigmoid. Diagnostic confidence was documented using a 5-point scale. RESULTS: Adding CT resulted in a change in anatomical tumor classification in 4/40 cases (10%) for the junior reader and in 6/40 cases (15%) for the senior reader. Diagnostic confidence increased significantly after adding CT for the junior reader (mean score 3.85 vs. 4.27; P < 0.001); confidence of the senior reader was not affected (4.28 vs. 4.25; P = 0.80). Inter-observer agreement was similarly good for MRI only (κ=0.77) and MRI + CT (κ=0.76). Readers reached consensus on the classification of rectal versus sigmoid cancer in 78%-85% of cases. CONCLUSION: Availability of a consistent axial imaging plane - in the case of this study provided by CT - in addition to a standard MRI protocol with sagittal and oblique-axial imaging views can be helpful to more confidently localize tumors using the STO as a landmark, especially for more junior readers.

Comparison of MRI response evaluation methods in rectal cancer: a multicentre and multireader validation study.

OBJECTIVES: To compare four previously published methods for rectal tumor response evaluation after chemoradiotherapy on MRI. METHODS: Twenty-two radiologists (5 rectal MRI experts, 17 general/abdominal radiologists) retrospectively reviewed the post-chemoradiotherapy MRIs of 90 patients, scanned at 10 centers (with non-standardized protocols). They applied four response methods; two based on T2W-MRI only (MRI tumor regression grade (mrTRG); split-scar sign), and two based on T2W-MRI+DWI (modified-mrTRG; DWI-patterns). Image quality was graded using a 0-6-point score (including slice thickness and in-plane resolution; sequence angulation; DWI b-values, signal-to-noise, and artefacts); scores < 4 were classified below average. Mixed model linear regression was used to calculate average sensitivity/specificity/accuracy to predict a complete response (versus residual tumor) and assess the impact of reader experience and image quality. Group interobserver agreement (IOA) was calculated using Krippendorff's alpha. Readers were asked to indicate their preferred scoring method(s). RESULTS: Average sensitivity/specificity/accuracy was 57%/64%/62% (mrTRG), 36%/79%/66% (split-scar), 40%/79%/67% (modified-mrTRG), and 37%/82%/68% (DWI-patterns); mrTRG showed higher sensitivity but lower specificity and accuracy (p < 0.001) compared to the other methods. IOA was lower for the split scar method (0.18 vs. 0.39-0.43). Higher reader experience had a significant positive effect on diagnostic performance and IOA (except for the split scar sign); below-average imaging quality had a significant negative effect on diagnostic performance. DWI pattern was selected as the preferred method by 73% of readers. CONCLUSIONS: Methods incorporating DWI showed the most favorable results when combining diagnostic performance, IOA, and reader preference. Reader experience and image quality clearly impacted diagnostic performance emphasizing the need for state-of-the-art imaging and dedicated radiologist training. KEY POINTS: • In a multireader study comparing 4 MRI methods for rectal tumor response evaluation, those incorporating DWI showed the best results when combining diagnostic performance, IOA, and reader preference. • The most preferred method (by 73% of readers) was the "DWI patterns" approach with an accuracy of 68%, high specificity of 82%, and group IOA of 0.43. • Reader experience level and MRI quality had an evident effect on diagnostic performance and IOA.

Pearls and pitfalls of structured staging and reporting of rectal cancer on MRI: an international multireader study.

OBJECTIVES: To investigate uniformity and pitfalls in structured radiological staging of rectal cancer. METHODS: Twenty-one radiologists (12 countries) staged 75 rectal cancers on MRI using a structured reporting template. Interobserver agreement (IOA) was calculated as the percentage agreement between readers (categorical variables) and Krippendorff's α (continuous variables). Agreement with an expert consensus served as a surrogate standard of reference to estimate diagnostic accuracy. Polychoric correlation coefficients were used to assess correlations between diagnostic confidence and accuracy (=agreement with expert consensus). RESULTS: Uniformity to diagnose high-risk (≥cT3 ab) versus low-risk (≤cT3 cd) cT-stage, cN0 versus cN+, lateral nodes and tumour deposits, MRF and sphincter involvement, and solid versus mucinous tumours was high with IOA > 80% in the majority of cases (and >80% agreement with expert consensus). Results for assessing extramural vascular invasion, cT-stage (cT1-2/cT3/cT4a/cT4b), cN-stage (cN0/N1/N2), relation to the peritoneal reflection, extent of sphincter involvement (internal/intersphincteric/external) and morphology (solid/annular/semi-annular) were considerably poorer. IOA was high (α = 0.72-0.84) for tumour height/length and extramural invasion depth, but low for tumour-MRF distance and number of (suspicious) nodes (α = 0.05-0.55). There was a significant positive correlation between diagnostic confidence and accuracy (=agreement with expert consensus) (p < 0.001-p = 0.003). CONCLUSIONS: - Several staging items lacked sufficient reproducibility.- Results for cT- and N-staging g improved when using a dichotomized stratification.- Considering the significant correlation between diagnostic confidence and accuracy, a confidence level may be incorporated into structured reporting for specific items with low reproducibility. ADVANCES IN KNOWLEDGE: Although structured reporting aims to achieve uniformity in reporting, several items lack sufficient reproducibility and might benefit from dichotomized assessment and incorporating confidence levels.

Evolutions in rectal cancer MRI staging and risk stratification in The Netherlands.

PURPOSE: To analyze how the MRI reporting of rectal cancer has evolved (following guideline updates) in The Netherlands. METHODS: Retrospective analysis of 712 patients (2011-2018) from 8 teaching hospitals in The Netherlands with available original radiological staging reports that were re-evaluated by a dedicated MR expert using updated guideline criteria. Original reports were classified as "free-text," "semi-structured," or "template" and completeness of reporting was documented. Patients were categorized as low versus high risk, first based on the original reports (high risk = cT3-4, cN+, and/or cMRF+) and then based on the expert re-evaluations (high risk = cT3cd-4, cN+, MRF+, and/or EMVI+). Evolutions over time were studied by splitting the inclusion period in 3 equal time periods. RESULTS: A significant increase in template reporting was observed (from 1.6 to 17.6-29.6%; p < 0.001), along with a significant increase in the reporting of cT-substage, number of N+ and extramesorectal nodes, MRF invasion and tumor-MRF distance, EMVI, anal sphincter involvement, and tumor morphology and circumference. Expert re-evaluation changed the risk classification from high to low risk in 18.0% of cases and from low to high risk in 1.7% (total 19.7%). In the majority (17.9%) of these cases, the changed risk classification was likely (at least in part) related to use of updated guideline criteria, which mainly led to a reduction in high-risk cT-stage and nodal downstaging. CONCLUSION: Updated concepts of risk stratification have increasingly been adopted, accompanied by an increase in template reporting and improved completeness of reporting. Use of updated guideline criteria resulted in considerable downstaging (of mainly high-risk cT-stage and nodal stage).

Sources of variation in multicenter rectal MRI data and their effect on radiomics feature reproducibility.

OBJECTIVES: To investigate sources of variation in a multicenter rectal cancer MRI dataset focusing on hardware and image acquisition, segmentation methodology, and radiomics feature extraction software. METHODS: T2W and DWI/ADC MRIs from 649 rectal cancer patients were retrospectively acquired in 9 centers. Fifty-two imaging features (14 first-order/6 shape/32 higher-order) were extracted from each scan using whole-volume (expert/non-expert) and single-slice segmentations using two different software packages (PyRadiomics/CapTk). Influence of hardware, acquisition, and patient-intrinsic factors (age/gender/cTN-stage) on ADC was assessed using linear regression. Feature reproducibility was assessed between segmentation methods and software packages using the intraclass correlation coefficient. RESULTS: Image features differed significantly (p < 0.001) between centers with more substantial variations in ADC compared to T2W-MRI. In total, 64.3% of the variation in mean ADC was explained by differences in hardware and acquisition, compared to 0.4% by patient-intrinsic factors. Feature reproducibility between expert and non-expert segmentations was good to excellent (median ICC 0.89-0.90). Reproducibility for single-slice versus whole-volume segmentations was substantially poorer (median ICC 0.40-0.58). Between software packages, reproducibility was good to excellent (median ICC 0.99) for most features (first-order/shape/GLCM/GLRLM) but poor for higher-order (GLSZM/NGTDM) features (median ICC 0.00-0.41). CONCLUSIONS: Significant variations are present in multicenter MRI data, particularly related to differences in hardware and acquisition, which will likely negatively influence subsequent analysis if not corrected for. Segmentation variations had a minor impact when using whole volume segmentations. Between software packages, higher-order features were less reproducible and caution is warranted when implementing these in prediction models. KEY POINTS: • Features derived from T2W-MRI and in particular ADC differ significantly between centers when performing multicenter data analysis. • Variations in ADC are mainly (> 60%) caused by hardware and image acquisition differences and less so (< 1%) by patient- or tumor-intrinsic variations. • Features derived using different image segmentations (expert/non-expert) were reproducible, provided that whole-volume segmentations were used. When using different feature extraction software packages with similar settings, higher-order features were less reproducible.

The sigmoid take-off as a landmark to distinguish rectal from sigmoid tumours on MRI: Reproducibility, pitfalls and potential impact on treatment stratification.

PURPOSE: The sigmoid take-off (STO) was recently introduced as a preferred landmark, agreed upon by expert consensus recommendation, to discern rectal from sigmoid cancer on imaging. Aim of this study was to assess the reproducibility of the STO, explore its potential treatment impact and identify its main interpretation pitfalls. METHODS: Eleven international radiologists (with varying expertise) retrospectively assessed n = 155 patients with previously clinically staged upper rectal/rectosigmoid tumours and re-classified them using the STO as completely below (rectum), straddling the STO (rectosigmoid) or completely above (sigmoid), after which scores were dichotomized as rectum (below/straddling STO) and sigmoid (above STO), being the clinically most relevant distinction. A random subset of n = 48 was assessed likewise by 6 colorectal surgeons. . RESULTS: Interobserver agreement (IOA) for the 3-category score ranged from κ0.19-0.82 (radiologists) and κ0.32-0.72 (surgeons), with highest scores for the most experienced radiologists (κ0.69-0.76). Of the 155 cases, 44 (28%) were re-classified by ≥ 80% of radiologists as sigmoid cancers; 36 of these originally received neoadjuvant treatment which in retrospect might have been omitted if the STO had been applied. Main interpretation pitfalls were related to anatomical variations, borderline cases near the STO and angulation of axial imaging planes. CONCLUSIONS: Good agreement was reached for experienced radiologists. Despite considerable variation among less-expert readers, use of the STO could have changed treatment in ±1/4 of patients in our cohort. Identified interpretation pitfalls may serve as a basis for teaching and to further optimize MR protocols.