Conversion of a colorectal cancer guideline into clinical decision trees with assessment of validity.

OBJECTIVE: The interpretation and clinical application of guidelines can be challenging and time-consuming, which may result in noncompliance to guidelines. The aim of this study was to convert the Dutch guideline for colorectal cancer (CRC) into decision trees and subsequently implement decision trees in an online decision support environment to facilitate guideline application. METHODS: The recommendations of the Dutch CRC guidelines (published in 2014) were translated into decision trees consisting of decision nodes, branches and leaves that represent data items, data item values and recommendations, respectively. Decision trees were discussed with experts in the field and published as interactive open access decision support software (available at www.oncoguide.nl). Decision tree validation and a concordance analysis were performed using consecutive reports (January 2016-January 2017) from CRC multidisciplinary tumour boards (MTBs) at Amsterdam University Medical Centers, location AMC. RESULTS: In total, we developed 34 decision trees driven by 101 decision nodes based on the guideline recommendations. Decision trees represented recommendations for diagnostics (n = 1), staging (n = 10), primary treatment (colon: n = 1, rectum: n = 5, colorectal: n = 9), pathology (n = 4) and follow-up (n = 3) and included one overview decision tree for optimal navigation. We identified several guideline information gaps and areas of inconclusive evidence. A total of 158 patients' MTB reports were eligible for decision tree validation and resulted in treatment recommendations in 80% of cases. The concordance rate between decision tree treatment recommendations and MTB advices was 81%. Decision trees reported in 22 out of 24 non-concordant cases (92%) that no guideline recommendation was available. CONCLUSIONS: We successfully converted the Dutch CRC guideline into decision trees and identified several information gaps and areas of inconclusive evidence, the latter being the main cause of the observed disagreement between decision tree recommendations and MTB advices. Decision trees may contribute to future strategies to optimize quality of care for CRC patients.

Practice Variation in the Adjuvant Treatment of Colon Cancer in the Netherlands: A Population-based Study.

BACKGROUND/AIM: Adjuvant chemotherapy is recommended for a subgroup of colon cancer patients based on patient and tumour characteristics. Population-based data on the adoption of the prevailing guideline recommendations including the assessment of tumour mismatch repair (MMR) status are limited, while variations in treatment strategies may influence patient outcomes. Therefore, the aim of the study was to assess practice variation in adjuvant chemotherapy administration in colon cancer patients. PATIENTS AND METHODS: We examined the association between patient, demographic and tumour characteristics on the odds of being treated with adjuvant chemotherapy in a random sample of adult stage II-III colon cancer patients from the Dutch National Cancer Registry (2008-2015) and assessed its association with survival. RESULTS: The study population consisted of 2,044 patients of whom 18% (79 out of 450) were high-risk stage II and 65% (645 out of 997) were stage III colon cancer and received adjuvant chemotherapy. Chemotherapy administration differed between individual hospitals (high-risk stage II: 0-39%; p=0.01; stage III: 50-78%; p=0.06). Type of hospital (teaching versus academic) and the presence of a pT4 tumour were positively associated (high-risk stage II), and bowel perforation and examined regional lymph nodes (<10) were negatively associated (stage III) with adjuvant treatment. Higher age was associated with non-administration of adjuvant chemotherapy for both stages. Tumour MMR-status assessment increased from 9% to 23% (p<0.001), but 62% of high-risk stage II and 13% of stage III patients did not undergo guideline-recommended MMR-status testing. Adjuvant chemotherapy was correlated with survival for stage III (HR=0.4; 95%CI=0.3-0.5) but not for high-risk stage II patients (HR=1.2; 95%CI=0.7-2.2). CONCLUSION: Significant practice variation in the adjuvant treatment of colon cancer on hospital level was demonstrated, predominantly in high-risk stage II patients. The implementation of MMR testing was suboptimal. We recommend continuous monitoring of treatment patterns using population-based data, which should facilitate hospital auditing and improve guideline implementation and quality of care for colon cancer patients.

Practice variation on hospital level in the systemic treatment of metastatic colorectal cancer in The Netherlands: a population-based study.

Introduction: Population-based data on the implementation of guidelines for cancer patients in daily practice are scarce, while practice variation may influence patient outcomes. Therefore, we evaluated treatment patterns and associated variables in the systemic treatment of metastatic colorectal cancer (mCRC) in the Netherlands.Material and methods: We selected a random sample of adult mCRC patients diagnosed from 2008 to 2015 from the National Cancer Registry in 20 (4 academic, 8 teaching and 8 regional) Dutch hospitals. We examined the influence of patient, demographic and tumour characteristics on the odds of being treated with systemic therapy according to the current guideline and assessed its association with survival.Results: Our study population consisted of 2222 mCRC patients of whom 1307 patients received systemic therapy for mCRC. Practice variation was most obvious in the use of bevacizumab and anti-EGFR therapy in patients with (K)RAS wild-type tumours. Administration rates did not differ between hospital types but fluctuated between individual hospitals for bevacizumab (8-92%; p < .0001) and anti-EGFR therapy (10-75%; p = .05). Bevacizumab administration was inversely correlated to higher age (OR:0.2; 95%CI: 0.1-0.3) comorbidity (OR:0.6; 95%CI: 0.5-0.8) and the presence of metachronous metastases (OR:0.5; 95%CI: 0.3-0.7), but patient characteristics did not differ between hospitals with low or high bevacizumab administration rates. The hazard ratios for exposure to bevacizumab and anti-EGFR therapy were 0.8 (95%CI: 0.7-0.9) and 0.6 (95%CI: 0.5-0.8), respectively.Discussion: We identified significant inter-hospital variation in targeted therapy administration for mCRC patients, which may affect outcome. Age and comorbidity were inversely correlated with non-administration of bevacizumab but did not explain inter-hospital practice variation. Our data suggest that practice variation is based on individual strategy of hospitals rather than guideline recommendations or patient-driven decisions. Individual hospital strategies are an additional factor that may explain the observed differences between real-life data and results obtained from clinical trials.

Implementation, participation and satisfaction rates of a web-based decision support tool for patients with metastatic colorectal cancer.

OBJECTIVE: To examine implementation and patients' and providers' participation and satisfaction of a newly developed decision support tool (DST) for patients with metastatic colorectal cancer (mCRC) in palliative setting. METHODS: Our DST consisted of a consultation sheet and web-based tailored information for mCRC treatment options. We conducted an implementation trajectory in 11 Dutch hospitals and evaluated implementation, participation and satisfaction rates. RESULTS: Implementation rates fluctuated between 3 and 72 handed out (median:23) consultation sheets per hospital with patients' login rates between 36% and 83% (median:57%). The majority of patients (68%) had (intermediate)-high participation scores. The median time spent using the DST was 38 min (IQR:18-56) and was highest for questions concerning patients' perspective (5 min). Seventy-six% of patients were (very) satisfied. The provider DST rating was 7.8 (scale 1-10) and participation ranged between 25 and 100%. Remaining implementation thresholds included providers' treatment preferences, resistance against shared decision-making and (over)confidence in shared decision-making concepts already in use. CONCLUSION: We implemented a DST with sufficient patient and oncologist satisfaction and high patient participation, but participation differed considerably between hospitals suggesting unequal adoption of our tool. PRACTICE IMPLICATIONS: Requirements for structural implementation are to overcome remaining thresholds and increase awareness for additional decision support.

The first steps in the evaluation of a "black-box" decision support tool: a protocol and feasibility study for the evaluation of Watson for Oncology.

BACKGROUND AND AIM: Medical specialists aim to provide evidence-based care based on the most recent scientific insights, but with the ongoing expansion of medical literature it seems unfeasible to remain updated. "Black-box" decision support tools such as Watson for Oncology (Watson) are gaining attention as they offer a promising opportunity to conquer this challenging issue, but it is not known if the advice given is congruent with guidelines or clinically valid in other settings. We present a protocol for the content evaluation of black-box decision support tools and a feasibility study to test the content and usability of Watson using this protocol. METHODS: The protocol consists of developing synthetic patient cases based on Dutch guidelines and expert opinion, entering the synthetic cases into Watson and Oncoguide, noting the response of each system and evaluating the result using a cross-tabulation scoring system resulting in a score range of -12 to +12. Treatment options that were not recommended according to the Dutch guideline were labeled with a "red flag" if Watson recommended it, and an "orange flag" if Watson suggested it for consideration. To test the feasibility of applying the protocol, we developed synthetic patient cases for the adjuvant treatment of stage I to stage III colon cancer based on relevant patient, clinical and tumor characteristics and followed our protocol. Additionally, for the feasibility study we also compared the recommendations from the NCCN guideline with Watson's advice, and evaluated usability by a cognitive walkthrough method. RESULTS: In total, we developed 190 synthetic patient cases (stage I: n=8; stage II: n=110; and stage III: n=72). Overall concordance scores per case for Watson versus Oncoguide ranged from a minimum score of -4 (n=6) to a maximum score of+12 (n=17) and from -4 (n=9) to +12 (n=24) for Watson versus the NCCN guidelines). In total, 69 cases (36%) were labeled with red flags, 96 cases (51%) with orange flags and 25 cases (13%) without flags. For the comparison of Watson with the NCCN guidelines, no red or orange flags were identified. CONCLUSIONS: We developed a research protocol for the evaluation of a black-box decision support tool, which proved useful and usable in testing the content and usability of Watson. Overall concordance scores ranged considerably between synthetic cases for both comparisons between Watson versus Oncoguide and Watson versus NCCN. Non-concordance is partially attributable to guideline differences between the United States and The Netherlands. This implies that further adjustments and localization are required before implementation of Watson outside the United States. RELEVANCE FOR PATIENTS: This study describes the first steps of content evaluation of a decision support tool before implementation in daily oncological patient care. The ultimate goal of the incorporation of decision support tools in daily practice is to improve personalized medicine and quality of care.

Evaluation of Guideline Adherence in Colorectal Cancer Treatment in The Netherlands: A Survey Among Medical Oncologists by the Dutch Colorectal Cancer Group.

BACKGROUND: Clinical guidelines are generated to preserve high-quality evidence-based care. Data on the implementation of guidelines into clinical practice are scarce, despite that guideline adherence prevents over- and undertreatment and correlates with survival. Therefore, we investigated guideline adherence for the systemic treatment in high-risk stage II and stage III colon cancer and metastatic colorectal cancer. PATIENTS AND METHODS: In all Dutch hospitals (n = 88) 1 medical oncologist involved in colorectal cancer care was approached to participate. An online survey was conducted regarding the local standard of care for adjuvant chemotherapy in high-risk stage II and stage III colon cancer and first-line treatment regimens in metastatic colorectal cancer. Frequency tables were provided for categorical variables and compared for differences in guideline adherence according to hospital type (academic/teaching/regional). RESULTS: The overall response rate was 70% (62 of 88). Reported guideline adherence was at least 60% of all presented settings. For high-risk stage II and stage III colon cancer, treatment strategies agreed with national guidelines in 66% and 84% of hospitals, and overtreatment patterns were identified in 28% and 13%, respectively. Targeted therapy was not routinely administered as first-line treatment in metastatic colorectal cancer (range from 63% to 71% in different settings). No differences in guideline adherence were observed among different hospital types. CONCLUSION: Guideline adherence as reported by medical oncologists in The Netherlands is suboptimal. Possible explanations include unawareness or disagreement with the guidelines, or local financial restrictions. Our results recommend additional support of guideline implementation and monitoring in clinical practice, and investigating underlying causes in case of nonadherence.

Evaluating the scientific basis of quality indicators in colorectal cancer care: A systematic review.

AIM: In colorectal cancer care, many indicators for assessment and improvement of quality of care are being used. These quality indicators serve as national and international benchmarks to compare health care on hospital and patient level. However, the scientific basis of these indicators is often unclear. Therefore, the aim of this systematic review is to examine reported quality indicators used in multidisciplinary colorectal cancer care and categorise these indicators based on scientific evidence. METHODS: We searched PubMed from 2005 to 2015 for original articles reporting on development, evaluation or validation of quality indicators in colorectal cancer care. Included articles were categorised in consensus-based, evidence-based and validation cohort studies. Extracted quality indicators were divided into structure, process and outcome indicators and grouped per discipline(s) involved. RESULTS: From 1163 studies, 41 articles were included: 12 (29%) consensus-based, 7 (17%) evidence-based and 22 (54%) validation cohort studies. In total, we identified 389 reported quality indicators: consensus-based (n = 349), evidence-based (n = 7) and validation (n = 33), respectively. Of all reported indicators, 45% (n = 186) concerned surgical items. The vast majority were process indicators (n = 315; 81%) and the remaining outcome (n = 57; 15%) or structure measurements (n = 17; 4%). Only 5 indicators were reported in the majority (≥7/12 articles) of consensus-based papers and 7 indicators were successfully validated. CONCLUSIONS: There is an abundance of reported colorectal cancer quality indicators, of which the majority are surgical, consensus-based process measures, which have not been validated in cohort studies. There is a need for international consensus on a limited evidence-based data set of validated quality indicators, with a focus on outcome indicators.