Calibration and Validation of the Colorectal Cancer and Adenoma Incidence and Mortality (CRC-AIM) Microsimulation Model Using Deep Neural Networks.

OBJECTIVES: Machine learning (ML)-based emulators improve the calibration of decision-analytical models, but their performance in complex microsimulation models is yet to be determined. METHODS: We demonstrated the use of an ML-based emulator with the Colorectal Cancer (CRC)-Adenoma Incidence and Mortality (CRC-AIM) model, which includes 23 unknown natural history input parameters to replicate the CRC epidemiology in the United States. We first generated 15,000 input combinations and ran the CRC-AIM model to evaluate CRC incidence, adenoma size distribution, and the percentage of small adenoma detected by colonoscopy. We then used this data set to train several ML algorithms, including deep neural network (DNN), random forest, and several gradient boosting variants (i.e., XGBoost, LightGBM, CatBoost) and compared their performance. We evaluated 10 million potential input combinations using the selected emulator and examined input combinations that best estimated observed calibration targets. Furthermore, we cross-validated outcomes generated by the CRC-AIM model with those made by CISNET models. The calibrated CRC-AIM model was externally validated using the United Kingdom Flexible Sigmoidoscopy Screening Trial (UKFSST). RESULTS: The DNN with proper preprocessing outperformed other tested ML algorithms and successfully predicted all 8 outcomes for different input combinations. It took 473 s for the trained DNN to predict outcomes for 10 million inputs, which would have required 190 CPU-years without our DNN. The overall calibration process took 104 CPU-days, which included building the data set, training, selecting, and hyperparameter tuning of the ML algorithms. While 7 input combinations had acceptable fit to the targets, a combination that best fits all outcomes was selected as the best vector. Almost all of the predictions made by the best vector laid within those from the CISNET models, demonstrating CRC-AIM's cross-model validity. Similarly, CRC-AIM accurately predicted the hazard ratios of CRC incidence and mortality as reported by UKFSST, demonstrating its external validity. Examination of the impact of calibration targets suggested that the selection of the calibration target had a substantial impact on model outcomes in terms of life-year gains with screening. CONCLUSIONS: Emulators such as a DNN that is meticulously selected and trained can substantially reduce the computational burden of calibrating complex microsimulation models. HIGHLIGHTS: Calibrating a microsimulation model, a process to find unobservable parameters so that the model fits observed data, is computationally complex.We used a deep neural network model, a popular machine learning algorithm, to calibrate the Colorectal Cancer Adenoma Incidence and Mortality (CRC-AIM) model.We demonstrated that our approach provides an efficient and accurate method to significantly speed up calibration in microsimulation models.The calibration process successfully provided cross-model validation of CRC-AIM against 3 established CISNET models and also externally validated against a randomized controlled trial.

Life-years gained resulting from screening colonoscopy compared with follow-up colonoscopy after a positive stool-based colorectal screening test.

Screening colonoscopies for colorectal cancer (CRC) are typically covered without patient cost-sharing, whereas follow-up colonoscopies for positive stool-based screening tests often incur patient costs. The objective of this analysis was to estimate and compare the life-years gained (LYG) per average-risk screening colonoscopy and follow-up colonoscopy after a positive stool-based test to better inform CRC coverage policy and reimbursement decisions. CRC outcomes from screening and follow-up colonoscopies versus no screening were estimated using CRC-AIM in a simulated population of average-risk individuals screened between ages 45-75 years. The LYG/colonoscopy per 1000 individuals was 0.09 for screening colonoscopy and 0.29 for follow-up colonoscopy. 0.01 and 0.04 CRC cases and 0.01 and 0.02 CRC deaths were averted per screening and follow-up colonoscopies, respectively. Coverage policies should be revised to encourage individuals to complete recommended screening processes.

Impact of Patient Adherence to Stool-Based Colorectal Cancer Screening and Colonoscopy Following a Positive Test on Clinical Outcomes.

Colorectal cancer-screening models commonly assume 100% adherence, which is inconsistent with real-world experience. The influence of adherence to initial stool-based screening [fecal immunochemical test (FIT), multitarget stool DNA (mt-sDNA)] and follow-up colonoscopy (after a positive stool test) on colorectal cancer outcomes was modeled using the Colorectal Cancer and Adenoma Incidence and Mortality Microsimulation Model. Average-risk individuals without diagnosed colorectal cancer at age 40 undergoing annual FIT or triennial mt-sDNA screening from ages 50 to 75 were simulated. Primary analyses incorporated published mt-sDNA (71%) or FIT (43%) screening adherence, with follow-up colonoscopy adherence ranging from 40% to 100%. Secondary analyses simulated 100% adherence for stool-based screening and colonoscopy follow-up (S1), published adherence for stool-based screening with 100% adherence to colonoscopy follow-up (S2), and published adherence for both stool-based screening and colonoscopy follow-up after positive mt-sDNA (73%) or FIT (47%; S3). Outcomes were life-years gained (LYG) and colorectal cancer incidence and mortality reductions (per 1,000 individuals) versus no screening. Adherence to colonoscopy follow-up after FIT had to be 4%-13% higher than mt-sDNA to reach equivalent LYG. The theoretical S1 favored FIT versus mt-sDNA (LYG 316 vs. 297; colorectal cancer incidence reduction 68% vs. 64%; colorectal cancer mortality reduction 76% vs. 72%). The more realistic S2 and S3 favored mt-sDNA versus FIT (S2: LYG 284 vs. 245, colorectal cancer incidence reduction 61% vs. 50%, colorectal cancer mortality reduction 69% vs. 59%; S3: LYG 203 vs. 113, colorectal cancer incidence reduction 43% vs. 23%, colorectal cancer mortality reduction 49% vs. 27%, respectively). Incorporating realistic adherence rates for colorectal cancer screening influences modeled outcomes and should be considered when assessing comparative effectiveness. PREVENTION RELEVANCE: Adherence rates for initial colorectal cancer screening by FIT or mt-sDNA and for colonoscopy follow-up of a positive initial test influence the comparative effectiveness of these screening strategies. Using adherence rates based on published data for stool-based testing and colonoscopy follow-up yielded superior outcomes with an mt-sDNA versus FIT-screening strategy.

Lowering the colorectal cancer screening age improves predicted outcomes in a microsimulation model.

AIMS: While most guidelines still recommend colorectal cancer (CRC) screening initiation at age 50 years in average-risk individuals, guideline-creating bodies are starting to lower the recommended age of initiation to 45 years to mitigate the trend of increasing CRC rates in younger populations. Using CRC-AIM, we modeled the impact of lowering the CRC screening initiation age, incorporating theoretical and reported adherence rates, for triennial multi-target stool DNA (mt-sDNA) or annual fecal immunochemical test (FIT) screening. METHODS AND MATERIALS: Screening strategies were simulated for individuals without CRC at age 40 and screened from ages 50 to 75 or 45 to 75 years. Outcomes included CRC incidence, CRC mortality, and life-years gained (LYG) per 1000 individuals screened (compared with no screening). Models used theoretically perfect (100%) and previously reported (71% mt-sDNA; 43% FIT) adherence rates. RESULTS: With perfect adherence, mt-sDNA and FIT resulted in 22.2 and 23.4 more predicted LYG, respectively, with screening initiation at age 45 versus 50 years; reported adherence resulted in 23.9 and 24.4 more LYG, respectively. With perfect adherence, screening initiation at age 45 versus 50 years resulted in 26.1 and 28.6 CRC cases, respectively, with mt-sDNA and 22.8 and 25.5 cases with FIT; with reported real-world adherence there were 28.5 and 31.2 cases, respectively, with mt-sDNA and 37.1 and 40.2 cases with FIT. Similar patterns were observed for CRC deaths. With screening initiation at age 45 and reported adherence, mt-sDNA averted 8.6 more CRC cases and 3.3 more deaths per 1000 individuals than FIT. CONCLUSIONS: Estimated CRC screening outcomes improved by lowering the initiation age from 50 to 45 years. Incorporating reported adherence rates yields greater benefits from triennial mt-sDNA versus annual FIT screening.

Impact of screening and follow-up colonoscopy adenoma sensitivity on colorectal cancer screening outcomes in the CRC-AIM microsimulation model.

BACKGROUND: Real-world data for patients with positive colorectal cancer (CRC) screening stool-tests demonstrate that adenoma detection rates are lower when endoscopists are blinded to the stool-test results. This suggests adenoma sensitivity may be lower for screening colonoscopy than for follow-up to a known positive stool-based test. Previous CRC microsimulation models assume identical sensitivities between screening and follow-up colonoscopies after positive stool-tests. The Colorectal Cancer and Adenoma Incidence and Mortality Microsimulation Model (CRC-AIM) was used to explore the impact on screening outcomes when assuming different adenoma sensitivity between screening and combined follow-up/surveillance colonoscopies. METHODS: Modeled screening strategies included colonoscopy every 10 years, triennial multitarget stool DNA (mt-sDNA), or annual fecal immunochemical test (FIT) from 50 to 75 years. Outcomes were reported per 1000 individuals without diagnosed CRC at age 40. Base-case adenoma sensitivity values were identical for screening and follow-up/surveillance colonoscopies. Ranges of adenoma sensitivity values for colonoscopy performance were developed using different slopes of odds ratio adjustments and were designated as small, medium, or large impact scenarios. RESULTS: As the differences in adenoma sensitivity for screening versus follow-up/surveillance colonoscopies became greater, life-years gained (LYG) and reductions in CRC-related incidence and mortality versus no screening increased for mt-sDNA and FIT and decreased for screening colonoscopy. The LYG relative to screening colonoscopy reached >90% with FIT in the base-case scenario and with mt-sDNA in a "medium impact" scenario. CONCLUSIONS: Assuming identical adenoma sensitivities for screening and follow-up/surveillance colonoscopies underestimate the potential benefits of stool-based screening strategies.

Estimating the impact of differential adherence on the comparative effectiveness of stool-based colorectal cancer screening using the CRC-AIM microsimulation model.

BACKGROUND: Real-world adherence to colorectal cancer (CRC) screening strategies is imperfect. The CRC-AIM microsimulation model was used to estimate the impact of imperfect adherence on the relative benefits and burdens of guideline-endorsed, stool-based screening strategies. METHODS: Predicted outcomes of multi-target stool DNA (mt-sDNA), fecal immunochemical tests (FIT), and high-sensitivity guaiac-based fecal occult blood tests (HSgFOBT) were simulated for 40-year-olds free of diagnosed CRC. For robustness, imperfect adherence was incorporated in multiple ways and with extensive sensitivity analysis. Analysis 1 assumed adherence from 0%-100%, in 10% increments. Analysis 2 longitudinally applied real-world first-round differential adherence rates (base-case imperfect rates = 40% annual FIT vs 34% annual HSgFOBT vs 70% triennial mt-sDNA). Analysis 3 randomly assigned individuals to receive 1, 5, or 9 lifetime (9 = 100% adherence) mt-sDNA tests and 1, 5, or 9 to 26 (26 = 100% adherence) FIT tests. Outcomes are reported per 1000 individuals compared with no screening. RESULTS: Each screening strategy decreased CRC incidence and mortality versus no screening. In individuals screened between ages 50-75 and adherence ranging from 10%a-100%, the life-years gained (LYG) for triennial mt-sDNA ranged from 133.1-300.0, for annual FIT from 96.3-318.1, and for annual HSgFOBT from 99.8-320.6. At base-case imperfect adherence rates, mt-sDNA resulted in 19.1% more LYG versus FIT, 25.4% more LYG versus HSgFOBT, and generally had preferable efficiency ratios while offering the most LYG. Completion of at least 21 FIT tests is needed to reach approximately the same LYG achieved with 9 mt-sDNA tests. CONCLUSIONS: Adherence assumptions affect the conclusions of CRC screening microsimulations that are used to inform CRC screening guidelines. LYG from FIT and HSgFOBT are more sensitive to changes in adherence assumptions than mt-sDNA because they require more tests be completed for equivalent benefit. At imperfect adherence rates, mt-sDNA provides more LYG than FIT or HSgFOBT at an acceptable tradeoff in screening burden.