Exploring Structural Uncertainty and Impact of Health State Utility Values on Lifetime Outcomes in Diabetes Economic Simulation Models: Findings from the Ninth Mount Hood Diabetes Quality-of-Life Challenge.

BACKGROUND: Structural uncertainty can affect model-based economic simulation estimates and study conclusions. Unfortunately, unlike parameter uncertainty, relatively little is known about its magnitude of impact on life-years (LYs) and quality-adjusted life-years (QALYs) in modeling of diabetes. We leveraged the Mount Hood Diabetes Challenge Network, a biennial conference attended by international diabetes modeling groups, to assess structural uncertainty in simulating QALYs in type 2 diabetes simulation models. METHODS: Eleven type 2 diabetes simulation modeling groups participated in the 9th Mount Hood Diabetes Challenge. Modeling groups simulated 5 diabetes-related intervention profiles using predefined baseline characteristics and a standard utility value set for diabetes-related complications. LYs and QALYs were reported. Simulations were repeated using lower and upper limits of the 95% confidence intervals of utility inputs. Changes in LYs and QALYs from tested interventions were compared across models. Additional analyses were conducted postchallenge to investigate drivers of cross-model differences. RESULTS: Substantial cross-model variability in incremental LYs and QALYs was observed, particularly for HbA1c and body mass index (BMI) intervention profiles. For a 0.5%-point permanent HbA1c reduction, LY gains ranged from 0.050 to 0.750. For a 1-unit permanent BMI reduction, incremental QALYs varied from a small decrease in QALYs (-0.024) to an increase of 0.203. Changes in utility values of health states had a much smaller impact (to the hundredth of a decimal place) on incremental QALYs. Microsimulation models were found to generate a mean of 3.41 more LYs than cohort simulation models (P = 0.049). CONCLUSIONS: Variations in utility values contribute to a lesser extent than uncertainty captured as structural uncertainty. These findings reinforce the importance of assessing structural uncertainty thoroughly because the choice of model (or models) can influence study results, which can serve as evidence for resource allocation decisions.HighlightsThe findings indicate substantial cross-model variability in QALY predictions for a standardized set of simulation scenarios and is considerably larger than within model variability to alternative health state utility values (e.g., lower and upper limits of the 95% confidence intervals of utility inputs).There is a need to understand and assess structural uncertainty, as the choice of model to inform resource allocation decisions can matter more than the choice of health state utility values.

Evaluating the Ability of Economic Models of Diabetes to Simulate New Cardiovascular Outcomes Trials: A Report on the Ninth Mount Hood Diabetes Challenge.

OBJECTIVES: The cardiovascular outcomes challenge examined the predictive accuracy of 10 diabetes models in estimating hard outcomes in 2 recent cardiovascular outcomes trials (CVOTs) and whether recalibration can be used to improve replication. METHODS: Participating groups were asked to reproduce the results of the Empagliflozin Cardiovascular Outcome Event Trial in Type 2 Diabetes Mellitus Patients (EMPA-REG OUTCOME) and the Canagliflozin Cardiovascular Assessment Study (CANVAS) Program. Calibration was performed and additional analyses assessed model ability to replicate absolute event rates, hazard ratios (HRs), and the generalizability of calibration across CVOTs within a drug class. RESULTS: Ten groups submitted results. Models underestimated treatment effects (ie, HRs) using uncalibrated models for both trials. Calibration to the placebo arm of EMPA-REG OUTCOME greatly improved the prediction of event rates in the placebo, but less so in the active comparator arm. Calibrating to both arms of EMPA-REG OUTCOME individually enabled replication of the observed outcomes. Using EMPA-REG OUTCOME-calibrated models to predict CANVAS Program outcomes was an improvement over uncalibrated models but failed to capture treatment effects adequately. Applying canagliflozin HRs directly provided the best fit. CONCLUSIONS: The Ninth Mount Hood Diabetes Challenge demonstrated that commonly used risk equations were generally unable to capture recent CVOT treatment effects but that calibration of the risk equations can improve predictive accuracy. Although calibration serves as a practical approach to improve predictive accuracy for CVOT outcomes, it does not extrapolate generally to other settings, time horizons, and comparators. New methods and/or new risk equations for capturing these CV benefits are needed.

Computer Modeling of Diabetes and Its Transparency: A Report on the Eighth Mount Hood Challenge.

OBJECTIVES: The Eighth Mount Hood Challenge (held in St. Gallen, Switzerland, in September 2016) evaluated the transparency of model input documentation from two published health economics studies and developed guidelines for improving transparency in the reporting of input data underlying model-based economic analyses in diabetes. METHODS: Participating modeling groups were asked to reproduce the results of two published studies using the input data described in those articles. Gaps in input data were filled with assumptions reported by the modeling groups. Goodness of fit between the results reported in the target studies and the groups' replicated outputs was evaluated using the slope of linear regression line and the coefficient of determination (R2). After a general discussion of the results, a diabetes-specific checklist for the transparency of model input was developed. RESULTS: Seven groups participated in the transparency challenge. The reporting of key model input parameters in the two studies, including the baseline characteristics of simulated patients, treatment effect and treatment intensification threshold assumptions, treatment effect evolution, prediction of complications and costs data, was inadequately transparent (and often missing altogether). Not surprisingly, goodness of fit was better for the study that reported its input data with more transparency. To improve the transparency in diabetes modeling, the Diabetes Modeling Input Checklist listing the minimal input data required for reproducibility in most diabetes modeling applications was developed. CONCLUSIONS: Transparency of diabetes model inputs is important to the reproducibility and credibility of simulation results. In the Eighth Mount Hood Challenge, the Diabetes Modeling Input Checklist was developed with the goal of improving the transparency of input data reporting and reproducibility of diabetes simulation model results.

Evaluation of a patient self-directed mealtime insulin titration algorithm: a US payer perspective.

Objective To model the potential economic impact of implementing the AUTONOMY once daily (Q1D) patient self-titration mealtime insulin dosing algorithm vs standard of care (SOC) among a population of patients with Type 2 diabetes living in the US. Methods Three validated models were used in this analysis: The Treatment Transitions Model (TTM) was used to generate the primary results, while both the Archimedes (AM) and IMS Core Diabetes Models (IMS) were used to test the veracity of the primary results produced by TTM. Models used data from a 'real world' representative sample of patients (2012 US National Health and Nutrition Examination Survey) that matched the characteristics of US patients enrolled in the randomized controlled trial 'AUTONOMY' cohort. The base-case time horizon was 10 years. Results The modeling results from TTM demonstrated that total costs in the base-case were reduced by $1732, with savings predicted to occur as early as year 1. Results from the three models were consistent, showing a reduction in total costs for all sensitivity analyses. Limitations Data from short-term clinical trials were used to develop long-term projections. The nature of such extrapolation leads to increased uncertainty. Conclusion The results from all three models indicate that the AUTONOMY Q1D algorithm has the potential to abate total costs as early as the first year.

The evaluation of clinical and cost outcomes associated with earlier initiation of insulin in patients with type 2 diabetes mellitus.

BACKGROUND: The treatment for patients with type 2 diabetes mellitus (T2DM) follows a stepwise progression. As a treatment loses its effectiveness, it is typically replaced with a more complex and frequently more costly treatment. Eventually this progression leads to the use of basal insulin typically with concomitant treatments (e.g., metformin, a GLP-1 RA [glucagon-like peptide-1 receptor agonist], a TZD [thiazolidinedione] or a DPP-4i [dipeptidyl peptidase 4 inhibitor]) and, ultimately, to basal-bolus insulin in some forms. As the cost of oral antidiabetics (OADs) and noninsulin injectables have approached, and in some cases exceeded, the cost of insulin, we reexamined the placement of insulin in T2DM treatment progression. Our hypothesis was that earlier use of insulin produces clinical and cost benefits due to its superior efficacy and treatment scalability at an acceptable cost when considered over a 5-year period. OBJECTIVES: To (a) estimate clinical and payer cost outcomes of initiating insulin treatment for patients with T2DM earlier in their treatment progression and (b) estimate clinical and payer cost outcomes resulting from delays in escalating treatment for T2DM when indicated by patient hemoglobin A1c levels. METHODS: We developed a Monte Carlo microsimulation model to estimate patients reaching target A1c, diabetes-related complications, mortality, and associated costs under various treatment strategies for newly diagnosed patients with T2DM. Treatment efficacies were modeled from results of randomized clinical trials, including the time and rate of A1c drift. A typical treatment progression was selected based on the American Diabetes Association and the European Association for the Study of Diabetes guidelines as the standard of care (SOC). Two treatment approaches were evaluated: two-stage insulin (basal plus antidiabetics followed by biphasic plus metformin) and single-stage insulin (biphasic plus metformin). For each approach, we analyzed multiple strategies. For each analysis, treatment steps were sequentially and cumulatively removed from the SOC until only the insulin steps remained. Delays in escalating treatment were evaluated by increasing the minimum time on a treatment within each strategy. The analysis time frame was 5 years. RESULTS: Relative to SOC, the two-stage insulin approach resulted in 0.10% to 1.79% more patients achieving target A1c (<7.0%), at incremental costs of $95 to $3,267. (The ranges are due to the different strategies within the approach.) With the single-stage approach, 0.50% to 2.63% more patients achieved the target A1c compared with SOC at an incremental cost of -$1,642 to $1,177. Major diabetes-related complications were reduced by 0.38% to 17.46% using the two-stage approach and 0.72% to 25.92% using the single-stage approach. Severe hypoglycemia increased by 17.97% to 60.43% using the two-stage approach and 6.44% to 68.87% using the single-stage approach. In the base case scenario, the minimum time on a specific treatment was 3 months. When the minimum time on each treatment was increased to 12 months (i.e., delayed), patients reaching A1c targets were reduced by 57%, complications increased by 13% to 76%, and mortality increased by 8% over 5 years when compared with the base case for the SOC. However, severe hypoglycemic events were reduced by 83%. CONCLUSIONS: As insulin was advanced earlier in therapy in the two-stage and single-stage approaches, patients reaching their A1c targets increased, severe hypoglycemic events increased, and diabetes-related complications and mortality decreased. Cost savings were estimated for 3 (of 4) strategies in the single-stage approach. Delays in treatment escalation substantially reduced patients reaching target A1c levels and increased the occurrence of major nonhypoglycemic diabetic complications. With the exception of substantial increases in severe hypoglycemic events, earlier use of insulin mitigates the clinical consequences of these delays.

Development, validation, and application of a microsimulation model to predict stroke and mortality in medically managed asymptomatic patients with significant carotid artery stenosis.

OBJECTIVE: To develop a model to predict stroke-free survival and mortality over a multiyear time frame for a trial-excluded population of medically managed asymptomatic patients with significant carotid artery stenosis. METHODS: We calibrated, validated, and applied a Monte Carlo microsimulation model. For calibration we adjusted general-population mortality and stroke risks to capture these risks specific to asymptomatic carotid stenosis patients. For validation, we compared model-predicted and actual stroke-free survival curves and stroke counts from a population of comparable patients. For application, the validated model predicted stroke-free survival for a hypothetical medically managed arm of a recent single-arm carotid revascularization trial. RESULTS: For each month in the 60-month time frame, the model-predicted and actual calibration trial stroke-free survival curves were not statistically different (P > 0.62). In validation, the calibrated model's stroke-free survival curvematched the actual curve from an independent population; beyond 24 months, the model-predicted and actual curves were not statistically different (P > 0.32). We also compared model-predicted and actual number of strokes from the independent trial. The model predicted 187.25 strokes (95% confidence interval 161.49-213.01), while the actual number was 171.6, within 1.22 standard deviations of the simulated mean. CONCLUSIONS: Given the absence of medically managed populations in recent carotid stenosis trials, our model can estimate stroke-free survival and mortality data for these patients. The model may also estimate the effectiveness of novel medical and procedural therapies for stroke prevention. These effectiveness estimates can inform the development of policies, guidelines, or cost-effectiveness analyses when only single-arm trial data exist.