General-Purpose Methods for Simulating Survival Data for Expected Value of Sample Information Calculations.

BACKGROUND: Expected value of sample information (EVSI) quantifies the expected value to a decision maker of reducing uncertainty by collecting additional data. EVSI calculations require simulating plausible data sets, typically achieved by evaluating quantile functions at random uniform numbers using standard inverse transform sampling (ITS). This is straightforward when closed-form expressions for the quantile function are available, such as for standard parametric survival models, but these are often unavailable when assuming treatment effect waning and for flexible survival models. In these circumstances, the standard ITS method could be implemented by numerically evaluating the quantile functions at each iteration in a probabilistic analysis, but this greatly increases the computational burden. Thus, our study aims to develop general-purpose methods that standardize and reduce the computational burden of the EVSI data-simulation step for survival data. METHODS: We developed a discrete sampling method and an interpolated ITS method for simulating survival data from a probabilistic sample of survival probabilities over discrete time units. We compared the general-purpose and standard ITS methods using an illustrative partitioned survival model with and without adjustment for treatment effect waning. RESULTS: The discrete sampling and interpolated ITS methods agree closely with the standard ITS method, with the added benefit of a greatly reduced computational cost in the scenario with adjustment for treatment effect waning. CONCLUSIONS: We present general-purpose methods for simulating survival data from a probabilistic sample of survival probabilities that greatly reduce the computational burden of the EVSI data-simulation step when we assume treatment effect waning or use flexible survival models. The implementation of our data-simulation methods is identical across all possible survival models and can easily be automated from standard probabilistic decision analyses. HIGHLIGHTS: Expected value of sample information (EVSI) quantifies the expected value to a decision maker of reducing uncertainty through a given data collection exercise, such as a randomized clinical trial. In this article, we address the problem of computing EVSI when we assume treatment effect waning or use flexible survival models, by developing general-purpose methods that standardize and reduce the computational burden of the EVSI data-generation step for survival data.We developed 2 methods for simulating survival data from a probabilistic sample of survival probabilities over discrete time units, a discrete sampling method and an interpolated inverse transform sampling method, which can be combined with a recently proposed nonparametric EVSI method to accurately estimate EVSI for collecting survival data.Our general-purpose data-simulation methods greatly reduce the computational burden of the EVSI data-simulation step when we assume treatment effect waning or use flexible survival models. The implementation of our data-simulation methods is identical across all possible survival models and can therefore easily be automated from standard probabilistic decision analyses.

An Efficient Method for Computing Expected Value of Sample Information for Survival Data from an Ongoing Trial.

BACKGROUND: Decisions about new health technologies are increasingly being made while trials are still in an early stage, which may result in substantial uncertainty around key decision drivers such as estimates of life expectancy and time to disease progression. Additional data collection can reduce uncertainty, and its value can be quantified by computing the expected value of sample information (EVSI), which has typically been described in the context of designing a future trial. In this article, we develop new methods for computing the EVSI of extending an existing trial's follow-up, first for an assumed survival model and then extending to capture uncertainty about the true survival model. METHODS: We developed a nested Markov Chain Monte Carlo procedure and a nonparametric regression-based method. We compared the methods by computing single-model and model-averaged EVSI for collecting additional follow-up data in 2 synthetic case studies. RESULTS: There was good agreement between the 2 methods. The regression-based method was fast and straightforward to implement, and scales easily to include any number of candidate survival models in the model uncertainty case. The nested Monte Carlo procedure, on the other hand, was extremely computationally demanding when we included model uncertainty. CONCLUSIONS: We present a straightforward regression-based method for computing the EVSI of extending an existing trial's follow-up, both where a single known survival model is assumed and where we are uncertain about the true survival model. EVSI for ongoing trials can help decision makers determine whether early patient access to a new technology can be justified on the basis of the current evidence or whether more mature evidence is needed. HIGHLIGHTS: Decisions about new health technologies are increasingly being made while trials are still in an early stage, which may result in substantial uncertainty around key decision drivers such as estimates of life-expectancy and time to disease progression. Additional data collection can reduce uncertainty, and its value can be quantified by computing the expected value of sample information (EVSI), which has typically been described in the context of designing a future trial.In this article, we have developed new methods for computing the EVSI of extending a trial's follow-up, both where a single known survival model is assumed and where we are uncertain about the true survival model. We extend a previously described nonparametric regression-based method for computing EVSI, which we demonstrate in synthetic case studies is fast, straightforward to implement, and scales easily to include any number of candidate survival models in the EVSI calculations.The EVSI methods that we present in this article can quantify the need for collecting additional follow-up data before making an adoption decision given any decision-making context.

Dangerous omissions: the consequences of ignoring decision uncertainty.

Institutions with the responsibility for making adoption (reimbursement) decisions in health care often lack the remit to demand or commission further research: adoption decisions are their only policy instrument. The decision to adopt a technology also influences the prospects of acquiring further evidence because the incentives to conduct research are reduced and the ethical basis of further clinical trials maybe undermined. In these circumstances the decision maker must consider whether the benefits of immediate access to a technology exceeds the value of the evidence which maybe forgone for future patients. We outline how these expected opportunity losses can be established from the perspective of a societal decision maker with and without the remit to commission research, and demonstrate how these considerations change the appropriate decision rules in cost-effectiveness analysis. Importantly, we identify those circumstances in which the approval of a technology that is expected to be cost-effective should be withheld, i.e. when an 'only in research' recommendation should be made. We demonstrate that a sufficient condition for immediate adoption of a technology can provide incentives for manufacturers to reduce the price or provide additional supporting evidence. However, decisions based solely on expected net benefit provide no such incentives, may undermine the evidence base for future clinical practice and reduce expected net health benefits for the patient population.

Using value of information analysis to prioritise health research: some lessons from recent UK experience.

Decisions to adopt, reimburse or issue guidance on the use of health technologies are increasingly being informed by explicit cost-effectiveness analyses of the alternative interventions. Healthcare systems also invest heavily in research and development to support these decisions. However, the increasing transparency of adoption and reimbursement decisions, based on formal analysis, contrasts sharply with research prioritisation and commissioning. This is despite the fact that formal measures of the value of evidence generated by research are readily available. The results of two recent opportunities to apply value of information analysis to directly inform policy decisions about research priorities in the UK are presented. These include a pilot study for the UK National Co-ordinating Centre for Health Technology Assessment (NCCHTA) and a pilot study for the National Institute for Health and Clinical Excellence (NICE). We demonstrate how these results can be used to address a series of policy questions, including: is further research required to support the use of a technology and, if so, what type of research would be most valuable? We also show how the results can be used to address other questions such as, which patient subgroups should be included in subsequent research, which comparators and endpoints should be included, and what length of follow up would be most valuable.