CHALLENGES AND METHODOLOGIES IN USING PROGRESSION FREE SURVIVAL AS A SURROGATE FOR OVERALL SURVIVAL IN ONCOLOGY.

OBJECTIVES: A primary outcome in oncology trials is overall survival (OS). However, to estimate OS accurately requires a sufficient number of patients to have died, which may take a long time. If an alternative end point is sufficiently highly correlated with OS, it can be used as a surrogate. Progression-free survival (PFS) is the surrogate most often used in oncology, but does not always satisfy the correlation conditions for surrogacy. We analyze the methodologies used when extrapolating from PFS to OS. METHODS: Davis et al. previously reviewed the use of surrogate end points in oncology, using papers published between 2001 and 2011. We extend this, reviewing papers published between 2012 and 2016. We also examine the reporting of statistical methods to assess the strength of surrogacy. RESULTS: The findings from 2012 to 2016 do not differ substantially from those of 2001 to 2011: the same factors are shown to affect the relationship between PFS and OS. The proportion of papers reporting individual patient data (IPD), strongly recommended for full assessment of surrogacy, remains low: 33 percent. A wide range of methods has been used to determine the appropriateness of surrogates. While usually adhering to reporting standards, the standard of scholarship appears sometimes to be questionable and the reporting of results often haphazard. CONCLUSIONS: Standards of analysis and reporting PFS to OS surrogate studies should be improved by increasing the rigor of statistical reporting and by agreeing to a minimum set of reporting guidelines. Moreover, the use of IPD to assess surrogacy should increase.

The cost-effectiveness of seven behavioural interventions to prevent drug misuse in vulnerable populations.

BACKGROUND: The National Institute for Health and Care Excellence (NICE) developed a guideline on drug misuse prevention in vulnerable populations. Part of the guideline development process involved evaluating cost-effectiveness and determining which interventions represented good value for money. METHODS: Economic models were developed for seven interventions which aimed to prevent drug use in vulnerable populations. The models compared the costs (to the health and crime sectors) and health benefits (in quality-adjusted life years (QALYs)) of each intervention and its comparator. Sensitivity analysis explored the uncertainty associated with the cost of each intervention and duration of its effect. RESULTS: The reduction in drug use for each intervention partly offset the costs of the intervention, and improved health outcomes (QALYs). However, with high intervention costs and low QALY gains, none of the interventions were estimated to be cost-effective in the base case. Sensitivity analysis found that some of the interventions could be cost-effective if they could be delivered at a lower cost, or if the effect could be sustained for more than two years. CONCLUSIONS: For drug misuse prevention to be prioritised by funders, the consequences of drug misuse need to be understood, and interventions need to be shown to be effective and cost-effective. Quantifying the wider harms of drug misuse and wider benefits of prevention interventions poses challenges in evaluating the cost-effectiveness of drug misuse prevention interventions. A greater understanding of the consequences of drug misuse and causal factors could facilitate development of cost-effective interventions to prevent drug misuse.

The cost-effectiveness of public health interventions examined by NICE from 2011 to 2016.

Background: A review of economic evaluations of public health interventions assessed by NICE between 2005 and 2010 found 85% were cost-effective. Owen et al. (The cost-effectiveness of public health interventions. J Public Health 2012;34(1):37-45). With significant pressure on budgets the role of economics in securing funding remains important. This study updates the earlier analysis. Methods: Economic evaluations carried out between 2011 and 2016 were categorized: cost-utility analysis (CUA), cost-effectiveness analysis (CEA), cost-benefit analysis (CBA) and cost-consequences analysis (CCA). Cost-effectiveness estimates were analysed and compared with Owen et al. (The cost-effectiveness of public health interventions. J Public Health 2012;34(1):37-45). Results: Of 43 guidelines examined, 23 used CUA for specific interventions yielding 138 base-case incremental cost-effectiveness ratio (ICER) estimates, 11 used CUA for a threshold or 'what if' analysis, 1 used CEA, 3 used CCA, 1 used CBA and CUA and 1 used CEA and CUA, 5 did not require economic modelling. Compared with the earlier period, the median ICER for the 138 estimates was substantially higher (£7843 versus £1053) and there was greater variability (a higher proportion in the later period was cost-saving, but a higher proportion was also over £20 000 per quality adjusted life year). Conclusions: Nearly two-thirds (63%) of public health interventions assessed were cost-effective. However, increased variability in estimates highlights the importance of assessing cost-effectiveness to ensure good use of scarce resources.

A cost-effectiveness analysis of condom distribution programmes for the prevention of sexually transmitted infections in England.

BACKGROUND: Prevention of sexually transmitted infection (STI) incidence in England is a high priority, particularly among young people, men who have sex with men (MSM) and black ethnic minorities. An economic evaluation of condom distribution programmes (CDPs) to reduce STI transmission is presented. METHODS: An economic model using a Bernoulli process estimated the number of people acquiring an STI as a function of its prevalence, transmission rate, condom use, condom failure rate and number of sexual contacts. Models were developed for young people (13-24 years), black ethnic minorities, MSM and the general English population. Effectiveness evidence came from a recent systematic review. For young people, a CDP was modelled (relative risk for condom use=1.23), along with an exploratory analysis of the impact on unintended pregnancies. For other populations, threshold analyses were used to identify the combination of costs and effect size required to make a programme cost-effective. RESULTS: The base case predicted that CDP for all young people in England could avert 5123 STI cases per annum, with an incremental cost-effectiveness ratio of £17 411. In addition, it could avert 118 pregnancies and 82 abortions and save £333 000 in associated costs. Schemes for black ethnic minorities and MSM could also be cost-effective even with relatively high costs and small effect sizes. CONCLUSION: CDPs for young people are likely to be cost-effective or cost-saving. CDPs for other high-risk populations may also be cost-effective if they can increase condom use, since high HIV prevalence in these groups imposes a considerable health and cost burden.

Designing and Undertaking a Health Economics Study of Digital Health Interventions.

This paper introduces and discusses key issues in the economic evaluation of digital health interventions. The purpose is to stimulate debate so that existing economic techniques may be refined or new methods developed. The paper does not seek to provide definitive guidance on appropriate methods of economic analysis for digital health interventions. This paper describes existing guides and analytic frameworks that have been suggested for the economic evaluation of healthcare interventions. Using selected examples of digital health interventions, it assesses how well existing guides and frameworks align to digital health interventions. It shows that digital health interventions may be best characterized as complex interventions in complex systems. Key features of complexity relate to intervention complexity, outcome complexity, and causal pathway complexity, with much of this driven by iterative intervention development over time and uncertainty regarding likely reach of the interventions among the relevant population. These characteristics imply that more-complex methods of economic evaluation are likely to be better able to capture fully the impact of the intervention on costs and benefits over the appropriate time horizon. This complexity includes wider measurement of costs and benefits, and a modeling framework that is able to capture dynamic interactions among the intervention, the population of interest, and the environment. The authors recommend that future research should develop and apply more-flexible modeling techniques to allow better prediction of the interdependency between interventions and important environmental influences.

Unrelated Future Costs and Unrelated Future Benefits: Reflections on NICE Guide to the Methods of Technology Appraisal.

In this editorial, we consider the vexing issue of 'unrelated future costs' (for example, the costs of caring for people with dementia or kidney failure after preventing their deaths from a heart attack). The National Institute of Health and Care Excellence (NICE) guidance is not to take such costs into account in technology appraisals. However, standard appraisal practice involves modelling the benefits of those unrelated technologies. We argue that there is a sound principled reason for including both the costs and benefits of unrelated care. Changing this practice would have material consequences for decisions about reimbursing particular technologies, and we urge future research to understand this better. Copyright © 2016 John Wiley & Sons, Ltd.

The Precautionary Principle, Evidence-Based Medicine, and Decision Theory in Public Health Evaluation.

The precautionary principle (PP) has been used in the evaluation of the effectiveness and/or cost-effectiveness of interventions designed to prevent future harms in a range of activities, particularly in the area of the environment. Here, we provide details of circumstances under which the PP can be applied to the topic of harm reduction in Public Health. The definition of PP that we use says that the PP reverses the onus of proof of effectiveness between an intervention and its comparator when the intervention has been designed to reduce harm. We first describe the two frameworks used for health-care evaluation: evidence-based medicine (EBM) and decision theory (DT). EBM is usually used in treatment effectiveness evaluation, while either EBM or DT may be used in evaluating the effectiveness of the prevention of illness. For cost-effectiveness, DT is always used. The expectation in Public Health is that interventions employed to reduce harm will not actually increase harm, where "harm" in this context does not include opportunity cost. That implies that an intervention's effectiveness can often be assumed. Attention should therefore focus on its cost-effectiveness. This view is consistent with the conclusions of DT. It is also very close to the PP notion of reversing the onus of proof, but is not consistent with EBM as normally practiced, where the onus is on showing a new practice to be superior to usual practice with a sufficiently high degree of certainty. Under our definitions, we show that where DT and the PP differ in their evaluation is in cost-effectiveness, but only for decisions that involve potential catastrophic circumstances, where the nation-state will act as if it is risk-averse. In those cases, it is likely that the state will pay more, and possibly much more, than DT would allow, in an attempt to mitigate impending disaster. That is, the rules that until now have governed all cost-effectiveness analyses are shown not to apply to catastrophic situations, where the PP applies.

Preventing dementia by promoting physical activity and the long-term impact on health and social care expenditures.

BACKGROUND: Preventing dementia has been proposed to increase population health as well as reduce the demand for health and social care. Our aim was to evaluate whether preventing dementia by promoting physical activity (PA) a) improves population health or b) reduces expenditure for both health and social care if one takes into account the additional demand in health and social care caused by increased life expectancy. METHODS: A simulation model was developed that models the relation between PA, dementia, mortality, and the use of health care and social care in England. With this model, scenarios were evaluated in which different assumptions were made about the increase in PA level in (part of) the population. RESULTS: Lifetime spending on health and social care related to dementia was highest for the physically inactive (£28,100/£28,900 for 40-year-old males/females), but spending on other diseases was highest for those that meet PA recommendations (£55,200/£43,300 for 40-year-old males/females) due to their longer life expectancies. If the English population aged 40-65 were to increase their PA by one level, life expectancy would increase by 0.23years and health and social care expenditures would decrease by £400 per person. CONCLUSIONS: Preventing dementia by increasing PA increases life expectancy and can result in decreased spending overall on health and social care, even after additional spending during life years gained has been taken into account. If prevention is targeted at the physically inactive, savings in dementia-related costs outweigh the additional spending in life years gained.

The appraisal of public health interventions: the use of theory.

BACKGROUND: Public health decision-making is hampered by inappropriate adherence to underpowered randomized controlled trials (RCTs) which give inconclusive results and lead to decision-makers being loath to recommend interventions with strong theoretical and observational support. METHODS: We outline situations in which robust decisions about health interventions can be made without trial evidence. We present a new approach in which theory, causal models and past observations are given proper regard in the decision-making process. RESULTS: Using our approach, we provide examples where the use of causal theories and observations in areas, such as salt reduction, smoking cessation and gardening to improve mental health, is sufficient for deciding that such interventions are effective for improving health without needing the support of underpowered RCTs. Particularly where RCT evidence is inconclusive, our approach may provide similar aggregate health outcomes for society for vastly lower cost. CONCLUSIONS: When knowledge and theoretical understanding are unable sufficiently to reduce doubt about the direction of effect from an intervention, decisions should be made using evidence-based medicine approaches. There are, however, many cases where the combination of robust theory, causal understanding and observation are able to provide sufficient evidence of the direction of effect from an intervention that current practice should be altered.

The cost-effectiveness of public health interventions.

BACKGROUND: The need to make best use of limited resources in the English National Health Service is now greater than ever. This paper contributes to this endeavour by synthesizing data from cost-effectiveness evidence produced to support the development of public health guidance at the National Institute of Health and Clinical Excellence (NICE). No comprehensive list of cost-effectiveness estimates for public health interventions has previously been published in England. METHODS: Cost-effectiveness estimates using English cost data were collected and analysed from 21 (of 26) economic analyses underpinning public health guidance published by NICE between 2006 and 2010. RESULTS: Two hundred base-case cost-effectiveness estimates were analysed, 15% were cost saving (i.e. the intervention was more effective and cheaper than comparator). Eighty-five per cent were cost-effective at a threshold of £20,000 per quality-adjusted life year and 89% at the higher threshold of £30,000. A further 5.5% were above £30,000 and 5.5% of the interventions were dominated (i.e. the intervention was more costly and less effective than comparator). CONCLUSIONS: The majority of public health interventions assessed are highly cost-effective. The next challenge is to provide commissioners with a framework that allows information from economic analyses to be combined with other criteria that supports making better investment decisions at a local level.

Evidence-based decision making: when should we wait for more information?

We discuss the challenge of managing innovation in and access to health care interventions in an evidence-based, cost-effective way, and we describe a decision-making framework (using U.S. and U.K. case studies) for health care payers considering the adoption of new technologies. We argue that providing reimbursement for what could be a cost-effective technology "only in the context of research" will be appropriate if the costs of delaying implementation are offset by the value of "keeping one's options open" by waiting for more information. We conclude that there is a need for better integration of health care decision-making processes with research policies.

Exploration of the difference in results of economic submissions to the National Institute of Clinical Excellence by manufacturers and assessment groups.

OBJECTIVES: A recent study showed that estimates of cost-effectiveness submitted to National Institute for Health and Clinical Excellence (NICE) by manufacturers had significantly lower incremental cost-effectiveness ratios (ICERs) than those submitted by university-based Assessment Groups. This study extends that analysis. METHODS: Data were abstracted from relevant NICE documentation for thirty-two of eighty-two possible appraisals. RESULTS: The results from the analysis showed that sources of the difference in ICERs appear to be the effectiveness estimates relating to the comparator technology and the cost estimates relating to the technology under evaluation. That is, manufacturers estimated lower average benefits for the comparator technology and lower costs relating to the technology under evaluation compared with estimates submitted by the Assessment Groups. CONCLUSIONS: These findings may be particularly important, given the introduction of the "Single Technology Appraisal." Considerable difficulties were encountered when undertaking this study, highlighting, above all else, the complexity of explaining why results from economic evaluations purporting to answer the same question diverge.

Cholinesterase inhibitors for Alzheimer's disease: variations in clinical practice in the north-west of England.

BACKGROUND: In the UK, the National Institute for Health and Clinical Excellence (NICE) is currently reviewing cost-effectiveness of cholinesterase inhibitors (ChEIs) for Alzheimer's disease (AD) in the NHS. The implementation of the original NICE (2001) guidance that made these drugs available in the UK was not examined in this review process, and was the objective of this study. METHOD: Ten hospitals in and around Manchester participated in the audit. Case records of all patients who were started on ChEIs between January 2002 and the date of the audit (November 2003 to December 2004) were examined to assess adherence to the NICE (2001) guidance. Information about available resources was gathered from lead clinicians of each hospital. RESULTS: A total of 1,092 case records were examined. The treatment of AD with ChEIs varied considerably with regard to pre-treatment investigations, waiting period for treatment, scales used to assess efficacy, adherence to the NICE (2001) guidance, and available resources. The actual number of patients with mild to moderate AD receiving ChEIs was much lower than that expected from known prevalence rates. Non-uniformity of clinical practice and missing data were the main limitations of the audit. CONCLUSION: There is a need to harmonise the use and monitoring of ChEIs for AD in the NHS so that the data from routine clinical practice could feed into future clinical and cost-effectiveness analyses.

The economics of diagnosis.

Any population can be divided into two groups, one with the presence of a given disease or condition, and the other without. Diagnosis consists of using tests to sort the population into these groups. Diagnostic tests use a threshold value of a diagnostic variable to distinguish between disease-positive and disease-negative individuals. The analysis of error in diagnostic tests has typically been undertaken using receiver-operator characteristic (ROC) curves. More recently, economic value of information (VOI) methods have characterised the costs and consequences of testing. This paper develops a new method for economic test evaluation, which we call ROTS analysis. The ROTS curve plots the costs and effects of changing test thresholds, in cost-effectiveness space. We illustrate the use of our method with a worked example, and show how it can answer three key questions: (1) Is there any test that is worth doing? (2) What is a test's optimum operating point in terms of sensitivity and specificity? (3) If two tests are available, which is best? We contrast the merits of our method with those of established ROC and VOI analysis. We argue that ROTS analysis more clearly reveals the link between changing test thresholds and the cost-effectiveness of different treatments.