Allocating Public Spending Efficiently: Is There a Need for a Better Mechanism to Inform Decisions in the UK and Elsewhere?

In the UK few if any regular processes explicitly address comparisons of value for money between spending in different government departments, despite the existence of mechanisms that could in principle achieve that. This leaves a very important gap in evidence and means that decisions about public spending allocations are likely to miss opportunities to improve social welfare from existing budgets. Greater attention to the development of methods and evidence to better inform the allocation of public sector spending between departments is therefore urgently needed. We identify a number of possible approaches to this-some of which are being used in different countries-and highlight their strengths and weaknesses. We propose a new, pragmatic approach that incorporates a generic descriptive system to measure the disparate outcomes produced by public sector activities in a commensurate manner. Discrete-choice experiments could be used to generate evidence of the relative importance placed on different aspects of public sector outcomes by members of the general public. The proposed approach would produce evidence on value for money across departments, and the generation of evidence on public preferences to support that.

Economic returns to medical research funding.

Following the publication of the final paper in a planned series of four studies estimating the economic returns from biomedical and health research, we reflect on what we have learnt from these types of assessment.

Estimating the returns to United Kingdom publicly funded musculoskeletal disease research in terms of net value of improved health outcomes.

BACKGROUND: Building on an approach applied to cardiovascular and cancer research, we estimated the economic returns from United Kingdom public- and charitable-funded musculoskeletal disease (MSD) research that arise from the net value of the improved health outcomes in the United Kingdom. METHODS: To calculate the economic returns from MSD-related research in the United Kingdom, we estimated (1) the public and charitable expenditure on MSD-related research in the United Kingdom between 1970 and 2013; (2) the net monetary benefit (NMB), derived from the health benefit in quality adjusted life years (QALYs) valued in monetary terms (using a base-case value of a QALY of £25,000) minus the cost of delivering that benefit, for a prioritised list of interventions from 1994 to 2013; (3) the proportion of NMB attributable to United Kingdom research; and (4) the elapsed time between research funding and health gain. The data collected from these four key elements were used to estimate the internal rate of return (IRR) from MSD-related research investments on health benefits. We analysed the uncertainties in the IRR estimate using a one-way sensitivity analysis. RESULTS: Expressed in 2013 prices, total expenditure on MSD-related research from 1970 to 2013 was £3.5 billion, and for the period used to estimate the rate of return, 1978-1997, was £1.4 billion. Over the period 1994-2013 the key interventions analysed produced 871,000 QALYs with a NMB of £16 billion, allowing for the net NHS costs resulting from them and valuing a QALY at £25,000. The proportion of benefit attributable to United Kingdom research was 30% and the elapsed time between funding and impact of MSD treatments was 16 years. Our best estimate of the IRR from MSD-related research was 7%, which is similar to the 9% for CVD and 10% for cancer research. CONCLUSIONS: Our estimate of the IRR from the net health gain to public and charitable funding of MSD-related research in the United Kingdom is substantial, and justifies the research investments made between 1978 and 1997. We also demonstrated the applicability of the approach previously used in assessing the returns from cardiovascular and cancer research. Inevitably, with a study of this kind, there are a number of important assumptions and caveats that we highlight, and these can inform future research.

Estimating the returns to UK publicly funded cancer-related research in terms of the net value of improved health outcomes.

BACKGROUND: Building on an approach developed to assess the economic returns to cardiovascular research, we estimated the economic returns from UK public and charitable funded cancer-related research that arise from the net value of the improved health outcomes. METHODS: To assess these economic returns from cancer-related research in the UK we estimated: 1) public and charitable expenditure on cancer-related research in the UK from 1970 to 2009; 2) net monetary benefit (NMB), that is, the health benefit measured in quality adjusted life years (QALYs) valued in monetary terms (using a base-case value of a QALY of GB£25,000) minus the cost of delivering that benefit, for a prioritised list of interventions from 1991 to 2010; 3) the proportion of NMB attributable to UK research; 4) the elapsed time between research funding and health gain; and 5) the internal rate of return (IRR) from cancer-related research investments on health benefits. We analysed the uncertainties in the IRR estimate using sensitivity analyses to illustrate the effect of some key parameters. RESULTS: In 2011/12 prices, total expenditure on cancer-related research from 1970 to 2009 was £15 billion. The NMB of the 5.9 million QALYs gained from the prioritised interventions from 1991 to 2010 was £124 billion. Calculation of the IRR incorporated an estimated elapsed time of 15 years. We related 17% of the annual NMB estimated to be attributable to UK research (for each of the 20 years 1991 to 2010) to 20 years of research investment 15 years earlier (that is, for 1976 to 1995). This produced a best-estimate IRR of 10%, compared with 9% previously estimated for cardiovascular disease research. The sensitivity analysis demonstrated the importance of smoking reduction as a major source of improved cancer-related health outcomes. CONCLUSIONS: We have demonstrated a substantive IRR from net health gain to public and charitable funding of cancer-related research in the UK, and further validated the approach that we originally used in assessing the returns from cardiovascular research. In doing so, we have highlighted a number of weaknesses and key assumptions that need strengthening in further investigations. Nevertheless, these cautious estimates demonstrate that the returns from past cancer research have been substantial, and justify the investments made during the period 1976 to 1995.

Understanding factors associated with the translation of cardiovascular research: a multinational case study approach.

BACKGROUND: Funders of health research increasingly seek to understand how best to allocate resources in order to achieve maximum value from their funding. We built an international consortium and developed a multinational case study approach to assess benefits arising from health research. We used that to facilitate analysis of factors in the production of research that might be associated with translating research findings into wider impacts, and the complexities involved. METHODS: We built on the Payback Framework and expanded its application through conducting co-ordinated case studies on the payback from cardiovascular and stroke research in Australia, Canada and the United Kingdom. We selected a stratified random sample of projects from leading medical research funders. We devised a series of innovative steps to: minimize the effect of researcher bias; rate the level of impacts identified in the case studies; and interrogate case study narratives to identify factors that correlated with achieving high or low levels of impact. RESULTS: Twenty-nine detailed case studies produced many and diverse impacts. Over the 15 to 20 years examined, basic biomedical research has a greater impact than clinical research in terms of academic impacts such as knowledge production and research capacity building. Clinical research has greater levels of wider impact on health policies, practice, and generating health gains. There was no correlation between knowledge production and wider impacts. We identified various factors associated with high impact. Interaction between researchers and practitioners and the public is associated with achieving high academic impact and translation into wider impacts, as is basic research conducted with a clinical focus. Strategic thinking by clinical researchers, in terms of thinking through pathways by which research could potentially be translated into practice, is associated with high wider impact. Finally, we identified the complexity of factors behind research translation that can arise in a single case. CONCLUSIONS: We can systematically assess research impacts and use the findings to promote translation. Research funders can justify funding research of diverse types, but they should not assume academic impacts are proxies for wider impacts. They should encourage researchers to consider pathways towards impact and engage potential research users in research processes.

Cost-effectiveness of initial stress cardiovascular MR, stress SPECT or stress echocardiography as a gate-keeper test, compared with upfront invasive coronary angiography in the investigation and management of patients with stable chest pain: mid-term outcomes from the CECaT randomised controlled trial.

OBJECTIVES: To compare outcomes and cost-effectiveness of various initial imaging strategies in the management of stable chest pain in a long-term prospective randomised trial. SETTING: Regional cardiothoracic referral centre in the east of England. PARTICIPANTS: 898 patients (69% man) entered the study with 869 alive at 2 years of follow-up. Patients were included if they presented for assessment of stable chest pain with a positive exercise test and no prior history of ischaemic heart disease. Exclusion criteria were recent infarction, unstable symptoms or any contraindication to stress MRI. PRIMARY OUTCOME MEASURES: The primary outcomes of this follow-up study were survival up to a minimum of 2 years post-treatment, quality-adjusted survival and cost-utility of each strategy. RESULTS: 898 patients were randomised. Compared with angiography, mortality was marginally higher in the groups randomised to cardiac MR (HR 2.6, 95% CI 1.1 to 6.2), but similar in the single photon emission CT-methoxyisobutylisonitrile (SPECT-MIBI; HR 1.0, 95% CI 0.4 to 2.9) and ECHO groups (HR 1.6, 95% CI 0.6 to 4.0). Although SPECT-MIBI was marginally superior to other non-invasive tests there were no other significant differences between the groups in mortality, quality-adjusted survival or costs. CONCLUSIONS: Non-invasive cardiac imaging can be used safely as the initial diagnostic test to diagnose coronary artery disease without adverse effects on patient outcomes or increased costs, relative to angiography. These results should be interpreted in the context of recent advances in imaging technology. TRIAL REGISTRATION: ISRCTN 47108462, UKCRN 3696.

Exploring what lies behind public preferences for avoiding health losses caused by lapses in healthcare safety and patient lifestyle choices.

BACKGROUND: Although many studies have identified public preferences for prioritising health care interventions based on characteristics of recipient or care, very few of them have examined the reasons for the stated preferences. We conducted an on-line person trade-off (PTO) study (N=1030) to investigate whether the public attach a premium to the avoidance of ill health associated with alternative types of responsibilities: lapses in healthcare safety, those caused by individual action or lifestyle choice; or genetic conditions. We found that the public gave higher priority to prevention of harm in a hospital setting such as preventing hospital associated infections than genetic disorder but drug administration errors were valued similar to genetic disorders. Prevention of staff injuries, lifestyle diseases and sports injuries, were given lower priority. In this paper we aim to understand the reasoning behind the responses by analysing comments provided by respondents to the PTO questions. METHOD: A majority of the respondents who participated in the survey provided brief comments explaining preferences in free text responses following PTO questions. This qualitative data was transformed into explicit codes conveying similar meanings. An overall coding framework was developed and a reliability test was carried out. Recurrent patterns were identified in each preference group. Comments which challenged the assumptions of hypothetical scenarios were also investigated. RESULTS: NHS causation of illness and a duty of care were the most cited reasons to prioritise lapses in healthcare safety. Personal responsibility dominated responses for lifestyle related contexts, and many respondents mentioned that health loss was the result of the individual's choice to engage in risky behaviour. A small proportion of responses questioned the assumptions underlying the PTO questions. However excluding these from the main analysis did not affect the conclusions. CONCLUSION: Although some responses indicated misunderstanding or rejection of assumptions we put forward, the results were still robust. The reasons put forward for responses differed between comparisons but responsibility was the most frequently cited. Most preference elicitation studies only focus on eliciting numerical valuations but allowing for qualitative data can augment understanding of preferences as well as verifying results.

Illustrating potential efficiency gains from using cost-effectiveness evidence to reallocate Medicare expenditures.

OBJECTIVES: The Centers for Medicare & Medicaid Services does not explicitly use cost-effectiveness information in national coverage determinations. The objective of this study was to illustrate potential efficiency gains from reallocating Medicare expenditures by using cost-effectiveness information, and the consequences for health gains among Medicare beneficiaries. METHODS: We included national coverage determinations from 1999 through 2007. Estimates of cost-effectiveness were identified through a literature review. For coverage decisions with an associated cost-effectiveness estimate, we estimated utilization and size of the "unserved" eligible population by using a Medicare claims database (2007) and diagnostic and reimbursement codes. Technology costs originated from the cost-effectiveness literature or were estimated by using reimbursement codes. We illustrated potential aggregate health gains from increasing utilization of dominant interventions (i.e., cost saving and health increasing) and from reallocating expenditures by decreasing investment in cost-ineffective interventions and increasing investment in relatively cost-effective interventions. RESULTS: Complete information was available for 36 interventions. Increasing investment in dominant interventions alone led to an increase of 270,000 quality-adjusted life-years (QALYs) and savings of $12.9 billion. Reallocation of a broader array of interventions yielded an additional 1.8 million QALYs, approximately 0.17 QALYs per affected Medicare beneficiary. Compared with the distribution of resources prior to reallocation, following reallocation a greater proportion was directed to oncology, diagnostic imaging/tests, and the most prevalent diseases. A smaller proportion of resources went to cardiology, treatments (including drugs, surgeries, and medical devices, as opposed to nontreatments such as preventive services), and the least prevalent diseases. CONCLUSIONS: Using cost-effectiveness information has the potential to increase the aggregate health of Medicare beneficiaries while maintaining existing spending levels.

Physical activity in England: who is meeting the recommended level of participation through sports and exercise?

BACKGROUND: Little is known about the correlates of meeting recommended levels of participation in physical activity (PA) and how this understanding informs public health policies on behaviour change. OBJECTIVE: To analyse who meets the recommended level of participation in PA in males and females separately by applying 'process' modelling frameworks (single vs. sequential 2-step process). METHODS: Using the Health Survey for England 2006, (n = 14 142; ≥ 16 years), gender-specific regression models were estimated using bivariate probit with selectivity correction and single probit models. A 'sequential, 2-step process' modelled participation and meeting the recommended level separately, whereas the 'single process' considered both participation and level together. RESULTS: In females, meeting the recommended level was associated with degree holders [Marginal effect (ME) = 0.013] and age (ME = -0.001), whereas in males, age was a significant correlate (ME = -0.003 to -0.004). The order of importance of correlates was similar across genders, with ethnicity being the most important correlate in both males (ME = -0.060) and females (ME = -0.133). In females, the 'sequential, 2-step process' performed better (ρ = -0.364, P < 0.001) than that in males (ρ = 0.154). CONCLUSION: The degree to which people undertake the recommended level of PA through vigorous activity varies between males and females, and the process that best predicts such decisions, i.e. whether it is a sequential, 2-step process or a single-step choice, is also different for males and females. Understanding this should help to identify subgroups that are less likely to meet the recommended level of PA (and hence more likely to benefit from any PA promotion intervention).

Cost-effectiveness of a European preventive cardiology programme in primary care: a Markov modelling approach.

OBJECTIVE: To investigate the longer-term cost-effectiveness of a nurse-coordinated preventive cardiology programme for primary prevention of cardiovascular disease (CVD) compared to routine practice from a health service perspective. DESIGN: A matched, paired cluster-randomised controlled trial. SETTING: Six pairs of general practices in six countries. PARTICIPANTS: 1019 patients were randomised to the EUROACTION intervention programme and 1005 patients to usual care (UC) and who completed the 1-year follow-up. OUTCOME MEASURES: Evidence on health outcomes and costs was based on patient-level data from the study, which had a 1-year follow-up period. Future risk of CVD events was modelled, using published risk models based on patient characteristics. An individual-level Markov model for each patient was used to extrapolate beyond the end of the trial, which was populated with data from published sources. We used an 11-year time horizon and investigated the impact on the cost-effectiveness of varying the duration of the effect of the intervention beyond the end of the trial. Results are expressed as incremental cost per quality-adjusted life-year gained. RESULTS: Unadjusted results found the intervention to be more costly and also more effective than UC. However, after adjusting for differences in age, gender, country and baseline risk factors, the intervention was dominated by UC, but this analysis was not able to take into account the lifestyle changes in terms of diet and physical activity. CONCLUSIONS: Although the EUROACTION study achieved healthier lifestyle changes and improvements in management of blood pressure and lipids for patients at high risk of CVD, compared to UC, it was not possible to show, using available risk equations which do not incorporate diet and physical activity, that the intervention reduced longer-term cardiovascular risk cost-effectively. Whether or not an intervention such as that offered by EUROACTION is cost-effective requires a longer-term trial with major cardiovascular events as the outcome. TRIAL REGISTRATION NUMBER: ISRCTN 71715857.

Does responsibility affect the public's valuation of health care interventions? A relative valuation approach to health care safety.

OBJECTIVE: Health services often spend more on safety interventions than seems cost-effective. This study investigates whether the public value safety-related health care improvements more highly than the same improvements in contexts where the health care system is not responsible. METHOD: An online survey was conducted to elicit the relative importance placed on preventing harms caused by 1) health care (hospital-acquired infections, drug administration errors, injuries to health care staff), 2) individuals (personal lifestyle choices, sports-related injuries), and 3) nature (genetic disorders). Direct valuations were obtained from members of the public by using a person trade-off or "matching" method. Participants were asked to choose between two preventative interventions of equal cost and equal health benefit per person for the same number of people, but differing in causation. If participants indicated a preference, their strength of preference was measured by using person trade-off. RESULTS: Responses were obtained from 1030 people, reflecting the sociodemographic mix of the UK population. Participants valued interventions preventing hospital-acquired infections (1.31) more highly than genetic disorders (1.0), although drug errors were valued similarly to genetic disorders (1.07), and interventions to prevent injury to health care staff were given less weight than genetic disorders (0.71). Less weight was also given to interventions related to lifestyle (0.65) and sports injuries (0.41). CONCLUSION: Our results suggest that people do not attach a simple fixed premium to "safety-related" interventions but that preferences depend more subtly on context. The use of the results of such public preference surveys to directly inform policy would therefore be premature.

Bayesian meta-analysis on medical devices: application to implantable cardioverter defibrillators.

OBJECTIVES: The aim of this study is to describe and illustrate a method to obtain early estimates of the effectiveness of a new version of a medical device. METHODS: In the absence of empirical data, expert opinion may be elicited on the expected difference between the conventional and modified devices. Bayesian Mixed Treatment Comparison (MTC) meta-analysis can then be used to combine this expert opinion with existing trial data on earlier versions of the device. We illustrate this approach for a new four-pole implantable cardioverter defibrillator (ICD) compared with conventional ICDs, Class III anti-arrhythmic drugs, and conventional drug therapy for the prevention of sudden cardiac death in high risk patients. Existing RCTs were identified from a published systematic review, and we elicited opinion on the difference between four-pole and conventional ICDs from experts recruited at a cardiology conference. RESULTS: Twelve randomized controlled trials were identified. Seven experts provided valid probability distributions for the new ICDs compared with current devices. The MTC model resulted in estimated relative risks of mortality of 0.74 (0.60-0.89) (predictive relative risk [RR] = 0.77 [0.41-1.26]) and 0.83 (0.70-0.97) (predictive RR = 0.84 [0.55-1.22]) with the new ICD therapy compared to Class III anti-arrhythmic drug therapy and conventional drug therapy, respectively. These results showed negligible differences from the preliminary results for the existing ICDs. CONCLUSIONS: The proposed method incorporating expert opinion to adjust for a modification made to an existing device may play a useful role in assisting decision makers to make early informed judgments on the effectiveness of frequently modified healthcare technologies.

The demand for sports and exercise: results from an illustrative survey.

There is a paucity of empirical evidence on the extent to which price and perceived benefits affect the level of participation in sports and exercise. Using an illustrative sample of 60 adults at Brunel University, West London, we investigate the determinants of demand for sports and exercise. The data were collected through face-to-face interviews that covered indicators of sports and exercise behaviour; money/time price and perceived benefits of participation; and socio-economic/demographic details. Count, linear and probit regression models were fitted as appropriate. Seventy eight per cent of the sample participated in sports and exercise and spent an average of £27 per month and an average of 20 min travelling per occasion of sports and exercise. The demand for sport and exercise was negatively associated with time (travel or access time) and 'variable' price and positively correlated with 'fixed' price. Demand was price inelastic, except in the case of meeting the UK government's recommended level of participation, which is time price elastic (elasticity = -2.2). The implications of data from a larger nationally representative sample as well as the role of economic incentives in influencing uptake of sports and exercise are discussed.

Factors predicting Medicare national coverage: an empirical analysis.

BACKGROUND: Interventions considered to be particularly controversial or expected to significantly impact the Medicare program in the United States are considered in National Coverage Determinations. Medicare coverage for such interventions is limited to those deemed "reasonable and necessary" for the diagnosis or treatment of an illness or injury. What constitutes reasonable and necessary has not, however, been clearly defined. OBJECTIVE: To determine factors associated with positive National Coverage Determinations. RESEARCH DESIGN: A dataset of coverage decisions from 1999 to 2007 (n=195) was created with the following variables: direction of coverage decision; quality of supporting evidence; availability of alternative interventions; cost-effectiveness of intervention; type of intervention; coverage requestor; and year of decision. Univariate and multivariate logistic regression analysis was used to determine factors associated with positive coverage. RESULTS: The following variables were independently associated with positive Medicare coverage: good or fair quality supporting evidence (adjusted odds ratio, OR=6.04, P<0.01); presence of an alternative intervention (OR=0.130, P<0.01); no associated estimate of cost-effectiveness (OR=0.190, P<0.05). In addition, in comparison with coverage decisions made in the years 1999 to 2001, those made in the years 2002 to 2003, 2004 to 2005, and 2006 to 2007, were associated with positive coverage [ORs of 0.311 (P<0.05), 0.310 (P<0.1), and 0.109 (P<0.01), respectively]. CONCLUSIONS: Findings suggest that good or fair quality supporting evidence is a strong predictor of positive coverage. Availability of alternative interventions, more recent decisions, and lack of an associated estimate of cost-effectiveness are associated with a decreased likelihood of positive coverage. The findings highlight Medicare's move to evidence-based coverage decisions, and suggest that coverage decisions are influenced by the availability of cost-effectiveness evidence.

Integrating health economics into the product development cycle: a case study of absorbable pins for treating hallux valgus.

BACKGROUND: The probability of reimbursement is a key factor in determining whether to proceed with or abandon a product during its development. The purpose of this article is to illustrate how the methods of iterative Bayesian economic evaluation proposed in the literature can be incorporated into the development process of new medical devices, adapting them to face the relative scarcity of data and time that characterizes the process. METHODS: A 3-stage economic evaluation was applied: an early phase in which simple methods allow for a quick prioritization of competing products; a mid-stage in which developers synthesize the data into a decision model, identify the parameters for which more information is most valuable, and explore uncertainty; and a late stage, in which all relevant information is synthesized. A retrospective analysis was conducted of the case study of absorbable pins, compared with metallic fixation, in osteotomy to treat hallux valgus. RESULTS: The results from the early analysis suggest absorbable pins to be cost-effective under the beliefs and assumptions applied. The outputs from the models at the mid-stage analyses show the device to be cost-effective with a high probability. Late-stage analysis synthesizes evidence from a randomized controlled trial and informative priors, which are based on previous evidence. It also suggests that absorbable pins are the most cost-effective strategy, although the uncertainty in the model output increased considerably. CONCLUSIONS: This example illustrates how the method proposed allows decisions in the product development cycle to be based on the best knowledge that is available at each stage.

Project Retrosight: Understanding the Returns from Cardiovascular and Stroke Research: The Policy Report.

This project explores the impacts arising from cardiovascular and stroke research funded 15-20 years ago and attempts to draw out aspects of the research, researcher or environment that are associated with high or low impact. The project is a case study-based review of 29 cardiovascular and stroke research grants, funded in Australia, Canada and UK between 1989 and 1993. The case studies focused on the individual grants but considered the development of the investigators and ideas involved in the research projects from initiation to the present day. Grants were selected through a stratified random selection approach that aimed to include both high- and low-impact grants. The key messages are as follows: 1) The cases reveal that a large and diverse range of impacts arose from the 29 grants studied. 2) There are variations between the impacts derived from basic biomedical and clinical research. 3) There is no correlation between knowledge production and wider impacts 4) The majority of economic impacts identified come from a minority of projects. 5) We identified factors that appear to be associated with high and low impact. This article presents the key observations of the study and an overview of the methods involved. It has been written for funders of biomedical and health research and health services, health researchers, and policy makers in those fields. It will also be of interest to those involved in research and impact evaluation.

Does Medicare have an implicit cost-effectiveness threshold?

BACKGROUND: Despite the huge cost of the program, the Centers for Medicare and Medicaid Services (CMS) has maintained a policy that cost-effectiveness is not considered in national coverage determinations (NCDs). OBJECTIVE: To assess whether an implicit cost-effectiveness threshold exists and to determine if economic evidence has been considered in previous NCDs. METHODS: A literature search was conducted to identify estimates of cost-effectiveness relevant to each NCD from 1999-2007 (n = 103). The economic evaluation that best represented each coverage decision was included in a review of the cost-effectiveness of medical interventions considered in NCDs. RESULTS: Of the 64 coverage decisions determined to have a corresponding cost-effectiveness estimate, 49 were associated with a positive coverage decision and 15 with a noncoverage decision. Of the positive decisions, 20 were associated with an economic evaluation that estimated the intervention to be dominant (costs less and was more effective than the alternative), 12 with an incremental cost-effectiveness ratio (ICER) of less than $50,000, 8 with an ICER greater than $50,000 but less than $100,000, and 9 with an ICER greater than $100,000. Fourteen of the sample of 64 decision memos cited or discussed cost-effectiveness information. CONCLUSIONS: CMS is covering a number of interventions that do not appear to be cost-effective, suggesting that resources could be allocated more efficiently. Although the authors identified several instances where cost-effectiveness evidence was cited in NCDs, they found no clear evidence of an implicit threshold.