How different is research done by the Patient-centered Outcomes Research Institute, and what difference does it make?

Aim: To determine whether research funded by the Patient-centered Outcomes Research Institute (PCORI) is consistent with the original aims of Congress and unique among other major USA funders. Methods: We compared a sample of funded projects from PCORI, NIH (Phase IV) and agency for healthcare research and quality (AHRQ; American Recovery and Reinvestment Act [ARRA]-based comparative effectiveness research funding) from 2014 to 2018 on number of outcomes/study, patient-centeredness of outcomes (those related to survival, function, symptoms and health-related quality of life) and other features that may characterize patient-centered research (e.g., whether conducted in a real-world setting) using PCORI portfolio data and ClinicalTrials.gov. Results: The mean number of outcomes in PCORI studies (≥9) appeared higher than NIH (≥3)/AHRQ (5.5); a higher percentage of outcomes/study were patient-centered: >85% PCORI versus 50% AHRQ and ≤30% NIH. The majority of PCORI studies (≥74%) were conducted in a real-world setting; this characteristic could not be identified for NIH/AHRQ studies. Conclusion: PCORI-funded studies appear to have unique aspects relative to NIH and AHRQ that are consistent with PCORI's aims of patient-centeredness.

Using Bayesian Adaptive Trial Designs for Comparative Effectiveness Research: A Virtual Trial Execution.

BACKGROUND: Bayesian and adaptive clinical trial designs offer the potential for more efficient processes that result in lower sample sizes and shorter trial durations than traditional designs. OBJECTIVE: To explore the use and potential benefits of Bayesian adaptive clinical trial designs in comparative effectiveness research. DESIGN: Virtual execution of ALLHAT (Antihypertensive and Lipid-Lowering Treatment to Prevent Heart Attack Trial) as if it had been done according to a Bayesian adaptive trial design. SETTING: Comparative effectiveness trial of antihypertensive medications. PATIENTS: Patient data sampled from the more than 42 000 patients enrolled in ALLHAT with publicly available data. MEASUREMENTS: Number of patients randomly assigned between groups, trial duration, observed numbers of events, and overall trial results and conclusions. RESULTS: The Bayesian adaptive approach and original design yielded similar overall trial conclusions. The Bayesian adaptive trial randomly assigned more patients to the better-performing group and would probably have ended slightly earlier. LIMITATIONS: This virtual trial execution required limited resampling of ALLHAT patients for inclusion in RE-ADAPT (REsearch in ADAptive methods for Pragmatic Trials). Involvement of a data monitoring committee and other trial logistics were not considered. CONCLUSION: In a comparative effectiveness research trial, Bayesian adaptive trial designs are a feasible approach and potentially generate earlier results and allocate more patients to better-performing groups. PRIMARY FUNDING SOURCE: National Heart, Lung, and Blood Institute.

Characteristics and temporal trends in patient registries: focus on the life sciences industry, 1981-2012.

PURPOSE: Patient registries are used to monitor safety, examine real-world effectiveness, and may potentially contribute to comparative effectiveness research. To our knowledge, life sciences industry (LSI)-sponsored registries have not been systematically categorized. This study represents a first step toward understanding such registries over time. METHODS: Studies described as registries were identified in the ClinicalTrials.gov database. Characteristics from these registry records were abstracted and analyzed. RESULTS: Of 1202 registries identified, approximately 47% reported LSI sponsorship. These 562 LSI registries varied in focus: medical devices (n = 193, 34%), specific drugs (n = 173, 31%), procedures (n = 29, 5%), or particular diseases (n = 139, 25%). Thirty-three registries (<6%) evaluated pregnancy outcomes. The most common therapeutic area was cardiovascular (n = 234, 42%); others included endocrinology, immunology, oncology, musculoskeletal disorders, and neurology. The two most often measured outcomes were clinical effectiveness and safety, each of which appeared in 363/562 (65%) of LSI registries. Other outcomes included real-world clinical practice patterns (n = 122, 22%), patient-reported outcomes (n = 106, 19%), disease epidemiology/natural history (n = 69, 12%), and economic outcomes (n = 30, 5%). The number of LSI registries and their geographic diversity has increased over time. CONCLUSIONS: The LSI registries represent a substantial proportion of all patient registries documented in ClinicalTrials.gov. These prospective studies are growing in number and encompass diverse therapeutic areas and geographic regions. Most registries measure multiple outcomes and capture real-world data that may be unavailable through other study designs. This classification of LSI registries documents their use for studying heterogeneity of diseases, examining treatment patterns, measuring patient-reported outcomes, examining economic outcomes, and performing comparative effectiveness research.

Do Bayesian adaptive trials offer advantages for comparative effectiveness research? Protocol for the RE-ADAPT study.

BACKGROUND: Randomized clinical trials, particularly for comparative effectiveness research (CER), are frequently criticized for being overly restrictive or untimely for health-care decision making. PURPOSE: Our prospectively designed REsearch in ADAptive methods for Pragmatic Trials (RE-ADAPT) study is a 'proof of concept' to stimulate investment in Bayesian adaptive designs for future CER trials. METHODS: We will assess whether Bayesian adaptive designs offer potential efficiencies in CER by simulating a re-execution of the Antihypertensive and Lipid Lowering Treatment to Prevent Heart Attack Trial (ALLHAT) study using actual data from ALLHAT. RESULTS: We prospectively define seven alternate designs consisting of various combinations of arm dropping, adaptive randomization, and early stopping and describe how these designs will be compared to the original ALLHAT design. We identify the one particular design that would have been executed, which incorporates early stopping and information-based adaptive randomization. LIMITATIONS: While the simulation realistically emulates patient enrollment, interim analyses, and adaptive changes to design, it cannot incorporate key features like the involvement of data monitoring committee in making decisions about adaptive changes. CONCLUSION: This article describes our analytic approach for RE-ADAPT. The next stage of the project is to conduct the re-execution analyses using the seven prespecified designs and the original ALLHAT data.

Principles for planning and conducting comparative effectiveness research.

AIMS: To develop principles for planning and conducting comparative effectiveness research (CER). METHODS: Beginning with a modified existing list of health technology assessment principles, we developed a set of CER principles using literature review, engagement of multiple experts and broad stakeholder feedback. RESULTS & CONCLUSION: Thirteen principles and actions to fulfill their intent are proposed. Principles include clarity of objectives, transparency, engagement of stakeholders, consideration of relevant perspectives, use of relevant comparators, and evaluation of relevant outcomes and treatment heterogeneity. Should these principles be found appropriate and useful, CER studies should be audited for adherence to them and monitored for their impact on care management, patient relevant outcomes and clinical guidelines.

The role of private industry in pragmatic comparative effectiveness trials.

Comparative effectiveness research (CER) includes pragmatic clinical trials (PCTs) to address 'real-world' effectiveness. CER interest would be expected to stimulate biopharmaceutical manufacturer PCT investment; however, this does not seem to be the case. In this article we identify all industry-sponsored PCT studies from 1996 to 2010; analyze them across a variety of characteristics, including sponsor, research question, design, comparators and results; and suggest methodological and policy changes to spur future manufacturer PCT investment. Nine 'naturalistic', head-to-head versus standard of care or similar agent PCTs were identified. Two included a 'usual care' arm. Chronic care trials' length averaged 12 months (range: 6-24 months), six of which reported equivocal or no difference in effectiveness; results of two chronic and the single acute care PCTs favored the sponsor drug. None reported the sponsor drug inferior. Of seven that evaluated utilization or costs, six reported no differences and four of five studies comparing brand-generic drugs reported no difference. Whereas private investment in PCTs is in the public interest, manufacturers apparently have not yet seen the business case. To induce investment, we propose several methodological and regulatory policy innovations designed to reduce business risk by decreasing outcome variability and increasing trial efficiency, flexibility and market applicability.

EBM, HTA, and CER: clearing the confusion.

CONTEXT: The terms evidence-based medicine (EBM), health technology assessment (HTA), comparative effectiveness research (CER), and other related terms lack clarity and so could lead to miscommunication, confusion, and poor decision making. The objective of this article is to clarify their definitions and the relationships among key terms and concepts. METHODS: This article used the relevant methods and policy literature as well as the websites of organizations engaged in evidence-based activities to develop a framework to explain the relationships among the terms EBM, HTA, and CER. FINDINGS: This article proposes an organizing framework and presents a graphic demonstrating the differences and relationships among these terms and concepts. CONCLUSIONS: More specific terminology and concepts are necessary for an informed and clear public policy debate. They are even more important to inform decision making at all levels and to engender more accountability by the organizations and individuals responsible for these decisions.

Bayesian meta-analyses for comparative effectiveness and informing coverage decisions.

BACKGROUND: Evidence-based medicine is increasingly expected in health care decision-making. The Centers for Medicare and Medicaid have initiated efforts to understand the applicability of Bayesian techniques for synthesizing evidence. As a case study, a Bayesian analysis of clinical trials of implantable cardioverter defibrillators was undertaken using patient-level data not typically available for analysis. PURPOSE: Conduct Bayesian meta-analyses of the defibrillator trials using published results to demonstrate a Bayesian approach useful to policy makers. DATA SOURCES, STUDY SELECTION, DATA EXTRACTION: We reconsidered trials in a 2007 systematic review by Ezekowitz et al (Ann Intern Med. 2007;147:251-262) and extracted information from the original published articles. Employing a Bayesian hierarchical approach, we developed a base model and 2 variants, and modeled hazard ratios separately within each year of follow-up. We considered sequential meta-analyses over time and found the predictive distribution of the results of the next trial, given its sample size. DATA SYNTHESIS: For the most robust of 3 models, the probability that the mean defibrillator effect (in the population of trials) is beneficial is greater than 0.999. In that model, about 5% of trials in the population of trials would have a detrimental effect. Despite the moderate amount of heterogeneity across the trials, there was stability of conclusions after the first 3 of the 12 total trials had been conducted. This stability enabled reasonable predictions for the results of future trials. LIMITATIONS: Inability to assess treatment effects within subsets of patients. CONCLUSIONS: Bayesian meta-analyses based on literature surveys can effectively inform coverage decisions. Bayesian modeling for endpoints such as mortality can elucidate treatment effects over time. The Bayesian approach used in a sequential manner over time can predict results and help assess the utility of future clinical trials.

Are Key Principles for improved health technology assessment supported and used by health technology assessment organizations?

Previously, our group-the International Working Group for HTA Advancement-proposed a set of fifteen Key Principles that could be applied to health technology assessment (HTA) programs in different jurisdictions and across a range of organizations and perspectives. In this commentary, we investigate the extent to which these principles are supported and used by fourteen selected HTA organizations worldwide. We find that some principles are broadly supported: examples include being explicit about HTA goals and scope; considering a wide range of evidence and outcomes; and being unbiased and transparent. Other principles receive less widespread support: examples are addressing issues of generalizability and transferability; being transparent on the link between HTA findings and decision-making processes; considering a full societal perspective; and monitoring the implementation of HTA findings. The analysis also suggests a lack of consensus in the field about some principles--for example, considering a societal perspective. Our study highlights differences in the uptake of key principles for HTA and indicates considerable room for improvement for HTA organizations to adopt principles identified to reflect good HTA practices. Most HTA organizations espouse certain general concepts of good practice--for example, assessments should be unbiased and transparent. However, principles that require more intensive follow-up--for example, monitoring the implementation of HTA findings--have received little support and execution.

Value of biologic therapy: a forecasting model in three disease areas.

OBJECTIVE: Forecast the return on investment (ROI) for advances in biologic therapies in years 2015 and 2030, based upon impact on disease prevalence, morbidity, and mortality for asthma, diabetes, and colorectal cancer. METHODS: A deterministic, spreadsheet-based, forecasting model was developed based on trends in demographics and disease epidemiology. 'Return' was defined as reductions in disease burden (prevalence, morbidity, mortality) translated into monetary terms; 'investment' was defined as the incremental costs of biologic therapy advances. Data on disease prevalence, morbidity, mortality, and associated costs were obtained from government survey statistics or published literature. Expected impact of advances in biologic therapies was based on expert opinion. Gains in quality-adjusted life years (QALYs) were valued at $100,000 per QALY. RESULTS: The base case analysis, in which reductions in disease prevalence and mortality predicted by the expert panel are not considered, shows the resulting ROIs remain positive for asthma and diabetes but fall below $1 for colorectal cancer. Analysis involving expert panel predictions indicated positive ROI results for all three diseases at both time points, ranging from $207 for each incremental dollar spent on biologic therapies to treat asthma in 2030, to $4 for each incremental dollar spent on biologic therapies to treat colorectal cancer in 2015. If QALYs are not considered, the resulting ROIs remain positive for all three diseases at both time points. CONCLUSIONS: Society may expect substantial returns from investments in innovative biologic therapies. These benefits are most likely to be realized in an environment of appropriate use of new molecules. LIMITATIONS: The potential variance between forecasted (from expert opinion) and actual future health outcomes could be significant. Similarly, the forecasted growth in use of biologic therapies relied upon unvalidated market forecasts.

Willingness to pay for cancer prevention.

Cancer inflicts great pain, burden and cost upon American society, and preventing cancer is important but not costless. The aim of this review was to explore the upper limits that American society is paying and appears willing to pay to prevent cancer, by enforced environmental regulations and implemented clinical practice guidelines. Cost-effectiveness studies of clinical and environmental cancer-prevention policies and programmes were identified through a comprehensive literature review and confirmed to be officially sanctioned and implemented, enforced or funded. Data were collected in 2005-6 and analysed in 2007. The incremental cost-effectiveness ratios (ICERs) for clinical prevention policies ranged from under $US2000 to over $US6 000 000 per life-year saved (LYS), exceeding $US100 000 per LYS for only 11 of 101 guidelines. Median ICERs for tobacco-related ($US3978/LYS), colorectal ($US22 694/LYS) and breast ($US25 687/LYS) cancer prevention were within generally accepted ranges and tended not to vary greatly, whereas those for prostate ($US73 603/LYS) and cervical ($US125 157/LYS) cancer-prevention policies were considerably higher and varied substantially more. In contrast, both the median and range of the environmental policies were enormous, with 90% exceeding $US100 000 per LYS, and ICERs ranging from $US61 004 to over $US24 billion per LYS. Notwithstanding a relatively large and accessible literature evaluating the cost effectiveness of clinical and environmental cancer-prevention policies as well as the availability of ICERs for the policies identified in this study, the apparent willingness to pay to prevent cancer in the US still varies greatly and can be extremely high, particularly for many of the environmental cancer-prevention policies.

Key principles for the improved conduct of health technology assessments for resource allocation decisions.

Health technology assessment (HTA) is a dynamic, rapidly evolving process, embracing different types of assessments that inform real-world decisions about the value (i.e., benefits, risks, and costs) of new and existing technologies. Historically, most HTA agencies have focused on producing high quality assessment reports that can be used by a range of decision makers. However, increasingly organizations are undertaking or commissioning HTAs to inform a particular resource allocation decision, such as listing a drug on a national or local formulary, defining the range of coverage under insurance plans, or issuing mandatory guidance on the use of health technologies in a particular healthcare system. A set of fifteen principles that can be used in assessing existing or establishing new HTA activities is proposed, providing examples from existing HTA programs. The principal focus is on those HTA activities that are linked to, or include, a particular resource allocation decision. In these HTAs, the consideration of both costs and benefits, in an economic evaluation, is critical. It is also important to consider the link between the HTA and the decision that will follow. The principles are organized into four sections: (i) "Structure" of HTA programs; (ii) "Methods" of HTA; (iii) "Processes for Conduct" of HTA; and (iv) "Use of HTAs in Decision Making."

Cost-effectiveness of live attenuated influenza vaccine versus inactivated influenza vaccine among children aged 24-59 months in the United States.

BACKGROUND: The US Advisory Committee on Immunization Practices (ACIP) recently expanded the influenza vaccine recommendation to include children 24-59 months of age. In a large head-to-head randomized controlled trial, live attenuated influenza vaccine, trivalent (LAIV) demonstrated a 54% relative reduction in culture-confirmed influenza illness compared with trivalent inactivated influenza vaccine (TIV) among children aged 24-59 months. OBJECTIVE: To evaluate the relative cost and benefit between two influenza vaccines (LAIV and TIV) for healthy children 24-59 months of age. METHODS: Using patient-level data from the clinical trial supplemented with cost data from published literature, we modeled the cost-effectiveness of these two vaccines. Effectiveness was measured in quality-adjusted life years (QALY) and cases of influenza avoided. The analysis used the societal perspective. RESULTS: Due to its higher acquisition cost, LAIV increased vaccination costs by USD7.72 per child compared with TIV. However, compared with TIV, LAIV reduced the number of influenza illness cases and lowered the subsequent healthcare use of children and productivity losses of parents. The estimated offsets in direct and indirect costs saved USD15.80 and USD37.72 per vaccinated child, respectively. LAIV had a net total cost savings of USD45.80 per child relative to TIV. One-way and probabilistic sensitivity analyses indicated that the model was robust across a wide range of relative vaccine efficacy and cost estimates. CONCLUSIONS: Due to its increased relative vaccine efficacy over TIV, LAIV reduced the burden of influenza and lowered both direct health care and societal costs among children 24-59 months of age.

Can managed care organizations partner with manufacturers for comparative effectiveness research?

OBJECTIVE: To describe 2 published pragmatic or practical clinical trials (PCTs) as case studies illustrating successful partnerships between managed care organizations (MCOs) and pharmaceutical manufacturers. STUDY DESIGN: In today's environment, there is increasing concern about the comparative effectiveness of medical interventions. Various opinion leaders and stakeholders lament the dearth of such evidence and are calling for the public and private sectors to invest up to billions of dollars to create better comparative evidence. METHODS: We selected 2 PCTs conducted at different points in the drug life cycle to highlight strengths, limitations, and policy implications. The phase IV study compared fluoxetine hydrochloride vs 2 generic tricyclic antidepressants in selected primary care clinics of a health maintenance organization from 1992 through 1994. The phase IIIb study compared daily budesonide via dry powder inhaler vs triamcinolone acetonide metered-dose inhaler in adult patients with persistent asthma in 25 MCOs from 1995 through 1998. RESULTS: Both PCTs were successfully sponsored and funded by pharmaceutical manufacturers in collaboration with MCOs and provided potentially useful evidence of real-world effectiveness and evidence of value to healthcare decision makers. CONCLUSIONS: Industry-sponsored PCTs in managed care are feasible when manufacturer and MCO incentives align and can provide real-world evidence of comparative effectiveness and value for money. These trials can be conducted successfully in the phase IIIb and phase IV environments.

The return on investment in health care: from 1980 to 2000.

OBJECTIVE: To estimate the return on US investment (ROI) in overall health as well as four specific conditions. METHODS: The study utilized three distinct approaches to "triangulate" the evidence as related to ROI in health care: 1) an estimation of the average ROI in additional health-care service expenditures in the United States for the year 2000 compared with the year 1980, based on US summaries of health expenditures and health outcomes; 2) an estimate of the ROI in Medicare services for the period from 1985 to 2000 for treatment of heart attack, stroke, type 2 diabetes, and breast cancer, based on National Long-term Care Survey data and Medicare claims; and 3) an estimate of the ROI for selected major treatment innovations for the same four conditions during the period from 1975 to 2000. RESULTS: We calculated that each additional dollar spent on overall health-care services produced health gains valued at Dollars 1.55 to Dollars 1.94 under our base case assumptions. The return on health gains associated with treatment for heart attack, stroke, type 2 diabetes, and breast cancer were Dollars 1.10, Dollars 1.49, Dollars 1.55, and Dollars 4.80, respectively, for every additional dollar spent by Medicare. The ROI for specific treatment innovations ranged from both savings in treatment costs and gains in health to gains in health valued at Dollars 1.12 to Dollars 38.00 for every additional dollar spent. CONCLUSION: The value of improved health in the US population in 2000 compared with 1980 significantly outweighs the additional health-care expenditures in 2000 compared with 1980.

Patient satisfaction with budesonide Turbuhaler versus triamcinolone acetonide administered via pressurized metered-dose inhaler in a managed care setting.

Dissatisfaction with medication may negatively affect compliance and thus the effectiveness of the treatment. However, no prospective well-controlled studies have assessed the relative patient satisfaction with competing inhaled corticosteroids in a real-life setting. The objective of the current study was to compare the relative patient satisfaction with budesonide inhalation powder administered via Turbuhaler (AstraZeneca LP, Wilmington, DE) (200 to 1600 microg/d using one of 3 dosing strengths: 100, 200, or 400 microg per inhalation) and triamcinolone acetonide administered via pressurized metered-dose inhaler (200 to 1600 microg/d) among persons treated in managed care settings. A total of 945 subjects 18 years of age or older diagnosed with asthma and enrolled in 25 managed care organizations participated in this prospective, randomized, open-label, parallel-group, 12-month study. As part of the study, subjects completed a self-administered, 17-item patient satisfaction questionnaire that addressed 4 domains: side effects, knowledge/ease of use, convenience, and overall satisfaction. Questionnaire reliability was assessed using Cronbach's alpha, and validity was examined by correlating subscale scores with symptom-free days and Medical Outcomes Study 36-Item Short-Form questionnaire and Asthma Quality of Life Questionnaire scores. The satisfaction questionnaire also included a previously validated section addressing patient compliance. Patients receiving budesonide had significantly higher scores for all four satisfaction subscales throughout the study period than did those receiving triamcinolone acetonide. Similarly, compliance scores were consistently higher for the budesonide group. The difference between the treatment groups in overall satisfaction scores at the end of the study was clinically meaningful. Patients treated with budesonide were significantly more satisfied and compliant with their inhaled corticosteroid regimen compared with patients treated with triamcinolone acetonide.

What will it take to make cost-effectiveness analysis acceptable in the United States?

Many believe cost-effectiveness analysis (CEA) to be an underused tool to assist healthcare decision-makers. Reasons given for its underuse have largely focused on unstandardized methods, potential for bias, lack of training particularly among potential consumers of studies, and lack of trust between sponsors and users of analyses. This commentary reflects on these and related issues, including legal and political constraints. It discounts many of the conventional arguments regarding the real obstacles to using CEA and suggests steps needed to make CEA more acceptable to US healthcare decision-makers.

Evidence based? Caveat emptor!

Medical practices, clinical practice guidelines, clinical performance measures and measurements, and a variety of health care-related administrative decisions, such as insurance coverage decisions, are claiming to be "evidence based" with increasing frequency. In this paper we examine the "evidence based" label; discuss how evidence ought to have been assembled, evaluated, and synthesized; and when evidence is sufficient for the "evidence-based" moniker to rightfully apply. We also highlight several considerations other than the strength of evidence that are relevant to several common types of health care-related administrative decisions and that influence the extent to which the resulting decisions are truly evidence based.