Developing Patient-Centered Real-World Evidence: Emerging Methods Recommendations From a Consensus Process.

OBJECTIVES: The Joint ISPOR-ISPE Special Task Force on Real-World Evidence included patient/stakeholder engagement as a recommended good procedural practice when designing, conducting, and disseminating real-world evidence (RWE). However, there are no guidelines describing how patient experience data (PED) can be applied when designing real-world data (RWD) studies. This article describes development of consensus recommendations to guide researchers in applying PED to develop patient-centered RWE. METHODS: A multidisciplinary advisory board, identified through recommendations of collaborators, was established to guide development of recommendations. Semistructured interviews were conducted to identify how experienced RWD researchers (n = 15) would apply PED when designing a hypothetical RWD study. Transcripts were analyzed and emerging themes developed into preliminary methods recommendations. An eDelphi survey (n = 26) was conducted to refine/develop consensus on the draft recommendations. RESULTS: We identified 13 recommendations for incorporating PED throughout the design, conduct, and translation of RWE. The recommendations encompass themes related to the development of a patient-centered research question, designing a study, disseminating RWE, and general considerations. For example, consider how patient input can inform population/subgroups, comparators, and study period. Researchers can leverage existing information describing PED and may be able to apply those insights to studies relying on traditional RWD sources and/or patient registries. CONCLUSIONS: Applying these emerging recommendations may improve the patient centricity of RWE through improved relevance of RWE to patient communities of interest and foster greater multidisciplinary participation and transparency in RWD research. As researchers gather experience by applying the methods recommendations, further refinement of these consensus recommendations may lead to "best practices."

Incidence of mental health conditions by seizure control among adults with epilepsy in the United States.

BACKGROUND: Mental health conditions (MHCs) are frequent comorbidities among people with epilepsy; however, the influence of seizure control on the incidence of MHCs is not well reported. This retrospective observational cohort study based on claims data evaluated the effects of indicators of poor seizure control on the incidence of MHCs among MHC-naïve people with epilepsy. We hypothesized that poor seizure control is associated with new-onset MHC diagnoses and/or new prescription drugs for MHCs. METHODS: This study utilized a sample of patients from HealthVerity Marketplace, which includes more than 150 US commercial, Medicare, and Medicaid payers, to identify a cohort of adults (age ≥18 years) with prevalent epilepsy. Follow-up started on day 1 (January 1) after a 1-year eligibility assessment period occurring in calendar year 2017 or 2018. Patients were followed up until the occurrence of an incident MHC event (primary outcome), defined as a mental health diagnosis or psychotropic drug prescription. Time from follow-up to incident MHC diagnosis or to a drug prescription specific to depression or anxiety disorder was analyzed as a secondary outcome. Multivariate Cox proportional hazards regressions were estimated with time-varying covariates, measured in 6-month intervals during follow-up. Time-varying covariates were based on the occurrence of 4 variables used as indicators of poor seizure control in the prior period: epilepsy-related emergent care admissions, epilepsy-related inpatient admissions, epilepsy electroencephalography referrals, and exposure to one or more new antiseizure medications (ASMs). RESULTS: From a random sample of 40,000 people with epilepsy, 2563 (mean age 46.1 years; 50.6% male) were included in the analysis. Incident MHC events were observed in 27.7% (incidence rate 24.4 events per 100 person-years over 2,915.7 total person-years of follow-up). Mean (standard deviation [SD]) time to event was 232.7 (186.3) days. Among the 4 variables, epilepsy-related emergent care admissions were associated with an increased risk of incident MHC events in the following 6-month period (hazard ratio [HR] = 1.676, 95% confidence interval [CI]: 1.386, 2.026, p < 0.001) as were prescriptions for new ASMs in the previous period (HR = 1.702, 95% CI: 1.359, 2.132, p < 0.001). Previous epilepsy-related emergent care admissions (HR = 1.650, 95% CI: 1.347, 2.021, p < 0.001) and new ASMs (HR = 1.632, 95% CI: 1.280, 2.081, p < 0.001) also predicted an increased risk of incident depression or anxiety in the following 6-month period. CONCLUSIONS: Previous indicators of poor seizure control, including epilepsy-related emergent care admissions and new ASMs, predicted increased risk of new MHC events, including depression and anxiety, during the following 6-month interval in MHC-naïve patients with prevalent epilepsy. These data suggest that poor seizure control can increase the subsequent risk of new mental health diagnoses and treatment among people with epilepsy.

The History and Future of the "ISPOR Value Flower": Addressing Limitations of Conventional Cost-Effectiveness Analysis.

OBJECTIVES: Since its publication as part of the 2018 ISPOR Special Task Force (STF) on US Value Assessments, the "ISPOR value flower," with its petals highlighting elements that may be overlooked or underappreciated in conventional drug value assessments, has been discussed and debated. We review the history of the value flower, describe recent developments, and consider implications for future value assessments. METHODS: We discuss various antecedents to the value flower, as well as conceptual and empirical articles published in the past 4 years. RESULTS: Since the publication of the ISPOR STF report, researchers have provided more rigorous theoretical and mathematical foundations for certain novel value elements (eg, severity of illness, value of insurance, value of hope) through "generalized risk-adjusted cost-effectiveness analysis," which incorporates risk aversion in people's preferences and uncertainty in treatment outcomes. Empirical estimates are also emerging to support key elements, such as insurance value, real option value, value of hope, and value of knowing. Although health technology assessment bodies have applied or are considering certain elements (eg, severity modifiers to cost-effectiveness thresholds), other elements have yet to gain traction. CONCLUSIONS: Five years after the STF began its work, the development of novel value measures continues to evolve. Although it is encouraging to see supporting empirical studies emerging, more are needed. Additional efforts are also needed to illustrate how the estimates can be used in the deliberative processes that are integral to health technology assessments.

Valuing treatment in oncology: embracing a broader notion of value.

Over the past decade, we have witnessed unprecedented, groundbreaking innovation in pharmaceuticals. This has been particularly true in oncology, where new therapies have increased survival and at times offered clinical cure. However, the impact of these promising treatments has been attenuated by persistent access and cost challenges that may limit their effect. A narrative has emerged that many of these so-called breakthroughs are not priced according to the value they provide. Traditional cost-effectiveness analyses would appear to support these doubts, often suggesting that innovative therapies do not represent value for money. However, there is a case to be made that innovative therapies require equally innovative value assessments. To explore this emerging viewpoint, this article provides a brief introduction to the current value debate and oncology-specific considerations when assessing elements of value. We offer a brief background on the nature and development of quality-adjusted life-years as a part of cost-effectiveness analyses and some of their key limitations; a primer on "novel" elements of value, which capture specific aspects of patient and societal preferences not included in quality-adjusted life-years; and their applicability to oncology including discussion on areas where further thought and research might be needed. We conclude with a potential checklist of novel elements of value that should be considered. DISCLOSURES: This Viewpoints article was funded by Novartis, Inc., which also provided funding to COVIA Health Solutions for manuscript development. The sponsor was involved in developing the manuscript. Kamal-Bahl and Puckett are employees of COVIA Health Solutions, a consulting firm that provides services to biopharmaceutical clients, trade organizations, and foundations. Kamal-Bahl holds stock in Merck and Pfizer. Singh is an employee of Novartis Pharmaceuticals. Willke received personal fees from COVIA Health Solutions for work on the manuscript.

Comparative effectiveness and safety of pharmaceuticals assessed in observational studies compared with randomized controlled trials.

BACKGROUND: There have been ongoing efforts to understand when and how data from observational studies can be applied to clinical and regulatory decision making. The objective of this review was to assess the comparability of relative treatment effects of pharmaceuticals from observational studies and randomized controlled trials (RCTs). METHODS: We searched PubMed and Embase for systematic literature reviews published between January 1, 1990, and January 31, 2020, that reported relative treatment effects of pharmaceuticals from both observational studies and RCTs. We extracted pooled relative effect estimates from observational studies and RCTs for each outcome, intervention-comparator, or indication assessed in the reviews. We calculated the ratio of the relative effect estimate from observational studies over that from RCTs, along with the corresponding 95% confidence interval (CI) for each pair of pooled RCT and observational study estimates, and we evaluated the consistency in relative treatment effects. RESULTS: Thirty systematic reviews across 7 therapeutic areas were identified from the literature. We analyzed 74 pairs of pooled relative effect estimates from RCTs and observational studies from 29 reviews. There was no statistically significant difference (based on the 95% CI) in relative effect estimates between RCTs and observational studies in 79.7% of pairs. There was an extreme difference (ratio < 0.7 or > 1.43) in 43.2% of pairs, and, in 17.6% of pairs, there was a significant difference and the estimates pointed in opposite directions. CONCLUSIONS: Overall, our review shows that while there is no significant difference in the relative risk ratios between the majority of RCTs and observational studies compared, there is significant variation in about 20% of comparisons. The source of this variation should be the subject of further inquiry to elucidate how much of the variation is due to differences in patient populations versus biased estimates arising from issues with study design or analytical/statistical methods.

Linked Pharmacometric-Pharmacoeconomic Modeling and Simulation in Clinical Drug Development.

Market access and pricing of pharmaceuticals are increasingly contingent on the ability to demonstrate comparative effectiveness and cost-effectiveness. As such, it is widely recognized that predictions of the economic potential of drug candidates in development could inform decisions across the product life cycle. This may be challenging when safety and efficacy profiles in terms of the relevant clinical outcomes are unknown or highly uncertain early in product development. Linking pharmacometrics and pharmacoeconomics, such that outputs from pharmacometric models serve as inputs to pharmacoeconomic models, may provide a framework for extrapolating from early-phase studies to predict economic outcomes and characterize decision uncertainty. This article reviews the published studies that have implemented this methodology and used simulation to inform drug development decisions and/or to optimize the use of drug treatments. Some of the key practical issues involved in linking pharmacometrics and pharmacoeconomics, including the choice of final outcome measures, methods of incorporating evidence on comparator treatments, approaches to handling multiple intermediate end points, approaches to quantifying uncertainty, and issues of model validation are also discussed. Finally, we have considered the potential barriers that may have limited the adoption of this methodology and suggest that closer alignment between the disciplines of clinical pharmacology, pharmacometrics, and pharmacoeconomics, may help to realize the potential benefits associated with linked pharmacometric-pharmacoeconomic modeling and simulation.

Improving Transparency to Build Trust in Real-World Secondary Data Studies for Hypothesis Testing-Why, What, and How: Recommendations and a Road Map from the Real-World Evidence Transparency Initiative.

Real-world data (RWD) and the derivations of these data into real-world evidence (RWE) are rapidly expanding from informing healthcare decisions at the patient and health system level to influencing major health policy decisions, including regulatory approvals and coverage. Recent examples include the approval of palbociclib in combination with endocrine therapy for male breast cancer and the inclusion of RWE in the label of paliperidone palmitate for schizophrenia. This interest has created an urgency to develop processes that promote trust in the evidence-generation process. Key stakeholders and decision-makers include patients and their healthcare providers; learning health systems; health technology assessment bodies and payers; pharmacoepidemiologists and other clinical reseachers, and policy makers interested in bioethical and regulatory issues. A key to optimal uptake of RWE is transparency of the research process to enable decision-makers to evaluate the quality of the methods used and the applicability of the evidence that results from the RWE studies. Registration of RWE studies-particularly for hypothesis evaluating treatment effectiveness (HETE) studies-has been proposed to improve transparency, trust, and research replicability. Although registration would not guarantee better RWE studies would be conducted, it would encourage the prospective disclosure of study plans, timing, and rationale for modifications. A joint task force of the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) and the International Society for Pharmacoepidemiology (ISPE) recommended that investigators preregister their RWE studies and post their study protocols in a publicly available forum before starting studies to reduce publication bias and improve the transparency of research methods. Recognizing that published recommendations alone are insufficient, especially without accessible registration options and with no incentives, a group of experts gathered on February 25 and 26, 2019, in National Harbor, Maryland, to explore the structural and practical challenges to the successful implementation of the recommendations of the ISPOR/ISPE task force for preregistration. This positioning article describes a plan for making registration of HETE RWE studies routine. The plan includes specifying the rationale for registering HETE RWE studies, the studies that should be registered, where and when these studies should be registered, how and when analytic deviations from protocols should be reported, how and when to publish results, and incentives to encourage registration. Table 1 summarizes the rationale, goals, and potential solutions that increase transparency, in addition to unique concerns about secondary data studies. Definitions of terms used throughout this report are provided in Table 2.

Improving transparency to build trust in real-world secondary data studies for hypothesis testing-Why, what, and how: recommendations and a road map from the real-world evidence transparency initiative.

Real-world data (RWD) and the derivations of these data into real-world evidence (RWE) are rapidly expanding from informing healthcare decisions at the patient and health system level to influencing major health policy decisions, including regulatory approvals and coverage. Recent examples include the approval of palbociclib in combination with endocrine therapy for male breast cancer and the inclusion of RWE in the label of paliperidone palmitate for schizophrenia. This interest has created an urgency to develop processes that promote trust in the evidence-generation process. Key stakeholders and decision-makers include patients and their healthcare providers; learning health systems; health technology assessment bodies and payers; pharmacoepidemiologists and other clinical reseachers, and policy makers interested in bioethical and regulatory issues. A key to optimal uptake of RWE is transparency of the research process to enable decision-makers to evaluate the quality of the methods used and the applicability of the evidence that results from the RWE studies. Registration of RWE studies-particularly for hypothesis evaluating treatment effectiveness (HETE) studies-has been proposed to improve transparency, trust, and research replicability. Although registration would not guarantee better RWE studies would be conducted, it would encourage the prospective disclosure of study plans, timing, and rationale for modifications. A joint task force of the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) and the International Society for Pharmacoepidemiology (ISPE) recommended that investigators preregister their RWE studies and post their study protocols in a publicly available forum before starting studies to reduce publication bias and improve the transparency of research methods. Recognizing that published recommendations alone are insufficient, especially without accessible registration options and with no incentives, a group of experts gathered on February 25 and 26, 2019, in National Harbor, Maryland, to explore the structural and practical challenges to the successful implementation of the recommendations of the ISPOR/ISPE task force for preregistration. This positioning article describes a plan for making registration of HETE RWE studies routine. The plan includes specifying the rationale for registering HETE RWE studies, the studies that should be registered, where and when these studies should be registered, how and when analytic deviations from protocols should be reported, how and when to publish results, and incentives to encourage registration. Table 1 summarizes the rationale, goals, and potential solutions that increase transparency, in addition to unique concerns about secondary data studies. Definitions of terms used throughout this report are provided in Table 2.

Criteria and Process for Initiating and Developing an ISPOR Good Practices Task Force Report.

The International Society for Pharmacoeconomics and Outcomes Research (ISPOR)'s "Good Practices Task Force" reports are highly cited, multistakeholder perspective expert guidance reports that reflect international standards for health economics and outcomes research (HEOR) and their use in healthcare decision making. In this report, we discuss the criteria, development, and evaluation/consensus review and approval process for initiating a task force. The rationale for a task force must include a justification, including why this good practice guidance is important and its potential impact on the scientific community. The criteria include: (1) necessity (why is this task force required?); (2) a methodology-oriented focus (focus on research methods, approaches, analysis, interpretation, and dissemination); (3) relevance (to ISPOR's mission and its members); (4) durability over time; (5) broad applicability; and 6) an evidence-based approach. In addition, the proposal must be a priority specifically for ISPOR. These reports are valuable to researchers, academics, students, health technology assessors, medical technology developers and service providers, those working in other commercial entities, regulators, and payers. These stakeholder perspectives are represented in task force membership to ensure the report's overall usefulness and relevance to the global ISPOR membership. We hope that this discussion will bring transparency to the process of initiating, approving, and producing these task force reports and encourage participation from a diverse range of experts within and outside ISPOR.

Reflections on the ISPOR Special Task Force on U.S. Value Frameworks: Implications of a Health Economics Approach for Managed Care Pharmacy.

In 2016, The Professional Society for Health Economics and Outcomes Research (ISPOR) formed a special task force (STF) to review approaches and methods to support the definition and use of high-quality U.S. value frameworks. As the leadership group of that initiative, we present our perspective, focusing on implications for the managed care pharmacy community. Our reflections are organized by 9 key observations and conclude with a summary recommendation. We begin by emphasizing the importance of distinguishing among "perspectives" and "decision contexts." Possible perspectives include patient, payer, provider, health care sector, and societal. Decision contexts range from formulary inclusion to guideline development to clinical shared decision making, and multiple perspectives can be taken on each of these decisions. The STF focused on value in the context of including a new medicine in a formulary and, thus, health plan, using a health economics approach that compares marginal benefit (gross value) and marginal (opportunity) cost, yielding the net value. Health care is unique compared with other markets. While economists often use market purchases as indicators of value, they also recognize that this does not work well in health care, since most patent-protected drugs are covered by insurance. To assess the likely health and economic impact, health economists often employ cost-effectiveness analysis, using the quality-adjusted life-year (QALY), a metric that combines mortality and morbidity into a single preference-based index. We strongly endorse the STF's recommendation that payers should use the cost-per-QALY metric as a starting point. However, like the STF, and many of those stakeholders who provided input, we recognize that this metric has some limitations in theory and in practice. Nonetheless, the cost-per-QALY metric is a pragmatic tool that can be augmented to address some of its limitations by integrating other elements of value, particularly those related to uncertainty, such as financial risk protection, health risk protection, the value of hope, real option value, and the value of knowing. The resulting adjusted ratio can be compared with a willingness-to-pay threshold or combined in a measure of net monetary benefit. Alternatively, the array of elements can be valued using multi-criteria decision analysis. We end with the key recommendation that further development and testing of these promising approaches is needed to improve the deliberative process of health technology assessment. DISCLOSURES: No outside funding supported the writing of this article. The authors are leaders of the ISPOR Special Task Force on U.S. Value Frameworks. Willke is employed by ISPOR. Garrison and Neumann have nothing to disclose. The opinions expressed in this article should be considered as belonging only to the authors.

An empirical tool for estimating the share of unmet need due to healthcare inefficiencies, suboptimal access, and lack of effective technologies.

BACKGROUND: Although there has been growing attention to the measurement of unmet need, which is the overall epidemiological burden of disease, current measures ignore the burden that could be eliminated from technological advances or more effective use of current technologies. METHODS: We developed a conceptual framework and empirical tool that separates unmet need from met need and subcategorizes the causes of unmet need into suboptimal access to and ineffective use of current technologies and lack of current technologies. Statistical models were used to model the relationship between health-related quality of life (HR-QOL) and treatment utilization using data from the National Health and Wellness Survey (NHWS). Predicted HR-QOL was combined with prevalence data from the Global Burden of Disease Study (GBD) to estimate met need and the causes of unmet need due to morbidity in the US and EU5 for five diseases: rheumatoid arthritis, breast cancer, Parkinson's disease, hepatitis C, and chronic obstructive pulmonary disease (COPD). RESULTS: HR-QOL was positively correlated with adherence to medication and patient-perceived quality and negatively correlated with financial barriers. Met need was substantial across all disease and regions, although significant unmet need remains. While the majority of unmet need was driven by lack of technologies rather than ineffective use of current technologies, there was considerable variation across diseases and regions. Overall unmet need was largest for COPD, which had the highest prevalence of all diseases in this study. CONCLUSION: We developed a methodology that can inform decisions about which diseases to invest in and whether those investments should focus on improving access to currently available technologies or inventing new technologies.

A Health Economics Approach to US Value Assessment Frameworks-Introduction: An ISPOR Special Task Force Report [1].

Concerns about rising spending on prescription drugs and other areas of health care have led to multiple initiatives in the United States designed to measure and communicate the value of pharmaceuticals and other technologies for decision making. In this section we introduce the work of the International Society for Pharmacoeconomics and Outcomes Research Special Task Force on US Value Assessment Frameworks formed to review relevant perspectives and appropriate approaches and methods to support the definition and use of high-quality value frameworks. The Special Task Force was part of the International Society for Pharmacoeconomics and Outcomes Research Initiative on US Value Assessment Frameworks, which enlisted the expertise of leading health economists, concentrating on what the field of health economics can provide to help inform the development and use of value assessment frameworks. We focus on five value framework initiatives: the American College of Cardiology/American Heart Association, the American Society of Clinical Oncology, the Institute for Clinical and Economic Review, the Memorial Sloan Kettering Cancer Center, and the National Comprehensive Cancer Network. These entities differ in their missions, scope of activities, and methodological approaches. Because they are gaining visibility and some traction in the United States, it is essential to scrutinize whether the frameworks use approaches that are transparent as well as conceptually and methodologically sound. Our objectives were to describe the conceptual bases for value and its use in decision making, critically examine existing value frameworks, discuss the importance of sound conceptual underpinning, identify key elements of value relevant to specific decision contexts, and recommend good practice in value definition and implementation as well as areas for further research.

An Overview of Value, Perspective, and Decision Context-A Health Economics Approach: An ISPOR Special Task Force Report [2].

The second section of our Special Task Force builds on the discussion of value and perspective in the previous article of the report by 1) defining a health economics approach to the concept of value in health care systems; 2) discussing the relationship of value to perspective and decision context, that is, how recently proposed value frameworks vary by the types of decisions being made and by the stakeholders involved; 3) describing the patient perspective on value because the patient is a key stakeholder, but one also wearing the hat of a health insurance purchaser; and 4) discussing how value is relevant in the market-based US system of mixed private and public insurance, and differs from its use in single-payer systems. The five recent value frameworks that motivated this report vary in the types of decisions they intend to inform, ranging from coverage, access, and pricing decisions to those defining appropriate clinical pathways and to supporting provider-clinician shared decision making. Each of these value frameworks must be evaluated in its own decision context for its own objectives. Existing guidelines for cost-effectiveness analysis emphasize the importance of clearly specifying the perspective from which the analysis is undertaken. Relevant perspectives may include, among others, 1) the health plan enrollee, 2) the patient, 3) the health plan manager, 4) the provider, 5) the technology manufacturer, 6) the specialty society, 7) government regulators, or 8) society as a whole. A valid and informative cost-effectiveness analysis could be conducted from the perspective of any of these stakeholders, depending on the decision context.

Review of Recent US Value Frameworks-A Health Economics Approach: An ISPOR Special Task Force Report [6].

The sixth section of our Special Task Force (STF) report reviews and comments on recent US-oriented value assessment frameworks, specifically those published by the American College of Cardiology/American Heart Association, the Institute for Clinical and Economic Research, the American Society of Clinical Oncology, the National Comprehensive Cancer Network, and the Memorial Sloan Kettering Cancer Center. We review published commentaries that address the validity, reliability, and conceptual underpinnings of these frameworks. We find common themes of critique regarding the strengths and limitations across frameworks. Particular shortcomings of some frameworks pose greater threats to their face validity and utility compared with others. The most significant limitations include lack of clear perspective (e.g., patient vs. health plan) and poor transparency in accounting for costs and benefits. We then review how each framework adheres to core STF recommendations, with particular emphasis on whether the framework can be used to support coverage decisions by health insurers, and whether it adheres to core principles of cost-effectiveness analysis. The Institute for Clinical and Economic Research framework most closely adheres to core STF recommendations. Others have significant limitations that vary widely from framework to framework. We also review how the frameworks follow STF recommendations for addressing potentially relevant issues beyond cost-effectiveness analysis - for example, equity in resource allocation and patient heterogeneity. Finally, we review whether and how each framework uses value thresholds and addresses affordability concerns. We conclude with suggestions for further research, particularly in the areas of testing the measurement and use of novel elements of value and deliberative processes.