A Structured Process to Identify Fit-for-Purpose Study Design and Data to Generate Valid and Transparent Real-World Evidence for Regulatory Uses.

Generating evidence from real-world data requires fit-for-purpose study design and data. In addition to validity, decision makers require transparency in the reasoning that underlies study design and data source decisions. The 2019 Structured Preapproval and Postapproval Comparative Study Design Framework to Generate Valid and Transparent Real-World Evidence (SPACE) and the 2021 Structured Process to Identify Fit-For-Purpose Data (SPIFD)-intended to be used together-provide a step-by-step guide to identify decision grade, fit-for-purpose study design and data. In this update (referred to as "SPIFD2" to encompass both the design and data aspects) we provide an update to these frameworks that combines the templates into one, more explicitly calls for articulation of the hypothetical target trial and sources of bias that may arise in the real-world emulation, and provides explicit references to the Structured Template and Reporting Tool for Real-World Evidence (STaRT-RWE) tables that we suggest using immediately after invoking the SPIFD2 framework. Following the steps recommended in the SPIFD2 process requires due diligence on the part of the researcher to ensure that every aspect of study design and data selection is rationalized and supported by evidence. The resulting stepwise documentation enables reproducibility and clear communication with decision makers, and it increases the likelihood that the evidence generated is valid, fit-for-purpose, and sufficient to support healthcare and regulatory decisions.

The Structured Process to Identify Fit-For-Purpose Data: A Data Feasibility Assessment Framework.

To complement real-world evidence (RWE) guidelines, the 2019 Structured Preapproval and Postapproval Comparative study design framework to generate valid and transparent real-world Evidence (SPACE) framework elucidated a process for designing valid and transparent real-world studies. As an extension to SPACE, here, we provide a structured framework for conducting feasibility assessments-a step-by-step guide to identify decision grade, fit-for-purpose data, which complements the United States Food and Drug Administration (FDA)'s framework for a RWE program. The process was informed by our collective experience conducting systematic feasibility assessments of existing data sources for pharmacoepidemiology studies to support regulatory decisions. Used with the SPACE framework, the Structured Process to Identify Fit-For-Purpose Data (SPIFD) provides a systematic process for conducting feasibility assessments to determine if a data source is fit for decision making, helping ensure justification and transparency throughout study development, from articulation of a specific and meaningful research question to identification of fit-for-purpose data and study design.

Integration of Real-World Data and Genetics to Support Target Identification and Validation.

Even modest improvements in the probability of success of selecting drug targets which are ultimately approved can substantially reduce the costs of research and development. Drug targets with human genetic evidence of disease association are twice as likely to lead to approved drugs. A key enabler of identifying and validating these genetically validated targets is access to association results from genome-wide genotyping, whole-exome sequencing, and whole-genome sequencing studies with observable traits (often diseases) across large numbers of individuals. Today, linkage between genotype and real-world data (RWD) provides significant opportunities to not only increase the statistical power of genome-wide association studies by ascertaining additional cases for diseases of interest, but also to improve diversity and coverage of association studies across the disease phenome. As RWD-genetics linked resources continue to grow in diversity of participants, breadth of data captured, length of observation, and number of participants, there is a greater need to leverage the experience of RWD experts, clinicians, and highly experienced geneticists together to understand which lessons and frameworks from general research using RWD sources are relevant to improve genetics-driven drug discovery and development. This paper describes new challenges and opportunities for phenotypes enabled by diverse RWD sources, considerations in the use of RWD phenotypes for disease gene identification across the disease phenome, and challenges and opportunities in leveraging RWD phenotypes in target validation. The paper concludes with views on the future directions for phenotype development using RWD, and key questions requiring further research and development to advance this nascent field.

A Structured Preapproval and Postapproval Comparative Study Design Framework to Generate Valid and Transparent Real-World Evidence for Regulatory Decisions.

Real-world evidence provides important information about the effects of medicines in routine clinical practice. To engender trust that evidence generated for regulatory purposes is sufficiently valid, transparency in the reasoning that underlies study design decisions is critical. Building on existing guidance and frameworks, we developed the Structured Preapproval and Postapproval Comparative study design framework to generate valid and transparent real-world Evidence (SPACE) as a process for identifying design elements and minimal criteria for feasibility and validity concerns, and for documenting decisions. Starting with an articulated research question, we identify key components of the randomized controlled trial needed to maximize validity, and pragmatic choices are considered when required. A causal diagram is used to justify the variables identified for confounding control, and key decisions, assumptions, and evidence are captured in a structured way. In this way, SPACE may improve dialogue and build trust among healthcare providers, patients, regulators, and researchers.

Do FDA label changes work? Assessment of the 2010 class label change for proton pump inhibitors using the Sentinel System's analytic tools.

PURPOSE: To pilot use of the US Food and Drug Administration's (FDA's) Sentinel System data and analytic tools by a non-FDA stakeholder through the Innovation in Medical Evidence Development and Surveillance system of the Reagan Udall Foundation. We evaluated the US FDA 2010 proton pump inhibitor (PPI) class label change that warned of increased risk of bone fracture, to use PPIs for the lowest dose and shortest duration, and to manage bone status for those at risk for osteoporosis. METHODS: The cohort consisted of adults aged 18 years or older prescribed PPIs without fracture risk factors. We evaluated incident and prevalent uses of the 8 PPIs noted in the label change. Outcomes evaluated before and after label change were PPI dispensing patterns, incident fractures, and osteoporosis screening or interventions. Consistent with FDA use of descriptive tools, we did not include direct comparisons or statistical testing. RESULTS: There were 1 488 869 and 2 224 420 incident PPI users in the before [PRE] and after [POST] periods, respectively. Users with 1 year or more of exposure decreased (8.4% vs 7.5%), as did mean days supplied/user (130.4 to 113.7 d among all users and 830.8 to 645.4 d among users with 1 y or more of exposure). Osteoporosis screening and interventions did not appear to increase, but the proportion of patients with fractures decreased (4.4% vs 3.1%). Prevalent user results were similar. CONCLUSIONS: This analysis demonstrated the ability to use Sentinel tools to assess the effectiveness of a label change and accompanying communication at the population level and suggests an influence on subsequent dispensing behavior.

Quantification of the risk of liver injury associated with flucloxacillin: a UK population-based cohort study.

Background: Flucloxacillin is an established cause of liver injury. Despite this, there are a lack of published data on both the strength of association after adjusting for potential confounders, and the absolute incidence among different subgroups of patients. Objectives: To assess the relative and absolute risks of liver injury following exposure to flucloxacillin and identify subgroups at potentially increased risk. Methods: A cohort study between 1 January 2000 and 1 January 2012 using the UK Clinical Practice Research Datalink, including 1 046 699 people with a first prescription for flucloxacillin (861 962) or oxytetracycline (184 737). Absolute risks of experiencing both symptom-defined (jaundice) and laboratory-confirmed liver injury within 1-45 and 46-90 days of antibiotic initiation were estimated. Multivariable logistic regression was used to estimate 1-45 day relative effects. Results: There were 183 symptom-defined cases (160 prescribed flucloxacillin) and 108 laboratory-confirmed cases (102 flucloxacillin). The 1-45 day adjusted risk ratio for laboratory-confirmed injury was 5.22 (95% CI 1.64-16.62) comparing flucloxacillin with oxytetracycline use. The 1-45 day risk of laboratory-confirmed liver injury was 8.47 per 100 000 people prescribed flucloxacillin (95% CI 6.64-10.65). People who received consecutive flucloxacillin prescriptions had a 1-45 day risk of jaundice of 39.00 per 100 000 (95% CI 26.85-54.77), while those aged >70 receiving consecutive prescriptions had a risk of 110.57 per 100 000 (95% CI 70.86-164.48). Conclusions: The short-term risk of laboratory-confirmed liver injury was >5-fold higher after a flucloxacillin prescription than an oxytetracycline prescription. The risk of flucloxacillin-induced liver injury is particularly high within those aged >70 and those who receive multiple flucloxacillin prescriptions. The stratified risk estimates from this study could help guide clinical care.

Optimising case detection within UK electronic health records: use of multiple linked databases for detecting liver injury.

OBJECTIVES: We aimed to create a 'multidatabase' algorithm for identification of cholestatic liver injury using multiple linked UK databases, before (1) assessing the improvement in case ascertainment compared to using a single database and (2) developing a new single-database case-definition algorithm, validated against the multidatabase algorithm. DESIGN: Method development for case ascertainment. SETTING: Three UK population-based electronic health record databases: the UK Clinical Practice Research Datalink (CPRD), the UK Hospital Episodes Statistics (HES) database and the UK Office of National Statistics (ONS) mortality database. PARTICIPANTS: 16 040 people over the age of 18 years with linked CPRD-HES records indicating potential cholestatic liver injury between 1 January 2000 and 1 January 2013. PRIMARY OUTCOME MEASURES: (1) The number of cases of cholestatic liver injury detected by the multidatabase algorithm. (2) The relative contribution of each data source to multidatabase case status. (3) The ability of the new single-database algorithm to discriminate multidatabase algorithm case status. RESULTS: Within the multidatabase case identification algorithm, 4033 of 16 040 potential cases (25%) were identified as definite cases based on CPRD data. HES data allowed possible cases to be discriminated from unlikely cases (947 of 16 040, 6%), but only facilitated identification of 1 definite case. ONS data did not contribute to case definition. The new single-database (CPRD-only) algorithm had a very good ability to discriminate multidatabase case status (area under the receiver operator characteristic curve 0.95). CONCLUSIONS: CPRD-HES-ONS linkage confers minimal improvement in cholestatic liver injury case ascertainment compared to using CPRD data alone, and a multidatabase algorithm provides little additional information for validation of a CPRD-only algorithm. The availability of laboratory test results within CPRD but not HES means that algorithms based on CPRD-HES-linked data may not always be merited for studies of liver injury, or for other outcomes relying primarily on laboratory test results.

The IMI PROTECT project: purpose, organizational structure, and procedures.

The Pharmacoepidemiological Research on Outcomes of Therapeutics by a European ConsorTium (PROTECT) initiative was a collaborative European project that sought to address limitations of current methods in the field of pharmacoepidemiology and pharmacovigilance. Initiated in 2009 and ending in 2015, PROTECT was part of the Innovative Medicines Initiative, a joint undertaking by the European Union and pharmaceutical industry. Thirty-five partners including academics, regulators, small and medium enterprises, and European Federation of Pharmaceuticals Industries and Associations companies contributed to PROTECT. Two work packages within PROTECT implemented research examining the extent to which differences in the study design, methodology, and choice of data source can contribute to producing discrepant results from observational studies on drug safety. To evaluate the effect of these differences, the project applied different designs and analytic methodology for six drug-adverse event pairs across several electronic healthcare databases and registries. This papers introduces the organizational structure and procedures of PROTECT, including how drug-adverse event and data sources were selected, study design and analyses documents were developed, and results managed centrally.

Multi-centre, multi-database studies with common protocols: lessons learnt from the IMI PROTECT project.

PURPOSE: To assess the impact of a variety of methodological parameters on the association between six drug classes and five key adverse events in multiple databases. METHODS: The selection of Drug-Adverse Event pairs was based on public health impact, regulatory relevance, and the possibility to study a broad range of methodological issues. Common protocols and data analytical specifications were jointly developed and independently and blindly executed in different databases in Europe with replications in the same and different databases. RESULTS: The association between antibiotics and acute liver injury, benzodiazepines and hip fracture, antidepressants and hip fracture, inhaled long-acting beta2-agonists and acute myocardial infarction was consistent in direction across multiple designs, databases and methods to control for confounding. Some variation in magnitude of the associations was observed depending on design, exposure and outcome definitions, but none of the differences were statistically significant. The association between anti-epileptics and suicidality was inconsistent across the UK CPRD, Danish National registries and the French PGRx system. Calcium channel blockers were not associated with the risk of cancer in the UK CPRD, and this was consistent across different classes of calcium channel blockers, cumulative durations of use up to >10 years and different types of cancer. CONCLUSIONS: A network for observational drug effect studies allowing the execution of common protocols in multiple databases was created. Increased consistency of findings across multiple designs and databases in different countries will increase confidence in findings from observational drug research and benefit/risk assessment of medicines.

Computer-assisted expert case definition in electronic health records.

PURPOSE: To describe how computer-assisted presentation of case data can lead experts to infer machine-implementable rules for case definition in electronic health records. As an illustration the technique has been applied to obtain a definition of acute liver dysfunction (ALD) in persons with inflammatory bowel disease (IBD). METHODS: The technique consists of repeatedly sampling new batches of case candidates from an enriched pool of persons meeting presumed minimal inclusion criteria, classifying the candidates by a machine-implementable candidate rule and by a human expert, and then updating the rule so that it captures new distinctions introduced by the expert. Iteration continues until an update results in an acceptably small number of changes to form a final case definition. RESULTS: The technique was applied to structured data and terms derived by natural language processing from text records in 29,336 adults with IBD. Over three rounds the technique led to rules with increasing predictive value, as the experts identified exceptions, and increasing sensitivity, as the experts identified missing inclusion criteria. In the final rule inclusion and exclusion terms were often keyed to an ALD onset date. When compared against clinical review in an independent test round, the derived final case definition had a sensitivity of 92% and a positive predictive value of 79%. CONCLUSION: An iterative technique of machine-supported expert review can yield a case definition that accommodates available data, incorporates pre-existing medical knowledge, is transparent and is open to continuous improvement. The expert updates to rules may be informative in themselves. In this limited setting, the final case definition for ALD performed better than previous, published attempts using expert definitions.

Exposure to benzodiazepines (anxiolytics, hypnotics and related drugs) in seven European electronic healthcare databases: a cross-national descriptive study from the PROTECT-EU Project.

PURPOSE: Studies on drug utilization usually do not allow direct cross-national comparisons because of differences in the respective applied methods. This study aimed to compare time trends in BZDs prescribing by applying a common protocol and analyses plan in seven European electronic healthcare databases. METHODS: Crude and standardized prevalence rates of drug prescribing from 2001-2009 were calculated in databases from Spain, United Kingdon (UK), The Netherlands, Germany and Denmark. Prevalence was stratified by age, sex, BZD type [(using ATC codes), i.e. BZD-anxiolytics BZD-hypnotics, BZD-related drugs and clomethiazole], indication and number of prescription. RESULTS: Crude prevalence rates of BZDs prescribing ranged from 570 to 1700 per 10,000 person-years over the study period. Standardization by age and sex did not substantially change the differences. Standardized prevalence rates increased in the Spanish (+13%) and UK databases (+2% and +8%) over the study period, while they decreased in the Dutch databases (-4% and -22%), the German (-12%) and Danish (-26%) database. Prevalence of anxiolytics outweighed that of hypnotics in the Spanish, Dutch and Bavarian databases, but the reverse was shown in the UK and Danish databases. Prevalence rates consistently increased with age and were two-fold higher in women than in men in all databases. A median of 18% of users received 10 or more prescriptions in 2008. CONCLUSION: Although similar methods were applied, the prevalence of BZD prescribing varied considerably across different populations. Clinical factors related to BZDs and characteristics of the databases may explain these differences.

Acute nonarteritic anterior ischemic optic neuropathy and exposure to phosphodiesterase type 5 inhibitors.

INTRODUCTION: Nonarteritic anterior ischemic optic neuropathy (NAION), a rare visual disorder, has been reported in men using phosphodiesterase type 5 inhibitors (PDE5i) for erectile dysfunction. AIM: We examined whether intermittent use of PDE5i is associated with acute NAION onset within approximately five half-lives following drug ingestion. METHODS: One hundred two ophthalmology centers in the United States and Europe identified potential cases of NAION. An expert adjudication committee conducted a blind review of the records of those with recent PDE5i use to classify cases as Definite, Possible, or not NAION. Subjects provided information on PDEi use via telephone interview. Each NAION case's PDE5i exposure immediately prior to onset was compared against his recent patterns of use in an observational case-crossover design. A sample size of 40 cases with intermittent PDE5i exposure in the 30 days prior to NAION onset was needed to detect an odds ratio (OR) of 3.0 with 80% power. MAIN OUTCOME MEASURES: The daily relative risk for acute NAION on days within five half-lives of PDE5i use vs. other days was estimated via an OR obtained from conditional logistic regression. RESULTS: Among 43 Definite NAION cases with PDE5i exposure in the prior 30 days, the OR was 2.15 (95% confidence interval [CI]: 1.06, 4.34). When 21 Possible NAION cases were included (n = 64), the OR was 2.36 (95% CI: 1.33, 4.19). CONCLUSIONS: We found an approximately twofold increased risk of acute NAION within five half-lives of PDE5i use compared with use in a more prior time period. Bias from inaccurate recall of exposure was unlikely to have substantially affected the results. Based on our results, we estimate that weekly use of PDE5i adds three NAION cases per 100,000 men 50 years and older annually.

Completed and attempted suicides among 18,154 subjects with schizophrenia included in a large simple trial.

OBJECTIVE: To characterize subgroups of subjects with schizophrenia from the Ziprasidone Observational Study of Cardiac Outcomes (ZODIAC) trial who either completed or attempted suicide and those who did not. METHOD: The ZODIAC, conducted between February 2002 and March 2007, was an open-label, randomized, large simple trial of patients with schizophrenia (N = 18,154) followed up for 1 year by unblinded investigators providing usual care in 18 countries; the primary outcome measure was nonsuicide mortality. Every report on a completed or attempted suicide was independently adjudicated using a predefined algorithm. Primary analysis for the current report examined the association between completed or attempted suicides and the baseline variables using descriptive statistics and multivariate logistic regression models. Usage of "hard" or "soft" methods for attempted or completed suicide and distribution of suicide-related events by geographical region were also summarized. RESULTS: Overall incidences of subjects who either completed (35/18,154) or attempted (108/18,154) suicide were low, as were rates per person-time on assigned treatment analysis (0.24 for completed and 0.74 for attempted suicides per 100 person-years of exposure). The highest suicide-related mortality was seen among subjects recently diagnosed with schizophrenia. Among all potential baseline risk factors for completed suicide examined, the variables most associated with completed suicide were history of suicide attempts (OR = 2.6; 95% CI, 1.33-5.12) and usage of antidepressant medication (OR = 3.5; 95% CI, 0.84-14.85). History of > 5 hospitalizations in the past (OR = 2.1; 95% CI, 1.35-3.31) and history of suicide attempts (OR = 5.0; 95% CI, 3.21-7.76) were the variables most associated with attempted suicide among potential baseline risk factors for suicide attempts. CONCLUSIONS: Our results, obtained in a large prospective randomized study, confirm current clinical understanding regarding completed or attempted suicide in schizophrenia and the associated risk factors. TRIAL REGISTRATION: ClinicalTrials.gov identifier: NCT00418171.

Bridging differences in outcomes of pharmacoepidemiological studies: design and first results of the PROTECT project.

BACKGROUND: Observational pharmacoepidemiological (PE) studies on drug safety have produced discrepant results that may be due to differences in design, conduct and analysis. PURPOSE: The pharmacoepidemiology work-package (WP2) of the Pharmacoepidemiological Research on Outcomes of Therapeutics by a European ConsorTium (PROTECT) project aims at developing, testing and disseminating methodological standards for design, conduct and analysis of pharmacoepidemiological studies applicable to different safety issues using different databases across European countries. This article describes the selection of the safety issues and the description of the databases to be systematically studied. METHODS: Based on two consensus meetings and a literature search, we selected five drug-adverse event (AE) pairs to be evaluated in different databases. This selection was done according to pre-defined criteria such as regulatory and public health impact, and the potential to investigate a broad range of methodological issues. RESULTS: The selected drug-AE pairs are: 1) inhaled long-acting beta-2 agonists and acute myocardial infarction; 2) antimicrobials and acute liver injury; 3) antidepressants and/or benzodiazepines and hip fracture; 4) anticonvulsants and suicide/suicide attempts; and 5) calcium channel blockers and malignancies. Six European databases, that will be used to evaluate the drug-AE pairs retrospectively, are also described. CONCLUSION: The selected drug-AE pairs will be evaluated in PE studies using common protocols. Based on consistencies and discrepancies of these studies, a framework for guiding methodological choices will be developed. This will increase the usefulness and reliability of PE studies for benefit-risk assessment and decision-making.

Development of predictive genetic tests for improving the safety of new medicines: the utilization of routinely collected electronic health records.

Serious adverse drug reactions are an important cause of hospitalization and can result in the withdrawal of licensed drugs. Genetic variation has been shown to influence adverse drug reaction susceptibility, and predictive genetic tests have been developed for a limited number of adverse drug reactions. The identification of patients with adverse drug reactions, obtaining samples for genetic analysis and rigorous evaluation of clinical test effectiveness represent significant challenges to predictive genetic test development. Using the example of serious drug-induced liver injury, we illustrate how a database of routinely collected electronic health records (EHRs) could be used to overcome these barriers by facilitating rapid recruitment to genome-wide association studies and supporting efficient randomized controlled trials of predictive genetic test effectiveness.

Lessons learned in the conduct of a global, large simple trial of treatments indicated for schizophrenia.

Large, "practical" or streamlined trials (LSTs) are used to study the effectiveness and/or safety of medicines in real world settings with minimal study imposed interventions. While LSTs have benefits over traditional randomized clinical trials and observational studies, there are inherent challenges to their conduct. Enrollment and follow-up of a large study sample of patients with mental illness pose a particular difficulty. To assist in overcoming operational barriers in future LSTs in psychiatry, this paper describes the recruitment and observational follow-up strategies used for the ZODIAC study, an international, open-label LST, which followed 18,239 persons randomly assigned to one of two treatments indicated for schizophrenia for 1 year. ZODIAC enrolled patients in 18 countries in North America, South America, Europe, and Asia using broad study entry criteria and required minimal clinical care intervention. Recruitment of adequate numbers and continued engagement of both study centers and subjects were significant challenges. Strategies implemented to mitigate these in ZODIAC include global study expansion, study branding, field coordinator and site relations programs, monthly site newsletters, collection of alternate contact information, conduct of national death index (NDI) searches, and frequent sponsor, contract research organization (CRO) and site interaction to share best practices and address recruitment challenges quickly. We conclude that conduct of large LSTs in psychiatric patient populations is feasible, but importantly, realistic site recruitment goals and maintaining site engagement are key factors that need to be considered in early study planning and conduct.

Photoperiodic regulation of the seasonal pattern of photosynthetic capacity and the implications for carbon cycling.

Although temperature is an important driver of seasonal changes in photosynthetic physiology, photoperiod also regulates leaf activity. Climate change will extend growing seasons if temperature cues predominate, but photoperiod-controlled species will show limited responsiveness to warming. We show that photoperiod explains more seasonal variation in photosynthetic activity across 23 tree species than temperature. Although leaves remain green, photosynthetic capacity peaks just after summer solstice and declines with decreasing photoperiod, before air temperatures peak. In support of these findings, saplings grown at constant temperature but exposed to an extended photoperiod maintained high photosynthetic capacity, but photosynthetic activity declined in saplings experiencing a naturally shortening photoperiod; leaves remained equally green in both treatments. Incorporating a photoperiodic correction of photosynthetic physiology into a global-scale terrestrial carbon-cycle model significantly improves predictions of seasonal atmospheric CO(2) cycling, demonstrating the benefit of such a function in coupled climate system models. Accounting for photoperiod-induced seasonality in photosynthetic parameters reduces modeled global gross primary production 2.5% (∼4 PgC y(-1)), resulting in a >3% (∼2 PgC y(-1)) decrease of net primary production. Such a correction is also needed in models estimating current carbon uptake based on remotely sensed greenness. Photoperiod-associated declines in photosynthetic capacity could limit autumn carbon gain in forests, even if warming delays leaf senescence.