Simplified Methods for Modelling Dependent Parameters in Health Economic Evaluations: A Tutorial.

BACKGROUND: In health economic evaluations, model parameters are often dependent on other model parameters. Although methods exist to simulate multivariate normal (MVN) distribution data and estimate transition probabilities in Markov models while considering competing risks, they are technically challenging for health economic modellers to implement. This tutorial introduces easily implementable applications for handling dependent parameters in modelling. METHODS: Analytical proofs and proposed simplified methods for handling dependent parameters in typical health economic modelling scenarios are provided, and implementation of these methods are illustrated in seven examples along with the SAS and R code. RESULTS: Methods to quantify the covariance and correlation coefficients of correlated variables based on published summary statistics and generation of MVN distribution data are demonstrated using examples of physician visits data and cost component data. The use of univariate normal distribution data instead of MVN distribution data to capture population heterogeneity is illustrated based on the results from multiple regression models with linear predictors, and two examples are provided (linear fixed-effects model and Cox proportional hazards model). A conditional probability method is introduced to handle two or more state transitions in a single Markov model cycle and applied in examples of one- and two-way state transitions. CONCLUSIONS: This tutorial proposes an extension of routinely used methods along with several examples. These simplified methods may be easily applied by health economic modellers with varied statistical backgrounds.

Understanding bias in probabilistic analysis in model-based health economic evaluation.

Guidelines of economic evaluations suggest that probabilistic analysis (using probability distributions as inputs) provides less biased estimates than deterministic analysis (using point estimates) owing to the non-linear relationship of model inputs and model outputs. However, other factors can also impact the magnitude of bias for model results. We evaluate bias in probabilistic analysis and deterministic analysis through three simulation studies. The simulation studies illustrate that in some cases, compared with deterministic analyses, probabilistic analyses may be associated with greater biases in model inputs (risk ratios and mean cost estimates using the smearing estimator), as well as model outputs (life-years in a Markov model). Point estimates often represent the most likely value of the parameter in the population, given the observed data. When model parameters have wide, asymmetric confidence intervals, model inputs with larger likelihoods (e.g., point estimates) may result in less bias in model outputs (e.g., costs and life-years) than inputs with lower likelihoods (e.g., probability distributions). Further, when the variance of a parameter is large, simulations from probabilistic analyses may yield extreme values that tend to bias the results of some non-linear models. Deterministic analysis can avoid extreme values that probabilistic analysis may encounter. We conclude that there is no definitive answer on which analytical approach (probabilistic or deterministic) is associated with a less-biased estimate in non-linear models. Health economists should consider the bias of probabilistic analysis and select the most suitable approach for their analyses.

Examining the correlation between treatment effects in clinical trials and economic modeling.

INTRODUCTION: Many diseases have a sequential treatment pathway. Compared with patients without previous treatment, patients who fail initial treatment may have lower success rates with a second treatment. This phenomenon can be explained by a correlation between treatment effects. METHODS: We developed a statistical model of covariance for the underlying unobserved correlation between treatments and established a mathematical expression for the magnitude of the latent correlation term. We conducted a simulation study of clinical trials to investigate the correlation between two treatments and explored clinical examples based on published literature to illustrate the identification and evaluation of these correlations. RESULTS: Our simulation study confirmed that a treatment correlation reduces the probability of success for the second treatment, compared with no correlation. We found that treatment correlations may be observable in clinical trials, such as for depression and lung cancer, and the magnitude of correlation may be estimated. We illustrated that treatment correlations can be incorporated into an economic model, with possible impacts on cost-effectiveness results. Additional applications of correlation concepts are also discussed. CONCLUSIONS: We evaluated the correlation between treatment effects and our approach can be applied to clinical trial design and economic modeling of sequential clinical treatment pathways.

Developing Economic Models for Assessing the Cost-Effectiveness of Multiple Diagnostic Tests: Methods and Applications.

BACKGROUND: Clinical pathways with multiple diagnostic tests are complex to model, but problematic and simplistic approaches are often used in economic evaluations. METHODS: We analyzed statistical methods of handling multiple diagnostic tests and provided guidance on applying these methods in economic modeling. We first introduced a statistical model to quantify the correlations between 2 tests and how those correlations can be incorporated within an economic model. We also presented the general form of conditional dependence among multiple tests. We then introduced net reclassification improvement (NRI), a measure that evaluates the added value of a new risk factor (e.g., biomarker) for risk prediction. We further provided 2 examples to illustrate the application of these methods. RESULTS: Our first example illustrated how to model an add-on test to an existing test, in the absence of a perfect reference standard. After accounting for the imperfect nature of both tests and the conditional dependence between tests, the potential health benefits from the additional test were reduced. This led to differential cost-effectiveness results when comparing models using the perfect test and conditional independence assumptions. The second example illustrated how to evaluate the added value of a new risk factor using the NRI measure. Using the new risk classification provides greater precision in risk prediction, and in the example, the strategy using the new risk classification with treatment for selected individuals led to more favorable cost-effectiveness results. CONCLUSIONS: These innovative methods for handling multiple diagnostic tests have improved the methodology within the field and should be adopted to provide more accurate estimates within cost-effectiveness analyses. HIGHLIGHTS: Economic evaluations of multiple diagnostic tests often apply problematic simplistic approaches, such as ignoring conditional dependence between 2 tests or assuming a perfect final test in the diagnostic pathway. We provided guidance on how to apply improved methods for economic modeling.We introduced methods to model conditional dependence between 2 imperfect tests. We used an example to illustrate how assumptions about perfect diagnostic test accuracy and conditional independence between tests affect cost-effectiveness.Compared with the results of the area under the receiver-operating-characteristic curve, net reclassification improvement has distinct advantages in measuring the added value of a new risk factor for model-based economic evaluation.Economic evaluations that appropriately account for the complexities of diagnostic test pathways can help decision makers ensure efficient use of resources.

Review and estimation of disutility for joint health states of severe and nonsevere hypoglycemic events in diabetes.

Aim: Many economic evaluations used linear or log-transformed additive methods to estimate the disutility of hypoglycemic events in diabetes, both nonsevere (NSHEs) and severe (SHEs). Methods: We conducted a literature search for studies of disutility for hypoglycemia. We used additive, minimum and multiplicative methods, and the adjusted decrement estimator to estimate the disutilities of joint health states with both NSHEs and SHEs in six scenarios. Results: Twenty-four studies reported disutilities for hypoglycemia in diabetes. Based on construct validity, the adjusted decrement estimator method likely provides less biased estimates, predicting that when SHEs occur, the additional impact from NSHEs is marginal. Conclusion: Our proposed new method provides a different perspective on the estimation of quality-adjusted life-years in economic evaluations of hypoglycemic treatments.

Noninvasive Prenatal Testing for Trisomies 21, 18, and 13, Sex Chromosome Aneuploidies, and Microdeletions in Average-Risk Pregnancies: A Cost-Effectiveness Analysis.

OBJECTIVE: The cost effectiveness of noninvasive prenatal testing (NIPT) has been established for high-risk pregnancies but remains unclear for pregnancies at other risk levels. The aim was to assess the cost effectiveness of NIPT in average-risk pregnancies from the perspective of a provincial public payer in Canada. METHODS: A model was developed to compare traditional prenatal screening (TPS), NIPT as a second-tier test (performed only after a positive TPS result), and NIPT as a first-tier test (performed instead of TPS) for trisomies 21, 18, and 13; sex chromosome aneuploidies; and microdeletions in a hypothetical annual population cohort of average-risk pregnancies (142 000 to 148,000) in Ontario, Canada. A probabilistic analysis was conducted with 5000 repetitions. RESULTS: Compared with TPS, NIPT as a second-tier test detected more affected fetuses with trisomies 21, 18, and 13 (188 vs. 158), substantially reduced the number of diagnostic tests (i.e., chorionic villus sampling and amniocentesis) performed (660 vs. 3107), and reduced the cost of prenatal screening ($26.7 million vs. $27.6 million) annually. Compared with second-tier NIPT, first-tier NIPT detected an additional 80 cases of trisomies 21, 18, and 13 at an additional cost of $33 million. The incremental cost per additional affected fetus detected was $412 411. Extending first-tier NIPT to include testing for sex chromosome aneuploidies and 22q11.2 deletion would increase the total screening cost. CONCLUSIONS: NIPT as a second-tier test is cost-saving compared with TPS alone. Compared with second-tier NIPT, first-tier NIPT detects more cases of chromosomal anomalies but at a substantially higher cost.

Comparison of the expected rewards between probabilistic and deterministic analyses in a Markov model.

Objectives: In Markov models that evaluate the cost-effectiveness of health-care technologies, it is generally recommended to use probabilistic analysis instead of deterministic analysis. We sought to compare the performance of probabilistic and deterministic analysis in estimating the expected rewards in a Markov model.Methods: We applied Jensen's inequality to compare the expected Markov rewards between probabilistic and deterministic analysis and conducted a simulation study to compare the bias and accuracy between the two approaches.Results: We provided mathematical justification why probabilistic analysis is associated with greater Markov rewards (life-years and quality-adjusted life-years) compared with deterministic analysis. In our simulations, probabilistic analyses tended to generate greater life-years, bias, and mean square error for the estimated rewards compared with deterministic analyses. When the expected values of transition probabilities were the same, weaker evidence derived from smaller sample sizes resulted in larger Markov rewards compared with stronger evidence derived from larger sample sizes. When longer time horizons were applied in cases of weak evidence, there was a substantial increase in bias where the rewards in both probabilistic and deterministic analysis were overestimated.Conclusion: Authors should be aware that probabilistic analysis may lead to increased bias when the evidence is weak.

Some issues for the evaluation of noninferiority trials.

Although published noninferiority trials (NITs) generally conclude that the experimental intervention being studied is noninferior compared with standard therapy or active control, NIT quality is often not satisfactory. We have proposed 14 questions to assist in evaluating the clinical evidence of the experimental versus standard therapy. The aim of these questions is to critically appraise NITs and support proper interpretation of study results. Readers should not only consider whether the confidence interval of the primary effect measure falls within the prespecified noninferiority margin (thus concluding noninferiority), but also assess the similarities between primary and secondary outcomes for the experimental and standard therapy. To conclude noninferiority conceptually is to synthesize evidence from both the current NIT comparing experimental therapy with standard therapy and historical data comparing standard therapy with placebo control. Therefore, readers should use external data sources (e.g., historical data) to validate the study design (e.g., selection of standard therapy, effect measure and the noninferiority margin), and assess the uncertainty of findings due to differences between the observed and expected incidence rates, follow-up time, effects of adjuvant therapy and the secondary outcomes of therapies. Following an explanation of the 14 questions, we then apply the questions to a NIT on intraoperative radiation therapy for early stage breast cancer, as an example.

A small-molecule IRF3 agonist functions as an influenza vaccine adjuvant by modulating the antiviral immune response.

Vaccine adjuvants are essential to drive a protective immune response in cases where vaccine antigens are weakly immunogenic, where vaccine antigen is limited, or where an increase in potency is needed for a specific population, such as the elderly. To discover novel vaccine adjuvants, we used a high-throughput screen (HTS) designed to identify small-molecule agonists of the RIG-I-like receptor (RLR) pathway leading to interferon regulatory factor 3 (IRF3) activation. RLRs are a group of cytosolic pattern-recognition receptors that are essential for the recognition of viral nucleic acids during infection. Upon binding of viral nucleic acid ligands, the RLRs become activated and signal to transcription factors, including IRF3, to initiate an innate immune transcriptional program to control virus infection. Among our HTS hits were a series of benzothiazole compounds from which we designed the lead analog, KIN1148. KIN1148 induced dose-dependent IRF3 nuclear translocation and specific activation of IRF3-responsive promoters. Prime-boost immunization of mice with a suboptimal dose of a monovalent pandemic influenza split virus H1N1 A/California/07/2009 vaccine plus KIN1148 protected against a lethal challenge with mouse-adapted influenza virus (A/California/04/2009) and induced an influenza virus-specific IL-10 and Th2 response by T cells derived from lung and lung-draining lymph nodes. Prime-boost immunization with vaccine plus KIN1148, but not prime immunization alone, induced antibodies capable of inhibiting influenza virus hemagglutinin and neutralizing viral infectivity. Nevertheless, a single immunization with vaccine plus KIN1148 provided increased protection over vaccine alone and reduced viral load in the lungs after challenge. These findings suggest that protection was at least partially mediated by a cellular immune component and that the induction of Th2 and immunoregulatory cytokines by a KIN1148-adjuvanted vaccine may be particularly beneficial for ameliorating the immunopathogenesis that is associated with influenza viruses.

Targeting Innate Immunity for Antiviral Therapy through Small Molecule Agonists of the RLR Pathway.

UNLABELLED: The cellular response to virus infection is initiated when pathogen recognition receptors (PRR) engage viral pathogen-associated molecular patterns (PAMPs). This process results in induction of downstream signaling pathways that activate the transcription factor interferon regulatory factor 3 (IRF3). IRF3 plays a critical role in antiviral immunity to drive the expression of innate immune response genes, including those encoding antiviral factors, type 1 interferon, and immune modulatory cytokines, that act in concert to restrict virus replication. Thus, small molecule agonists that can promote IRF3 activation and induce innate immune gene expression could serve as antivirals to induce tissue-wide innate immunity for effective control of virus infection. We identified small molecule compounds that activate IRF3 to differentially induce discrete subsets of antiviral genes. We tested a lead compound and derivatives for the ability to suppress infections caused by a broad range of RNA viruses. Compound administration significantly decreased the viral RNA load in cultured cells that were infected with viruses of the family Flaviviridae, including West Nile virus, dengue virus, and hepatitis C virus, as well as viruses of the families Filoviridae (Ebola virus), Orthomyxoviridae (influenza A virus), Arenaviridae (Lassa virus), and Paramyxoviridae (respiratory syncytial virus, Nipah virus) to suppress infectious virus production. Knockdown studies mapped this response to the RIG-I-like receptor pathway. This work identifies a novel class of host-directed immune modulatory molecules that activate IRF3 to promote host antiviral responses to broadly suppress infections caused by RNA viruses of distinct genera. IMPORTANCE: Incidences of emerging and reemerging RNA viruses highlight a desperate need for broad-spectrum antiviral agents that can effectively control infections caused by viruses of distinct genera. We identified small molecule compounds that can selectively activate IRF3 for the purpose of identifying drug-like molecules that can be developed for the treatment of viral infections. Here, we report the discovery of a hydroxyquinoline family of small molecules that can activate IRF3 to promote cellular antiviral responses. These molecules can prophylactically or therapeutically control infection in cell culture by pathogenic RNA viruses, including West Nile virus, dengue virus, hepatitis C virus, influenza A virus, respiratory syncytial virus, Nipah virus, Lassa virus, and Ebola virus. Our study thus identifies a class of small molecules with a novel mechanism to enhance host immune responses for antiviral activity against a variety of RNA viruses that pose a significant health care burden and/or that are known to cause infections with high case fatality rates.

A validation study of administrative data algorithms to identify patients with Parkinsonism with prevalence and incidence trends.

BACKGROUND: Epidemiological studies for identifying patients with Parkinson's disease (PD) or Parkinsonism (PKM) have been limited by their nonrandom sampling techniques and mainly veteran populations. This reduces their use for health services planning. The purpose of this study was to validate algorithms for the case ascertainment of PKM from administrative databases using primary care patients as the reference standard. METHODS: We conducted a retrospective chart abstraction using a random sample of 73,003 adults aged ≥ 20 years from a primary care Electronic Medical Record Administrative data Linked Database (EMRALD) in Ontario, Canada. Physician diagnosis in the EMR was used as the reference standard and population-based administrative databases were used to identify patients with PKM from the derivation of algorithms. We calculated algorithm performance using sensitivity, specificity, and predictive values and then determined the population-level prevalence and incidence trends with the most accurate algorithms. RESULTS: We selected, '2 physician billing codes in 1 year' as the optimal administrative data algorithm in adults and seniors (≥ 65 years) due to its sensitivity (70.6-72.3%), specificity (99.9-99.8%), positive predictive value (79.5-82.8%), negative predictive value (99.9-99.7%), and prevalence (0.28-1.20%), respectively. CONCLUSIONS: Algorithms using administrative databases can reliably identify patients with PKM with a high degree of accuracy.

Assessing the validity of using administrative data to identify patients with epilepsy.

OBJECTIVE: Previous validation studies assessing the use of administrative data to identify patients with epilepsy have used targeted sampling or have used a reference standard of patients in the neurologist, hospital, or emergency room setting. Therefore, the validity of using administrative data to identify patients with epilepsy in the general population has not been previously assessed. The purpose of this study was to determine the validity of using administrative data to identify patients with epilepsy in the general population. METHODS: A retrospective chart abstraction study was performed using primary care physician records from 83 physicians distributed throughout Ontario and contributing data to the Electronic Medical Record Administrative data Linked Database (EMRALD) A random sample of 7,500 adult patients, from a possible 73,014 eligible, was manually chart abstracted to identify patients who had ever had epilepsy. These patients were used as a reference standard to test a variety of administrative data algorithms. RESULTS: An algorithm of three physician billing codes (separated by at least 30 days) in 2 years or one hospitalization had a sensitivity of 73.7% (95% confidence interval [CI] 64.8-82.5%), specificity of 99.8% (95% CI 99.6-99.9%), positive predictive value (PPV) of 79.5% (95% CI 71.1-88.0%), and negative predictive value (NPV) of 99.7% (95% CI 99.5-99.8%) for identifying patients who had ever had epilepsy. SIGNIFICANCE: The results of our study showed that administrative data can reasonably accurately identify patients who have ever had epilepsy, allowing for a "lifetime" population prevalence determination of epilepsy in Ontario and the rest of Canada with similar administrative databases. This will facilitate future studies on population level patterns and outcomes of care for patients living with epilepsy.

Validity of administrative data for identifying patients who have had a stroke or transient ischemic attack using EMRALD as a reference standard.

BACKGROUND: Surveillance for stroke/transient ischemic attack (TIA) using administrative data has traditionally been limited to reporting patients who had an acute event and were hospitalized. This underestimates the true prevalence because many events do not result in hospitalization. We examined whether the accuracy of administrative data for identifying prevalent stroke/TIA could be improved by using data from both inpatient and outpatient visits. METHODS: An administrative data validation reference standard was developed through chart abstraction of 5000 adult patients randomly sampled from 73,014 patients of 83 family physicians who participate in the Electronic Medical Record Administrative Data Linked Database (EMRALD), in Ontario, Canada. RESULTS: The prevalence of stroke/TIA in our adult population was 3.0%. An algorithm of 1 hospital record had a sensitivity of 35.3% (27.7%-43.0%) and specificity of 99.8% (99.7%-99.9%), whereas an algorithm of 2 physician billings within 1 year or 1 hospitalization had a sensitivity of 68.0% (95% confidence interval [CI], 60.5%-75.5%) and specificity of 98.9% (95% CI, 98.6%-99.2%) for the identification of patients who had ever had a stroke/TIA. We found that hospitalization data underestimated the prevalence of stroke by > 50% and TIA by > 66% compared with using both hospitalization and physician claims data. CONCLUSIONS: The use of outpatient physician claims data in addition to hospitalization data improves the sensitivity of administrative data for the identification of prevalent stroke/TIA and may be used to estimate the prevalence of cerebrovascular events in large populations and over time.

Isoflavone agonists of IRF-3 dependent signaling have antiviral activity against RNA viruses.

There is a growing need for novel antiviral therapies that are broad spectrum, effective, and not subject to resistance due to viral mutations. Using high-throughput screening methods, including computational docking studies and an interferon-stimulated gene 54 (ISG54)-luciferase reporter assay, we identified a class of isoflavone compounds that act as specific agonists of innate immune signaling pathways and cause activation of the interferon regulatory factor (IRF-3) transcription factor. The isoflavone compounds activated the ISG54 promoter, mediated nuclear translocation of IRF-3, and displayed highly potent activity against hepatitis C virus (HCV) and influenza virus. Additionally, these agonists efficiently activated IRF-3 in the presence of the HCV protease NS3-4A, which is known to blunt the host immune response. Furthermore, genomic studies showed that discrete innate immune pathways centered on IRF signaling were regulated following agonist treatment without causing global changes in host gene expression. Following treatment, the expression of only 64 cellular genes was significantly induced. This report provides the first evidence that innate immune pathways dependent on IRF-3 can be successfully targeted by small-molecule drugs for the development of novel broad-spectrum antiviral compounds.

Gene expression signature of c-MYC-immortalized human fibroblasts reveals loss of growth inhibitory response to TGFß.

Cancer cells exhibit the ability to proliferate indefinitely, but paradoxically, overexpression of cellular oncogenes in primary cells can result in a rapid and irreversible cell cycle arrest known as oncogene-induced senescence (OIS). However, we have shown that constitutive overexpression of the oncogene c-MYC in primary human foreskin fibroblasts results in a population of cells with unlimited lifespan; these immortalized cells are henceforth referred to as iMYC. Here, in order to further elucidate the mechanisms underlying the immortalization process, a gene expression signature of three independently established iMYC cell lines compared to matched early passage c-MYC overexpressing cells was derived. Network analysis of this "iMYC signature" indicated that a large fraction of the down-regulated genes were functionally connected and major nodes centered around the TGFß, IL-6 and IGF-1 signaling pathways. Here, we focused on the functional validation of the alteration of TGFß response during c-MYC-mediated immortalization. The results demonstrate loss of sensitivity of iMYC cells to activation of TGFß signaling upon ligand addition. Furthermore, we show that aberrant regulation of the p27 tumor suppressor protein in iMYC cells is a key event that contributes to loss of response to TGFß. These findings highlight the potential to reveal key pathways contributing to the self-renewal of cancer cells through functional mining of the unique gene expression signature of cells immortalized by c-MYC.

Epigenetic down-regulation of ARF expression is a selection step in immortalization of human fibroblasts by c-Myc.

The transcription factor c-Myc is implicated in the pathogenesis of many cancers. Among the multiple functions of c-Myc, activation of hTert and other genes involved in cellular life span contributes to its role as an oncogene. However, the ability of c-Myc to directly immortalize human cells remains controversial. We show here that overexpression of c-Myc reproducibly immortalizes freshly isolated human foreskin fibroblasts. c-Myc-immortalized cells displayed no gross karyotypic abnormalities but consisted of an oligoclonal population, suggesting that additional events cooperated to achieve immortalization. Levels of p53 and p16 were increased, but both p53-dependent DNA damage response and growth arrest in response to p16 overexpression remained intact. A marked decrease in expression of the tumor suppressor ARF occurred in several independently established c-Myc-immortalized cell lines. Methylation-specific PCR showed that the ARF gene was methylated in immortalized but not early-passage c-Myc cells, whereas p16 was unmethylated in both cell populations. Restoration of ARF expression by treatment with a demethylating agent or overexpression by a retroviral vector coincided with inhibition of proliferation and senescence of c-Myc-immortalized cells. Our findings predict that epigenetic events play a significant role in human tumors that express high levels of c-Myc.