Disease misclassification in electronic healthcare database studies: Deriving validity indices-A contribution from the ADVANCE project.

There is a strong and continuously growing interest in using large electronic healthcare databases to study health outcomes and the effects of pharmaceutical products. However, concerns regarding disease misclassification (i.e. classification errors of the disease status) and its impact on the study results are legitimate. Validation is therefore increasingly recognized as an essential component of database research. In this work, we elucidate the interrelations between the true prevalence of a disease in a database population (i.e. prevalence assuming no disease misclassification), the observed prevalence subject to disease misclassification, and the most common validity indices: sensitivity, specificity, positive and negative predictive value. Based on this, we obtained analytical expressions to derive all the validity indices and true prevalence from the observed prevalence and any combination of two other parameters. The analytical expressions can be used for various purposes. Most notably, they can be used to obtain an estimate of the observed prevalence adjusted for outcome misclassification from any combination of two validity indices and to derive validity indices from each other which would otherwise be difficult to obtain. To allow researchers to easily use the analytical expressions, we additionally developed a user-friendly and freely available web-application.

ADVANCE system testing: Estimating the incidence of adverse events following pertussis vaccination in healthcare databases with incomplete exposure data.

The Accelerated Development of VAccine beNefit-risk Collaboration in Europe (ADVANCE) is a public-private collaboration aiming to develop and test a system for rapid vaccine benefit-risk monitoring using existing European healthcare databases. Incidence rate (IR) estimates of vaccination-associated adverse events that are needed to model vaccination risks can be calculated from existing healthcare databases when vaccination (exposure) data are available. We assessed different methods to derive IRs in risk periods following vaccination when exposure data are missing in one database, using estimated IRs and IRRs from other databases for febrile seizures, fever and persistent crying. IRs were estimated for children aged 0-5 years in outcome-specific risk and non-risk periods following the first dose of acellular pertussis (aP) vaccination in four primary care databases and one hospital database. We compared derived and observed IRs in each database using three methods: 1) multiplication of non-risk period IR for database i by IR ratio (IRR) obtained from meta-analysis of IRRs estimated using the self-controlled case-series method, from databases other than i; 2) same method as 1, but multiplying with background IR; and 3) meta-analyses of observed IRs from databases other than i. IRs for febrile seizures were lower in primary care databases than the hospital database. The derived IR for febrile seizures using data from primary care databases was lower than that observed in the hospital database, and using data from the hospital database gave a higher derived IR than that observed in the primary care database. For fever and persistent crying the opposite was observed. We demonstrated that missing IRs for a post-vaccination period can be derived but that the type of database and the method of event data capture can have an impact on potential bias. We recommend IRs are derived using data from similar database types (hospital or primary care) with caution as even this can give heterogeneous results.

ADVANCE system testing: Can safety studies be conducted using electronic healthcare data? An example using pertussis vaccination.

INTRODUCTION: The Accelerated Development of Vaccine benefit-risk Collaboration in Europe (ADVANCE) public-private collaboration, aimed to develop and test a system for rapid benefit-risk monitoring of vaccines using healthcare databases in Europe. The objective of this proof-of-concept (POC) study was to test the feasibility of the ADVANCE system to generate incidence rates (IRs) per 1000 person-years and incidence rate ratios (IRRs) for risks associated with whole cell- (wP) and acellular- (aP) pertussis vaccines, occurring in event-specific risk windows in children prior to their pre-school-entry booster. METHODS: The study population comprised almost 5.1 million children aged 1 month to <6 years vaccinated with wP or aP vaccines during the study period from 1 January 1990 to 31 December 2015. Data from two Danish hospital (H) databases (AUH and SSI) and five primary care (PC) databases from, UK (THIN and RCGP RSC), Spain (SIDIAP and BIFAP) and Italy (Pedianet) were analysed. Database-specific IRRs between risk vs. non-risk periods were estimated in a self-controlled case series study and pooled using random-effects meta-analyses. RESULTS: The overall IRs were: fever, 58.2 (95% CI: 58.1; 58.3), 96.9 (96.7; 97.1) for PC DBs and 8.56 (8.5; 8.6) for H DBs; convulsions, 7.6 (95% CI: 7.6; 7.7), 3.55 (3.5; 3.6) for PC and 12.87 (12.8; 13) for H; persistent crying, 3.9 (95% CI: 3.8; 3.9) for PC, injection-site reactions, 2.2 (95% CI 2.1; 2.2) for PC, hypotonic hypo-responsive episode (HHE), 0.4 (95% CI: 0.4; 0.4), 0.6 (0.6; 0.6) for PC and 0.2 (0.2; 0.3) for H; and somnolence: 0.3 (95% CI: 0.3; 0.3) for PC. The pooled IRRs for persistent crying, fever, and ISR, adjusted for age and healthy vaccinee period were higher after wP vs. aP vaccination, and lower for convulsions, for all doses. The IRR for HHE was slightly lower for wP than aP, while wP was associated with somnolence only for dose 1 and dose 3 compared with aP. CONCLUSIONS: The estimated IRs and IRRs were comparable with published data, therefore demonstrating that the ADVANCE system was able to combine several European healthcare databases to assess vaccine safety data for wP and aP vaccination.

ADVANCE system testing: Benefit-risk analysis of a marketed vaccine using multi-criteria decision analysis and individual-level state transition modelling.

BACKGROUND: The Accelerated Development of VAccine beNefit-risk Collaboration in Europe (ADVANCE) is a public-private collaboration aiming to develop and test a system for rapid benefit-risk (B/R) monitoring of vaccines using electronic health record (eHR) databases in Europe. Proof-of-concept studies were designed to assess the proposed processes and system for generating the required evidence to perform B/R assessment and near-real time monitoring of vaccines. We aimed to test B/R methodologies for vaccines, using the comparison of the B/R profiles of whole-cell (wP) and acellular pertussis (aP) vaccine formulations in children as an example. METHODS: We used multi-criteria decision analysis (MCDA) to structure the B/R assessment combined with individual-level state transition modelling to build the B/R effects table. In the state transition model, we simulated the number of events in two hypothetical cohorts of 1 million children followed from first pertussis dose till pre-school-entry booster (or six years of age, whichever occurred first), with one cohort receiving wP, and the other aP. The benefits were reductions in pertussis incidence and complications. The risks were increased incidences of febrile convulsions, fever, hypotonic-hyporesponsive episodes, injection-site reactions and persistent crying. Most model parameters were informed by estimates (coverage, background incidences, relative risks) from eHR databases from Denmark (SSI), Spain (BIFAP and SIDIAP), Italy (Pedianet) and the UK (RCGP-RSC and THIN). Preferences were elicited from clinical and epidemiological experts. RESULTS: Using state transition modelling to build the B/R effects table facilitated the comparison of different vaccine effects (e.g. immediate vaccine risks vs long-term vaccine benefits). Estimates from eHR databases could be used to inform the simulation model. The model results could be easily combined with preference weights to obtain B/R scores. CONCLUSION: Existing B/R methodology, modelling and estimates from eHR databases can be successfully used for B/R assessment of vaccines.

ADVANCE: Towards near real-time monitoring of vaccination coverage, benefits and risks using European electronic health record databases.

BACKGROUND: The Accelerated Development of VAccine beNefit-risk Collaboration in Europe (ADVANCE) is a public-private partnership aiming to develop and test a system for rapid benefit-risk (B/R) monitoring of vaccines using European electronic health record (eHR) databases. This proof-of-concept study aimed to test the feasibility of near real-time (NRT) monitoring of vaccination coverage, benefits and risks based on multiple European eHR databases, using acellular pertussis vaccination in children aged <6 years as test case. METHODS: A qualitative feasibility assessment on NRT monitoring was carried out using a survey and face-to-face discussion with ADVANCE data partners. Subsequently, a dynamic cohort study was conducted containing two distinct observation periods: a first period to establish a baseline (Jan 2014 to Mar 2018) and a subsequent 3-month period to test the actual feasibility of weekly NRT monitoring, based on which data latencies were calculated. An interactive web-application was additionally developed to facilitate the visual monitoring of vaccination coverage, the vaccine preventable disease incidence rates (benefits) and the incidence rates of adverse events (risks). RESULTS: Nine databases from four countries (Denmark, Italy, Spain and UK) participated in the qualitative feasibility assessment. Of them, five databases took part in the dynamic cohort study, with 5 databases providing baseline data and 3 databases participating to the NRT monitoring, providing data extractions on an almost weekly basis. The median data latency (time between event date and data release date) was between 1 and 2 weeks except for the benefit and risk events in one of the databases (latency 16 weeks). CONCLUSION: Three European eHR databases successfully demonstrated the feasibility of providing data for weekly NRT monitoring, with short data latencies of 1-2 weeks for most events.

Bias due to differential and non-differential disease- and exposure misclassification in studies of vaccine effectiveness.

BACKGROUND: Studies of vaccine effectiveness (VE) rely on accurate identification of vaccination and cases of vaccine-preventable disease. In practice, diagnostic tests, clinical case definitions and vaccination records often present inaccuracies, leading to biased VE estimates. Previous studies investigated the impact of non-differential disease misclassification on VE estimation. METHODS: We explored, through simulation, the impact of non-differential and differential disease- and exposure misclassification when estimating VE using cohort, case-control, test-negative case-control and case-cohort designs. The impact of misclassification on the estimated VE is demonstrated for VE studies on childhood seasonal influenza and pertussis vaccination. We additionally developed a web-application graphically presenting bias for user-selected parameters. RESULTS: Depending on the scenario, the misclassification parameters had differing impacts. Decreased exposure specificity had greatest impact for influenza VE estimation when vaccination coverage was low. Decreased exposure sensitivity had greatest impact for pertussis VE estimation for which high vaccination coverage is typically achieved. The impact of the exposure misclassification parameters was found to be more noticeable than that of the disease misclassification parameters. When misclassification is limited, all study designs perform equally. In case of substantial (differential) disease misclassification, the test-negative design performs worse. CONCLUSIONS: Misclassification can lead to significant bias in VE estimates and its impact strongly depends on the scenario. We developed a web-application for assessing the potential (joint) impact of possibly differential disease- and exposure misclassification that can be modified by users to their own study scenario. Our results and the simulation tool may be used to guide better design, conduct and interpretation of future VE studies.

Methodology for computing the burden of disease of adverse events following immunization.

PURPOSE: Composite disease burden measures such as disability-adjusted life-years (DALY) have been widely used to quantify the population-level health impact of disease or injury, but application has been limited for the estimation of the burden of adverse events following immunization. Our objective was to assess the feasibility of adapting the DALY approach for estimating adverse event burden. METHODS: We developed a practical methodological framework, explicitly describing all steps involved: acquisition of relative or absolute risks and background event incidence rates, selection of disability weights and durations, and computation of the years lived with disability (YLD) measure, with appropriate estimation of uncertainty. We present a worked example, in which YLD is computed for 3 recognized adverse reactions following 3 childhood vaccination types, based on background incidence rates and relative/absolute risks retrieved from the literature. RESULTS: YLD provided extra insight into the health impact of an adverse event over presentation of incidence rates only, as severity and duration are additionally incorporated. As well as providing guidance for the deployment of DALY methodology in the context of adverse events associated with vaccination, we also identified where data limitations potentially occur. CONCLUSIONS: Burden of disease methodology can be applied to estimate the health burden of adverse events following vaccination in a systematic way. As with all burden of disease studies, interpretation of the estimates must consider the quality and accuracy of the data sources contributing to the DALY computation.

Benefit-Risk Monitoring of Vaccines Using an Interactive Dashboard: A Methodological Proposal from the ADVANCE Project.

INTRODUCTION: New vaccines are launched based on their benefit-risk (B/R) profile anticipated from clinical development. Proactive post-marketing surveillance is necessary to assess whether the vaccination uptake and the B/R profile are as expected and, ultimately, whether further public health or regulatory actions are needed. There are several, typically not integrated, facets of post-marketing vaccine surveillance: the surveillance of vaccination coverage, vaccine safety, effectiveness and impact. OBJECTIVE: With this work, we aim to assess the feasibility and added value of using an interactive dashboard as a potential methodology for near real-time monitoring of vaccine coverage and pre-specified health benefits and risks of vaccines. METHODS: We developed a web application with an interactive dashboard for B/R monitoring. The dashboard is demonstrated using simulated electronic healthcare record data mimicking the introduction of rotavirus vaccination in the UK. The interactive dashboard allows end users to select certain parameters, including expected vaccine effectiveness, age groups, and time periods and allows calculation of the incremental net health benefit (INHB) as well as the incremental benefit-risk ratio (IBRR) for different sets of preference weights. We assessed the potential added value of the dashboard by user testing amongst a range of stakeholders experienced in the post-marketing monitoring of vaccines. RESULTS: The dashboard was successfully implemented and demonstrated. The feedback from the potential end users was generally positive, although reluctance to using composite B/R measures was expressed. CONCLUSION: The use of interactive dashboards for B/R monitoring is promising and received support from various stakeholders. In future research, the use of such an interactive dashboard will be further tested with real-life data as opposed to simulated data.

Childhood leukaemia near nuclear sites in Belgium, 2002-2008.

This paper describes an ecological study investigating whether there is an excess incidence of acute leukaemia among children aged 0-14 years living in the vicinity of the nuclear sites in Belgium. Poisson regression modelling was carried out for proximity areas of varying sizes. In addition, the hypothesis of a gradient in leukaemia incidence with increasing levels of surrogate exposures was explored by means of focused hypothesis tests and generalized additive models. For the surrogate exposures, three proxies were used, that is, residential proximity to the nuclear site, prevailing winds and simulated radioactive discharges, on the basis of mathematical dispersion modelling. No excess incidence of acute leukaemia was observed around the nuclear power plants of Doel or Tihange nor around the nuclear site of Fleurus, which is a major manufacturer of radioactive isotopes in Europe. Around the site of Mol-Dessel, however, two- to three-fold increased leukaemia incidence rates were found in children aged 0-14 years living in the 0-5, 0-10 and the 0-15 km proximity areas. For this site, there was evidence for a gradient in leukaemia incidence with increased proximity, prevailing winds and simulated radioactive discharges, suggesting a potential link with the site that needs further investigation. An increased incidence of acute leukaemia in children aged 0-14 years was observed around one nuclear site that hosted reprocessing activities in the past and where nuclear research activities and radioactive waste treatment are ongoing.

Comparative long-term trend analysis of daily weather conditions with daily pollen concentrations in Brussels, Belgium.

A clear rise in seasonal and annual temperatures, a gradual increase of total radiation, and a relative trend of change in seasonal precipitation have been observed for the last four decades in Brussels (Belgium). These local modifications may have a direct and indirect public health impact by altering the timing and intensity of allergenic pollen seasons. In this study, we assessed the statistical correlations (Spearman's test) between pollen concentration and meteorological conditions by using long-term daily datasets of 11 pollen types (8 trees and 3 herbaceous plants) and 10 meteorological parameters observed in Brussels between 1982 and 2015. Furthermore, we analyzed the rate of change in the annual cycle of the same selected pollen types by the Mann-Kendall test. We revealed an overall trend of increase in daily airborne tree pollen (except for the European beech tree) and an overall trend of decrease in daily airborne pollen from herbaceous plants (except for Urticaceae). These results revealed an earlier onset of the flowering period for birch, oak, ash, plane, grasses, and Urticaceae. Finally, the rates of change in pollen annual cycles were shown to be associated with the rates of change in the annual cycles of several meteorological parameters such as temperature, radiation, humidity, and rainfall.

Thyroid Cancer Incidence around the Belgian Nuclear Sites, 2000-2014.

The present study investigates whether there is an excess incidence of thyroid cancer among people living in the vicinity of the nuclear sites in Belgium. Adjusted Rate Ratios were obtained from Poisson regressions for proximity areas of varying sizes. In addition, focused hypothesis tests and generalized additive models were performed to test the hypothesis of a gradient in thyroid cancer incidence with increasing levels of surrogate exposures. Residential proximity to the nuclear site, prevailing dominant winds frequency from the site, and simulated radioactive discharges were used as surrogate exposures. No excess incidence of thyroid cancer was observed around the nuclear power plants of Doel or Tihange. In contrast, increases in thyroid cancer incidence were found around the nuclear sites of Mol-Dessel and Fleurus; risk ratios were borderline not significant. For Mol-Dessel, there was evidence for a gradient in thyroid cancer incidence with increased proximity, prevailing winds, and simulated radioactive discharges. For Fleurus, a gradient was observed with increasing prevailing winds and, to a lesser extent, with increasing simulated radioactive discharges. This study strengthens earlier findings and suggests increased incidences in thyroid cancer around two of the four Belgian nuclear sites. Further analyses will be performed at a more detailed geographical level.

Biomarkers in patients admitted to the emergency department after exposure to acrylonitrile in a major railway incident involving bulk chemical material.

BACKGROUND: A railway incident with victims of exposure to the cyanogenic substance acrylonitrile (ACN). AIMS: We retrospectively (i)built an inventory of the clinical characteristics of individuals admitted to surrounding emergency departments (ED's) and (ii)studied the correlation between N-2-cyanoethylvaline (CEV), a biomarker used in a population study for evaluating exposure to ACN, with lactate and thiocyanate (SCN), biomarkers determined during emergency care. RESULTS: 438 patients from 11 ED's were included and presented with known symptoms of ACN poisoning but also with concern about the risks. A comparison of CEV with lactate or SCN was possible in 108 and 73 patients respectively. CEV was very high in a critically ill patient with a high lactate. There was no correlation with CEV in the patients with normal or slightly elevated lactate concentrations. A correlation of CEV with SCN was only observed in smokers. LIMITATIONS: First there is a lack of data in some clinical files concerning the time and duration of exposure and the smoking-status. A second limitation is that blood samples for biomarkers were not taken systematically in all patients, which may have induced bias. A third limitation is that blood sampling was possibly done outside the correct time window related to the delayed toxicity of ACN. Finally the number of severely-intoxicated patients was low and ACN exposure may not have taken place e.g. in individuals consulting with psychological symptoms. These aspects may have contributed to the below detection limits' analyses of biomarkers. CONCLUSIONS: CEV was markedly elevated in a severely-intoxicated patient with high lactate, a sensitive marker for CN intoxication. We found no correlation of CEV with normal or slightly elevated lactate concentrations but clinicians should consider the possibility of subsequent rises due to the delay in ACN toxicity. CEV correlated with SCN in smokers, which may be explained by ACN in tobacco smoke and deserves further exploration. Further studies are necessary to evaluate the correlation between biomarkers in acute chemical exposures to ACN and these should be carried out prospectively using a preplanned template.

Nicotine Dependence and Urinary Nicotine, Cotinine and Hydroxycotinine Levels in Daily Smokers.

INTRODUCTION: Nicotine dependence and smoking frequency are critical factors for smoking cessation. The aims of this study are (1) to determine if nicotine dependence Fagerström Test for Nicotine Dependence (FTND) scores are associated with urinary levels of nicotine metabolites, (2) to assess the relationship of hydroxycotinine/cotinine ratio with FTND score and cigarettes smoked per day (CPD), and (3) to identify significant predictors of cigarettes per day among biomarker concentrations and individual FTND items. METHODS: Urine samples and questionnaire data of 239 daily smokers were obtained. Nicotine, cotinine and hydroxycotinine urinary levels were determined by UPLC MS/MS.Multiple linear regression models were developed to explore the relationship between nicotine, cotinine, hydroxycotinine levels and separate FTND scores (for all six items). RESULTS: We found significant correlations between the different urinary biomarker concentrations, and the FTND score. The time before the first cigarette after waking (TTFC) was significantly associated with the nicotine, cotinine and hydroxycotinine concentrations. No association was found between the ratio of hydroxycotinine to cotinine and either the FTND or the CPD. A model including four FTND questions, sex, age, and the cotinine concentration, accounted for 45% of the variance of CPD. CONCLUSIONS: There are significant relationships between urinary levels of nicotine, cotinine, and hydroxycotinine and the FTND score. Especially the FTND question about TTFC is relevant for explaining the biomarker concentrations. CPD (below 15) was significantly explained by four FTND dependence items and urinary cotinine levels in a regression model. IMPLICATIONS: We investigated associations between urinary levels of nicotine, cotinine, and hydroxycotinine in daily smokers and the FTND scores for nicotine dependence. We did not find association between the hydroxycotinine/cotinine ratio and CPD. We developed a model that explains the cigarettes smoked daily (CPD) in a group of light smokers by combining FTND items, urinary cotinine levels, sex, and age. Our results might be of importance for clinical use or future studies on larger smoking populations.