The development and evaluation of triage algorithms for early discovery of adverse drug interactions.

BACKGROUND: Around 20 % of all adverse drug reactions (ADRs) are due to drug interactions. Some of these will only be detected in the postmarketing setting. Effective screening in large collections of individual case safety reports (ICSRs) requires automated triages to identify signals of adverse drug interactions. Research so far has focused on statistical measures, but clinical information and pharmacological characteristics are essential in the clinical assessment and may be of great value in first-pass filtering of potential adverse drug interaction signals. OBJECTIVE: The aim of this study was to develop triages for adverse drug interaction surveillance, and to evaluate these prospectively relative to clinical assessment. METHODS: A broad set of variables were considered for inclusion in the triages, including cytochrome P450 (CYP) activity, explicit suspicions of drug interactions as noted by the reporter, dose and treatment overlap, and a measure of interaction disproportionality. Their unique contributions in predicting signals of adverse drug interactions were determined through logistic regression. This was based on the reporting in the WHO global ICSR database, VigiBase™, for a set of known adverse drug interactions and corresponding negative controls. Three triages were developed, each producing an estimated probability that a given drug-drug-ADR triplet constitutes an adverse drug interaction signal. The triages were evaluated against two separate benchmarks derived from expert clinical assessment: adverse drug interactions known in the literature and prospective adverse drug interaction signals. For reference, the triages were compared with disproportionality analysis alone using the same benchmarks. RESULTS: The following were identified as valuable predictors of adverse drug interaction signals: plausible CYP metabolism; notes of suspected interaction by the reporter; and reports of unexpected therapeutic response, altered therapeutic effect with dose information and altered therapeutic effect when only two drugs had been used. The new triages identified reporting patterns corresponding to both prospective signals of adverse drug interactions and already established ones. They perform better than disproportionality analysis alone relative to both benchmarks. CONCLUSIONS: A range of predictors for adverse drug interaction signals have been identified. They substantially improve signal detection capacity compared with disproportionality analysis alone. The value of incorporating clinical and pharmacological information in first-pass screening is clear.

Shrinkage observed-to-expected ratios for robust and transparent large-scale pattern discovery.

Large observational data sets are a great asset to better understand the effects of medicines in clinical practice and, ultimately, improve patient care. For an empirical pattern in observational data to be of practical relevance, it should represent a substantial deviation from the null model. For the purpose of identifying such deviations, statistical significance tests are inadequate, as they do not on their own distinguish the magnitude of an effect from its data support. The observed-to-expected (OE) ratio on the other hand directly measures strength of association and is an intuitive basis to identify a range of patterns related to event rates, including pairwise associations, higher order interactions and temporal associations between events over time. It is sensitive to random fluctuations for rare events with low expected counts but statistical shrinkage can protect against spurious associations. Shrinkage OE ratios provide a simple but powerful framework for large-scale pattern discovery. In this article, we outline a range of patterns that are naturally viewed in terms of OE ratios and propose a straightforward and effective statistical shrinkage transformation that can be applied to any such ratio. The proposed approach retains emphasis on the practical relevance and transparency of highlighted patterns, while protecting against spurious associations.

Impact of stratification on adverse drug reaction surveillance.

BACKGROUND AND OBJECTIVES: Automated screening for excessive adverse drug reaction (ADR) reporting rates has proven useful as a tool to direct clinical review in large-scale drug safety signal detection. Some measures of disproportionality can be adjusted to eliminate any undue influence on the ADR reporting rate of covariates, such as patient age or country of origin, by using a weighted average of stratum-specific measures of disproportionality. Arguments have been made in favour of routine adjustment for a set of common potential confounders using stratification. The aim of this paper is to investigate the impact of using adjusted observed-to-expected ratios, as implemented for the Empirical Bayes Geometric Mean (EBGM) and the information component (IC) measures of disproportionality, for first-pass analysis of the WHO database. METHODS: A simulation study was carried out to investigate the impact of simultaneous adjustment for several potential confounders based on stratification. Comparison between crude and adjusted observed-to-expected ratios were made based on random allocation of reports to a set of strata with a realistic distribution of stratum sizes. In a separate study, differences between the crude IC value and IC values adjusted for (combinations of) patient sex, age group, reporting quarter and country of origin, with respect to their concordance with a literature comparison were analysed. Comparison was made to the impact on signal detection performance of a triage criterion requiring reports from at least two countries before a drug-ADR pair was highlighted for clinical review. RESULTS: The simulation study demonstrated a clear tendency of the adjusted observed-to-expected ratio to spurious (and considerable) underestimation relative to the crude one, in the presence of any very small strata in a stratified database. With carefully implemented stratification that did not yield any very small strata, this tendency could be avoided. Routine adjustment for potential confounders improved signal detection performance relative to the literature comparison, but the magnitude of the improvement was modest. The improvement from the triage criterion was more considerable. DISCUSSION AND CONCLUSIONS: Our results indicate that first-pass screening based on observed-to-expected ratios adjusted with stratification may lead to missed signals in ADR surveillance, unless very small strata are avoided. In addition, the improvement in signal detection performance due to routine adjustment for a set of common confounders appears to be smaller than previously assumed. Other approaches to improving signal detection performance such as the development of refined triage criteria may be more promising areas for future research.