Development of a novel regulatory pharmacovigilance prioritisation system: an evaluation of its performance at the UK Medicines and Healthcare products Regulatory Agency.

BACKGROUND: The prioritisation of drug safety issues for further evaluation or regulatory action is critical to ensure that acceptable timelines and appropriate resource allocation are defined to meet public health and regulatory obligations. OBJECTIVE: Our objective was to develop, pilot and implement a novel tool for prioritising pharmacovigilance issues within the Medicines and Healthcare products Regulatory Agency (MHRA). METHODS: An initial system was developed empirically and then piloted over a 10-month period in the pharmacovigilance signal management meeting at the MHRA that discusses potential pharmacovigilance issues, and determines, through consensus, their priority and a timescale for action. The priority assigned by the tool was compared with the priority decided by collective judgement at the meeting. Once an acceptable level of concordance between the tool and the meeting had been achieved, the finalised tool was implemented into routine use at the MHRA, with an evaluation of its performance conducted after the first year. RESULTS: The Regulatory Pharmacovigilance Prioritisation System (RPPS) tool prioritises pharmacovigilance issues according to the following four broad categories, each with four inputs: strength of evidence, public health implications, agency regulatory obligations and public perceptions. A weighted scoring system links the inputs to a pre-defined number of points where if a threshold is reached then the points are awarded. The overall priority is determined by the sum of all points obtained from each of the inputs. The pilot study included a total of 73 pharmacovigilance issues during the 10-month study period, with an overall exact agreement between the RPPS priority and the collective judgement of the meeting of 60.3 %. Where exact agreement was not obtained, the RPPS generally prioritised the issues slightly higher than the meeting. Over the first year following implementation, the RPPS achieved an overall exact agreement of 82.2 %. CONCLUSION: Following the pilot study and implementation at the UK MHRA, the RPPS has provided a systematic approach to drug safety issue prioritisation that should help to reduce the subjectivity of reliance on individual judgement.

Developing tools for the safety specification in risk management plans: lessons learned from a pilot project.

PURPOSE: Following the adoption of the ICH E2E guideline, risk management plans (RMP) defining the cumulative safety experience and identifying limitations in safety information are now required for marketing authorisation applications (MAA). A collaborative research project was conducted to gain experience with tools for presenting and evaluating data in the safety specification. This paper presents those tools found to be useful and the lessons learned from their use. METHODS: Archive data from a successful MAA were utilised. Methods were assessed for demonstrating the extent of clinical safety experience, evaluating the sensitivity of the clinical trial data to detect treatment differences and identifying safety signals from adverse event and laboratory data to define the extent of safety knowledge with the drug. RESULTS: The extent of clinical safety experience was demonstrated by plots of patient exposure over time. Adverse event data were presented using dot plots, which display the percentages of patients with the events of interest, the odds ratio, and 95% confidence interval. Power and confidence interval plots were utilised for evaluating the sensitivity of the clinical database to detect treatment differences. Box and whisker plots were used to display laboratory data. CONCLUSIONS: This project enabled us to identify new evidence-based methods for presenting and evaluating clinical safety data. These methods represent an advance in the way safety data from clinical trials can be analysed and presented. This project emphasises the importance of early and comprehensive planning of the safety package, including evaluation of the use of epidemiology data.

An analysis of the exclusion criteria used in observational pharmacoepidemiological studies.

PURPOSE: The application of exclusion criteria in pharmacoepidemiological studies could have a major impact on the findings but there appears to have been no previous research to examine the types of exclusion criteria applied. METHODS: We searched the literature and identified 10 senior pharmacoepidemiologists who had published five or more relevant papers between 1999 and 2004. All their published drug safety studies during this period were reviewed. A classification system was developed to categorise the exclusion criteria, with 5 categories and 11 sub-categories. The categories were: (1) data quality and validation, (2) disease-related, (3) exposure-related, (4) patient characteristics and (5) miscellaneous reasons. Within each sub-category, only the first exclusion criterion identified for that study was counted. RESULTS: We identified 200 studies, from which a total of 752 exclusion criteria sub-categories had been applied (mean 3.8 per study; between-author range of means 2.8-5.1). At the category level, exclusion criteria relating to data quality and validation were the most commonly applied (87% of publications), followed by patient characteristics (75%), disease-related (69%), exposure-related (38%) and miscellaneous (3%). The main categories for which research practice appeared to differ were those relating to diseases and exposures. The application of sub-category 'risk factors and alternative causes' varied between authors from 0% to 81% of studies, and for the sub-category 'medication of interest' it varied from 5% to 93%. CONCLUSIONS: There are important differences between investigators in the application of exclusion criteria in pharmacoepidemiological studies. It is likely that a substantial part of the observed variation reflects different research practices of investigators.

Making the most of spontaneous adverse drug reaction reporting.

The primary purpose of spontaneous adverse drug reaction reporting is to provide early warnings or "signals" of previously unrecognized drug toxicity. The method was developed in the 1960s in response to the thalidomide tragedy and is now well-established throughout the developed world. Health professionals are the key original source of reports, the value of patient reporting is yet unclear. Electronic transmission of all reports is likely to become the norm within a few years. This is well-advanced between pharmaceutical companies and regulatory authorities but still in its infancy for health professionals in many parts of the world. Considered globally, the process may be inefficient and movement towards centralization of databases with appropriate access controls is logical. Alternative methods for capturing clinical suspicions of adverse drug reactions should be investigated and could provide more systematic data. However much it can be improved, spontaneous adverse drug reaction reporting is unlikely to identify all important unrecognized drug safety hazards. Complementary approaches therefore still need to be identified and developed.

Drug safety and the media.

This article reviews the problems that may arise as a result of media coverage of drug safety issues. In order to promote more balanced coverage and avoid unnecessary scares, professionals working in the area of drug safety should rethink their strategies for dealing with the media.

Comparison of hospital episodes with 'drug-induced' disorders and spontaneously reported adverse drug reactions.

AIMS: To compare Hospital Episode Statistics for 'drug-related' admissions with spontaneously reported adverse drug reactions (ADRs) using UK Yellow Card data for the period 1996-2000. METHODS: This was a descriptive study for which we matched the relevant datasets in respect of time, place, evidence of hospitalization and disease terminology. The principal outcome was the ratio of ADRs leading to hospitalization which had been reported spontaneously during the whole study period. RESULTS: Twenty types of ADR were included and between them there was a wide spread of overall ratios (range 0-130%). The general tendency was for under-reporting on Yellow Cards but for ADRs with a fatal outcome this appeared to be less (range 7-168%). CONCLUSIONS: This study provides some broad indications of the degree of under-reporting of ADRs that occurs despite a clinical diagnosis of a serious ADR being made and recorded.

Impact analysis of signals detected from spontaneous adverse drug reaction reporting data.

This paper describes a new method of prioritising signals of potential adverse drug reactions (ADRs) detected from spontaneous reports that is called impact analysis. This is an interim step between signal detection and detailed signal evaluation. Using mathematical screening tools, large numbers of signals may now be detected from spontaneous ADR databases. Regulatory authorities need to rapidly prioritise them and focus on those that are most likely to require significant action. Using two scores ranging from one to 100, each with three input variables, signals may be categorised in terms of the strength of evidence (E) and the potential public health impact (P). In a two-by-two figure with empirically derived cut-off points of ten (the logarithmic mean) for each score, signals are placed in one of four categories (A-D) that are ranked according to their priority (A being the highest and D the lowest). A sensitivity analysis is then performed that tests the robustness of the categorisation in relation to each of the six input variables. A computer program has been written to facilitate the process and reduce error. Further work is required to test the feasibility and value of impact analysis in practice.

Testing and implementing signal impact analysis in a regulatory setting: results of a pilot study.

BACKGROUND AND AIM: Statistical signal detection methods such as proportional reporting ratios (PRRs) detect many drug safety signals when applied to databases of spontaneous suspected adverse drug reactions (ADRs). Impact analysis is a tool that was developed as an aid to prioritisation of such signals. This paper describes a pilot project whereby impact analysis was simultaneously introduced into practice in a regulatory setting and tested in comparison with the existing approach. METHODS: Impact analysis was run on signals detected during a 26-week period from the UK Adverse Drug Reactions On-line Information Tracking (ADROIT) database of spontaneous ADRs that met minimum criteria (PRR>or=3.0, chi2>or=4.0 and >or=3 reported cases) and related to established drugs (i.e. those that have been available for at least 2 years and no longer carry the 'black triangle' symbol). The current method of signal prioritisation (i.e. the collective judgement at a weekly meeting) was initially performed without knowledge of the findings of impact analysis. Subsequently, the meeting was presented with the findings and, where appropriate, given the opportunity to reconsider the judgement made. The categories arising from the two methods were compared and the ultimate action recorded. Inter-observer variation between scientists performing impact analysis was also assessed. RESULTS: Eighty-six separate signals were analysed by impact analysis, of which 5% were categorised as high priority (A), 14% as requiring further information (B), 31% as low priority (C) and 50% as no action required (D). In general, the new method tended to give a higher level of priority to signals than the existing approach. Overall, there was 59% agreement between the impact analysis and the collective judgement at the meetings (kappa statistic=0.30). There was slightly greater agreement between impact analysis and the final action taken (kappa statistic=0.39), indicating that the findings of an impact analysis had an influence on the outcome. Assessment of inter-observer variation demonstrated that the method is repeatable (kappa statistic for overall category=0.77). Almost 70% of those who participated in the pilot study believed that impact analysis represented an improvement in how signals were prioritised. CONCLUSIONS: Impact analysis is a repeatable method of signal prioritisation that tended to give a higher level of priority to signals than the standard approach and which had an influence on the ultimate outcome.

Hospital admissions for 'drug-induced' disorders in England: a study using the Hospital Episodes Statistics (HES) database.

AIMS: To review Hospital Episode Statistics (HES) data for England coded as being 'drug induced' during 1996-2000 and to consider their potential utility for assessing the public health burden of adverse drug reactions (ADRs) and studying drug safety. METHODS: ICD-10 codes including the words 'drug-induced' or 'due to' a medicine or which are recognized to be invariably caused by a drug were extracted along with external cause codes indicating that a drug was implicated (i.e. Y40-59 in ICD-10). We also calculated the proportions of patients with each 'drug-induced' code for whom an external cause code had been applied. RESULTS: During the 5-year study period there were almost 53.8 million hospital admissions in England, of which 44 411 (0.083%) were coded as 'drug-induced' and 168 958 (0.314%) were associated with a relevant external cause code. The numbers of patients with 'drug-induced' codes used were generally stable during the study period (range 7454-8860 per year) but the application of external cause codes increased in each year and by 40% overall (from 24 786 in 1996 to 34 843 in 2000). The overall proportion of 'drug-induced' codes associated with a relevant external cause code was quite low (12-15%) but there was considerable variation between codes. CONCLUSIONS: Comparisons with published studies indicate that HES data grossly underestimate the burden of drug-induced disorders as a cause of hospital admission. There are likely to be multiple underlying reasons including under-recognition, under-recording and limitations of the coding system. The potential of these data for identifying previously unrecognized serious ADRs is limited by constraints on the availability of detailed data regarding individual cases.

A model for the future conduct of pharmacovigilance.

A scientific model to support excellence in pharmacovigilance has been developed from first principles by brainstorming sessions and discussions with experts in the field. The model represents a long-term vision of how pharmacovigilance could be conducted in the future. So far it has been developed without any consideration of constraints such as resources or the need for legislative change. Although the vision is holistic, it would be possible to test and implement parts of the model in a piecemeal fashion.