Research Priorities to Increase Confidence in and Acceptance of Health Preference Research: What Questions Should be Prioritized Now?

BACKGROUND AND OBJECTIVE: There has been an increase in the study and use of stated-preference methods to inform medicine development decisions. The objective of this study was to identify prioritized topics and questions relating to health preferences based on the perspective of members of the preference research community. METHODS: Preference research stakeholders from industry, academia, consultancy, health technology assessment/regulatory, and patient organizations were recruited using professional networks and preference-targeted e-mail listservs and surveyed about their perspectives on 19 topics and questions for future studies that would increase acceptance of preference methods and their results by decision makers. The online survey consisted of an initial importance prioritization task, a best-worst scaling case 1 instrument, and open-ended questions. Rating counts were used for analysis. The best-worst scaling used a balanced incomplete block design. RESULTS: One hundred and one participants responded to the survey invitation with 66 completing the best-worst scaling. The most important research topics related to the synthesis of preferences across studies, transferability across populations or related diseases, and method topics including comparison of methods and non-discrete choice experiment methods. Prioritization differences were found between respondents whose primary affiliation was academia versus other stakeholders. Academic researchers prioritized methodological/less studied topics; other stakeholders prioritized applied research topics relating to consistency of practice. CONCLUSIONS: As the field of health preference research grows, there is a need to revisit and communicate previous work on preference selection and study design to ensure that new stakeholders are aware of this work and to update these works where necessary. These findings might encourage discussion and alignment among different stakeholders who might hold different research priorities. Research on the application of previous preference research to new contexts will also help increase the acceptance of health preference information by decision makers.

A nonparametric method to detect increased frequencies of adverse drug reactions over time.

Signal detection is routinely applied to spontaneous report safety databases in the pharmaceutical industry and by regulators. As an example, methods that search for increases in the frequencies of known adverse drug reactions for a given drug are routinely applied, and the results are reported to the health authorities on a regular basis. Such methods need to be sensitive to detect true signals even when some of the adverse drug reactions are rare. The methods need to be specific and account for multiplicity to avoid false positive signals when the list of known adverse drug reactions is long. To apply them as part of a routine process, the methods also have to cope with very diverse drugs (increasing or decreasing number of cases over time, seasonal patterns, very safe drugs versus drugs for life-threatening diseases). In this paper, we develop new nonparametric signal detection methods, directed at detecting differences between a reporting and a reference period, or trends within a reporting period. These methods are based on bootstrap and permutation distributions, and they combine statistical significance with clinical relevance. We conducted a large simulation study to understand the operating characteristics of the methods. Our simulations show that the new methods have good power and control the family-wise error rate at the specified level. Overall, in all scenarios that we explored, the method performs much better than our current standard in terms of power, and it generates considerably less false positive signals as compared to the current standard.

Are all quantitative postmarketing signal detection methods equal? Performance characteristics of logistic regression and Multi-item Gamma Poisson Shrinker.

PURPOSE: The detection of safety signals with medicines is an essential activity to protect public health. Despite widespread acceptance, it is unclear whether recently applied statistical algorithms provide enhanced performance characteristics when compared with traditional systems. Novartis has adopted a novel system for automated signal detection on the basis of disproportionality methods within a safety data mining application (Empirica™ Signal System [ESS]). ESS uses two algorithms for routine analyses: empirical Bayes Multi-item Gamma Poisson Shrinker and logistic regression (LR). METHODS: A model was developed comprising 14 medicines, categorized as "new" or "established." A standard was prepared on the basis of safety findings selected from traditional sources. ESS results were compared with the standard to calculate the positive predictive value (PPV), specificity, and sensitivity. PPVs of the lower one-sided 5% and 0.05% confidence limits of the Bayes geometric mean (EB05) and of the LR odds ratio (LR0005) almost coincided for all the drug-event combinations studied. RESULTS: There was no obvious difference comparing the PPV of the leading Medical Dictionary for Regulatory Activities (MedDRA) terms to the PPV for all terms. The PPV of narrow MedDRA query searches was higher than that for broad searches. The widely used threshold value of EB05 = 2.0 or LR0005 = 2.0 together with more than three spontaneous reports of the drug-event combination produced balanced results for PPV, sensitivity, and specificity. CONCLUSIONS: Consequently, performance characteristics were best for leading terms with narrow MedDRA query searches irrespective of applying Multi-item Gamma Poisson Shrinker or LR at a threshold value of 2.0. This research formed the basis for the configuration of ESS for signal detection at Novartis.