Adverse Events in Twitter-Development of a Benchmark Reference Dataset: Results from IMI WEB-RADR.

INTRODUCTION AND OBJECTIVE: Social media has been suggested as a source for safety information, supplementing existing safety surveillance data sources. This article summarises the activities undertaken, and the associated challenges, to create a benchmark reference dataset that can be used to evaluate the performance of automated methods and systems for adverse event recognition. METHODS: A retrospective analysis of public English-language Twitter posts (Tweets) was performed. We sampled 57,473 Tweets out of 5,645,336 Tweets created between 1 March, 2012 and 1 March, 2015 that mentioned at least one of six medicinal products of interest (insulin glargine, levetiracetam, methylphenidate, sorafenib, terbinafine, zolpidem). Products, adverse events, indications, product-event combinations, and product-indication combinations were extracted and coded by two independent teams of safety reviewers. RESULTS: The benchmark reference dataset consisted of 1056 positive controls ("adverse event Tweets") and 56,417 negative controls ("non-adverse event Tweets"). The 1056 adverse event Tweets contained 1396 product-event combinations referring to personal adverse event experiences, comprising 292 different MedDRA® Preferred Terms. The 1171 product-event combinations (83.9%) were confined to four MedDRA® System Organ Classes. The 195 Tweets (18.5%) contained indication information, comprising 25 different Preferred Terms. CONCLUSIONS: A manually curated benchmark reference dataset based on Twitter data has been created and is made available to the research community to evaluate the performance of automated methods and systems for adverse event recognition in unstructured free-text information.

Recommendations for the Use of Social Media in Pharmacovigilance: Lessons from IMI WEB-RADR.

Over a period of 3 years, the European Union's Innovative Medicines Initiative WEB-RADR project has explored the value of social media (i.e., information exchanged through the internet, typically via online social networks) for identifying adverse events as well as for safety signal detection. Many patients and clinicians have taken to social media to discuss their positive and negative experiences of medications, creating a source of publicly available information that has the potential to provide insights into medicinal product safety concerns. The WEB-RADR project has developed a collaborative English language workspace for visualising and analysing social media data for a number of medicinal products. Further, novel text and data mining methods for social media analysis have been developed and evaluated. From this original research, several recommendations are presented with supporting rationale and consideration of the limitations. Recommendations for further research that extend beyond the scope of the current project are also presented.

Assessment of the Utility of Social Media for Broad-Ranging Statistical Signal Detection in Pharmacovigilance: Results from the WEB-RADR Project.

INTRODUCTION AND OBJECTIVE: Social media has been proposed as a possibly useful data source for pharmacovigilance signal detection. This study primarily aimed to evaluate the performance of established statistical signal detection algorithms in Twitter/Facebook for a broad range of drugs and adverse events. METHODS: Performance was assessed using a reference set by Harpaz et al., consisting of 62 US Food and Drug Administration labelling changes, and an internal WEB-RADR reference set consisting of 200 validated safety signals. In total, 75 drugs were studied. Twitter/Facebook posts were retrieved for the period March 2012 to March 2015, and drugs/events were extracted from the posts. We retrieved 4.3 million and 2.0 million posts for the WEB-RADR and Harpaz drugs, respectively. Individual case reports were extracted from VigiBase for the same period. Disproportionality algorithms based on the Information Component or the Proportional Reporting Ratio and crude post/report counting were applied in Twitter/Facebook and VigiBase. Receiver operating characteristic curves were generated, and the relative timing of alerting was analysed. RESULTS: Across all algorithms, the area under the receiver operating characteristic curve for Twitter/Facebook varied between 0.47 and 0.53 for the WEB-RADR reference set and between 0.48 and 0.53 for the Harpaz reference set. For VigiBase, the ranges were 0.64-0.69 and 0.55-0.67, respectively. In Twitter/Facebook, at best, 31 (16%) and four (6%) positive controls were detected prior to their index dates in the WEB-RADR and Harpaz references, respectively. In VigiBase, the corresponding numbers were 66 (33%) and 17 (27%). CONCLUSIONS: Our results clearly suggest that broad-ranging statistical signal detection in Twitter and Facebook, using currently available methods for adverse event recognition, performs poorly and cannot be recommended at the expense of other pharmacovigilance activities.

Intraoperative MRI to guide the resection of primary supratentorial glioblastoma multiforme--a quantitative radiological analysis.

Patients with supratentorial high-grade glioma underwent surgery within a vertically open 0.5-T magnetic resonance (MR) system to evaluate the efficacy of intraoperative MR guidance in achieving gross-total resection. For 31 patients, preoperative clinical data and MR findings were consistent with the putative diagnosis of a high-grade glioma, in 23 cases in eloquent regions. Tumor resections were carried out within a 0.5-T MR SIGNA SP/i (GE Medical Systems, USA). The resection of the lesion was carried out using fully MR compatible neurosurgical equipment and was stopped at the point when the operation was considered complete by the surgeon viewing the operation field with the microscope. We repeated imaging to determine the residual tumor volume only visible with MRI. Areas of tissue that were abnormal on these images were localized in the bed of resection by using interactive MR guidance. The procedure of resection, imaging control and interactive image guidance was repeated where necessary. Almost all tissue with abnormal characteristics was resected, with the exception of tissue localized in eloquent brain areas. The diagnosis of glioblastoma was confirmed in all 31 cases. When comparing the tumor volume before resection and at the point where the neurosurgeon would otherwise have terminated surgery ("first control"), residual tumor tissue was detectable in 29/31 patients; the mean residual tumor volume was 30.7 +/- 24%. After repeated resections under interactive image guidance the mean residual tumor volume was 15.1%. At this step we found tumor remnants only in 20/31 patients. The perioperative morbidity (12.9%) was low. Twenty-seven patients underwent sufficient postoperative radiotherapy. We found a significant difference (log(rank)p = 0.0037) in the mean survival times of the two groups with complete resection (n = 10, median survival time 537 days) and incomplete resection (n = 17, median survival time 237 days). The resection of primary glioblastoma multiforme under intraoperative MR guidance as demonstrated is a possibility to achieve a more complete removal of the tumor than with conventional techniques. In our small but homogeneous patient group we found an increase in the median survival time in patients with MRI for complete tumor resection, and the overall surgical morbidity was low.