Capturing Emerging Experiential Knowledge for Vaccination Guidelines Through Natural Language Processing: Proof-of-Concept Study.

BACKGROUND: Experience-based knowledge and value considerations of health professionals, citizens, and patients are essential to formulate public health and clinical guidelines that are relevant and applicable to medical practice. Conventional methods for incorporating such knowledge into guideline development often involve a limited number of representatives and are considered to be time-consuming. Including experiential knowledge can be crucial during rapid guidance production in response to a pandemic but it is difficult to accomplish. OBJECTIVE: This proof-of-concept study explored the potential of artificial intelligence (AI)-based methods to capture experiential knowledge and value considerations from existing data channels to make these insights available for public health guideline development. METHODS: We developed and examined AI-based methods in relation to the COVID-19 vaccination guideline development in the Netherlands. We analyzed Dutch messages shared between December 2020 and June 2021 on social media and on 2 databases from the Dutch National Institute for Public Health and the Environment (RIVM), where experiences and questions regarding COVID-19 vaccination are reported. First, natural language processing (NLP) filtering techniques and an initial supervised machine learning model were developed to identify this type of knowledge in a large data set. Subsequently, structural topic modeling was performed to discern thematic patterns related to experiences with COVID-19 vaccination. RESULTS: NLP methods proved to be able to identify and analyze experience-based knowledge and value considerations in large data sets. They provide insights into a variety of experiential knowledge that is difficult to obtain otherwise for rapid guideline development. Some topics addressed by citizens, patients, and professionals can serve as direct feedback to recommendations in the guideline. For example, a topic pointed out that although travel was not considered as a reason warranting prioritization for vaccination in the national vaccination campaign, there was a considerable need for vaccines for indispensable travel, such as cross-border informal caregiving, work or study, or accessing specialized care abroad. Another example is the ambiguity regarding the definition of medical risk groups prioritized for vaccination, with many citizens not meeting the formal priority criteria while being equally at risk. Such experiential knowledge may help the early identification of problems with the guideline's application and point to frequently occurring exceptions that might initiate a revision of the guideline text. CONCLUSIONS: This proof-of-concept study presents NLP methods as viable tools to access and use experience-based knowledge and value considerations, possibly contributing to robust, equitable, and applicable guidelines. They offer a way for guideline developers to gain insights into health professionals, citizens, and patients' experience-based knowledge, especially when conventional methods are difficult to implement. AI-based methods can thus broaden the evidence and knowledge base available for rapid guideline development and may therefore be considered as an important addition to the toolbox of pandemic preparedness.

Monitoring stance towards vaccination in twitter messages.

BACKGROUND: We developed a system to automatically classify stance towards vaccination in Twitter messages, with a focus on messages with a negative stance. Such a system makes it possible to monitor the ongoing stream of messages on social media, offering actionable insights into public hesitance with respect to vaccination. At the moment, such monitoring is done by means of regular sentiment analysis with a poor performance on detecting negative stance towards vaccination. For Dutch Twitter messages that mention vaccination-related key terms, we annotated their stance and feeling in relation to vaccination (provided that they referred to this topic). Subsequently, we used these coded data to train and test different machine learning set-ups. With the aim to best identify messages with a negative stance towards vaccination, we compared set-ups at an increasing dataset size and decreasing reliability, at an increasing number of categories to distinguish, and with different classification algorithms. RESULTS: We found that Support Vector Machines trained on a combination of strictly and laxly labeled data with a more fine-grained labeling yielded the best result, at an F1-score of 0.36 and an Area under the ROC curve of 0.66, considerably outperforming the currently used sentiment analysis that yielded an F1-score of 0.25 and an Area under the ROC curve of 0.57. We also show that the recall of our system could be optimized to 0.60 at little loss of precision. CONCLUSION: The outcomes of our study indicate that stance prediction by a computerized system only is a challenging task. Nonetheless, the model showed sufficient recall on identifying negative tweets so as to reduce the manual effort of reviewing messages. Our analysis of the data and behavior of our system suggests that an approach is needed in which the use of a larger training dataset is combined with a setting in which a human-in-the-loop provides the system with feedback on its predictions.

Fora fuelling the discovery of fortified dietary supplements - An exploratory study directed at monitoring the internet for contaminated food supplements based on the reported effects of their users.

Dietary supplements are products that are widely used for instance as energisers or to lose weight. There have been cases reported where undeclared ingredients present in such supplements have caused adverse effects on the health of the user. As there are many different products to choose from, it seems impossible to predict which might contain harmful components and to ban them from the market. Nonetheless, the use of dietary supplements and the experiences of users are shared in online discussions. We describe the development of a search engine to retrieve products associated with certain effects. Upon application we were able to retrieve a list of dietary supplements that are repeatedly associated with excessive effects by users on public fora. The top of the list contains supplements that have previously been banned because they contained undeclared harmful components. The use of the search engine as described here is a powerful method for making a risk-based selection of dietary supplements which can then be analysed for the presence of illegal or other unwanted components.

Too Far to Care? Measuring Public Attention and Fear for Ebola Using Twitter.

BACKGROUND: In 2014, the world was startled by a sudden outbreak of Ebola. Although Ebola infections and deaths occurred almost exclusively in Guinea, Sierra Leone, and Liberia, few potential Western cases, in particular, caused a great stir among the public in Western countries. OBJECTIVE: This study builds on the construal level theory to examine the relationship between psychological distance to an epidemic and public attention and sentiment expressed on Twitter. Whereas previous research has shown the potential of social media to assess real-time public opinion and sentiment, generalizable insights that further the theory development lack. METHODS: Epidemiological data (number of Ebola infections and fatalities) and media data (tweet volume and key events reported in the media) were collected for the 2014 Ebola outbreak, and Twitter content from the Netherlands was coded for (1) expressions of fear for self or fear for others and (2) psychological distance of the outbreak to the tweet source. Longitudinal relations were compared using vector error correction model (VECM) methodology. RESULTS: Analyses based on 4500 tweets revealed that increases in public attention to Ebola co-occurred with severe world events related to the epidemic, but not all severe events evoked fear. As hypothesized, Web-based public attention and expressions of fear responded mainly to the psychological distance of the epidemic. A chi-square test showed a significant positive relation between proximity and fear: χ22=103.2 (P<.001). Public attention and fear for self in the Netherlands showed peaks when Ebola became spatially closer by crossing the Mediterranean Sea and Atlantic Ocean. Fear for others was mostly predicted by the social distance to the affected parties. CONCLUSIONS: Spatial and social distance are important predictors of public attention to worldwide crisis such as epidemics. These factors need to be taken into account when communicating about human tragedies.