ABSTRACT
BACKGROUND: Social media has become an established platform for individuals to discuss and debate various subjects, including vaccination. With growing conversations on the web and less than desired maternal vaccination uptake rates, these conversations could provide useful insights to inform future interventions. However, owing to the volume of web-based posts, manual annotation and analysis are difficult and time consuming. Automated processes for this type of analysis, such as natural language processing, have faced challenges in extracting complex stances such as attitudes toward vaccination from large amounts of text. OBJECTIVE: The aim of this study is to build upon recent advances in transposer-based machine learning methods and test whether transformer-based machine learning could be used as a tool to assess the stance expressed in social media posts toward vaccination during pregnancy. METHODS: A total of 16,604 tweets posted between November 1, 2018, and April 30, 2019, were selected using keyword searches related to maternal vaccination. After excluding irrelevant tweets, the remaining tweets were coded by 3 individual researchers into the categories Promotional, Discouraging, Ambiguous, and Neutral or No Stance. After creating a final data set of 2722 unique tweets, multiple machine learning techniques were trained on a part of this data set and then tested and compared with the human annotators. RESULTS: We found the accuracy of the machine learning techniques to be 81.8% (F score=0.78) compared with the agreed score among the 3 annotators. For comparison, the accuracies of the individual annotators compared with the final score were 83.3%, 77.9%, and 77.5%. CONCLUSIONS: This study demonstrates that we are able to achieve close to the same accuracy in categorizing tweets using our machine learning models as could be expected from a single human coder. The potential to use this automated process, which is reliable and accurate, could free valuable time and resources for conducting this analysis, in addition to informing potentially effective and necessary interventions.
ABSTRACT
OBJECTIVE: To understand the predominant topics of discussion, stance and associated language used on social media platforms relating to maternal vaccines in 15 countries over a six-month period. BACKGROUND: In 2019, the World Health Organisation prioritised vaccine hesitancy as a top ten global health threat and recognized the role of viral misinformation on social media as propagating vaccine hesitancy. Maternal vaccination offers the potential to improve maternal and child health, and to reduce the risk of severe morbidity and mortality in pregnancy. Understanding the topics of discussion, stance and language used around maternal vaccines on social media can inform public health bodies on how to combat vaccine misinformation and vaccine hesitancy. METHODS: Social media data was extracted (Twitter, forums, blogs and comments) for six months from 15 countries (Australia, Brazil, Canada, France, Germany, India, Italy, Korea, Mexico, Panama, South Africa, Spain, United Kingdom and United States). We used stance, discourse and topic analysis to provide insight into the most frequent and weighted keywords, hashtags and themes of conversation within and across countries. RESULTS: We exported a total of 19,192 social media posts in 16 languages obtained between 1st November 2018 and 30th April 2019. After screening all posts, 16,000 were included in analyses, while excluding retweets, 2,722 were annotated for sentiment. Main topics of discussion were the safety of the maternal influenza and pertussis vaccines. Discouraging posts were most common in Italy (44.9%), and the USA (30.8%). CONCLUSION: The content and stance of maternal vaccination posts from November 2018 to April 2019 differed across countries, however specific topics of discussion were not limited to geographical location. These discussions included the promotion of vaccination, involvement of pregnant women in vaccine research, and the trust and transparency of institutions. Future research should examine the relationship between stance (promotional, neutral, ambiguous, discouraging) online and maternal vaccination uptake in the respective regions.
