Evaluating Web-Based Automatic Transcription for Alzheimer Speech Data: Transcript Comparison and Machine Learning Analysis.

BACKGROUND: Speech data for medical research can be collected noninvasively and in large volumes. Speech analysis has shown promise in diagnosing neurodegenerative disease. To effectively leverage speech data, transcription is important, as there is valuable information contained in lexical content. Manual transcription, while highly accurate, limits the potential scalability and cost savings associated with language-based screening. OBJECTIVE: To better understand the use of automatic transcription for classification of neurodegenerative disease, namely, Alzheimer disease (AD), mild cognitive impairment (MCI), or subjective memory complaints (SMC) versus healthy controls, we compared automatically generated transcripts against transcripts that went through manual correction. METHODS: We recruited individuals from a memory clinic ("patients") with a diagnosis of mild-to-moderate AD, (n=44, 30%), MCI (n=20, 13%), SMC (n=8, 5%), as well as healthy controls (n=77, 52%) living in the community. Participants were asked to describe a standardized picture, read a paragraph, and recall a pleasant life experience. We compared transcripts generated using Google speech-to-text software to manually verified transcripts by examining transcription confidence scores, transcription error rates, and machine learning classification accuracy. For the classification tasks, logistic regression, Gaussian naive Bayes, and random forests were used. RESULTS: The transcription software showed higher confidence scores (P<.001) and lower error rates (P>.05) for speech from healthy controls compared with patients. Classification models using human-verified transcripts significantly (P<.001) outperformed automatically generated transcript models for both spontaneous speech tasks. This comparison showed no difference in the reading task. Manually adding pauses to transcripts had no impact on classification performance. However, manually correcting both spontaneous speech tasks led to significantly higher performances in the machine learning models. CONCLUSIONS: We found that automatically transcribed speech data could be used to distinguish patients with a diagnosis of AD, MCI, or SMC from controls. We recommend a human verification step to improve the performance of automatic transcripts, especially for spontaneous tasks. Moreover, human verification can focus on correcting errors and adding punctuation to transcripts. However, manual addition of pauses is not needed, which can simplify the human verification step to more efficiently process large volumes of speech data.

Tracking Public Attitudes Toward COVID-19 Vaccination on Tweets in Canada: Using Aspect-Based Sentiment Analysis.

BACKGROUND: The development and approval of COVID-19 vaccines have generated optimism for the end of the COVID-19 pandemic and a return to normalcy. However, vaccine hesitancy, often fueled by misinformation, poses a major barrier to achieving herd immunity. OBJECTIVE: We aim to investigate Twitter users' attitudes toward COVID-19 vaccination in Canada after vaccine rollout. METHODS: We applied a weakly supervised aspect-based sentiment analysis (ABSA) technique, which involves the human-in-the-loop system, on COVID-19 vaccination-related tweets in Canada. Automatically generated aspect and opinion terms were manually corrected by public health experts to ensure the accuracy of the terms and make them more domain-specific. Then, based on these manually corrected terms, the system inferred sentiments toward the aspects. We observed sentiments toward key aspects related to COVID-19 vaccination, and investigated how sentiments toward "vaccination" changed over time. In addition, we analyzed the most retweeted or liked tweets by observing most frequent nouns and sentiments toward key aspects. RESULTS: After applying the ABSA system, we obtained 170 aspect terms (eg, "immunity" and "pfizer") and 6775 opinion terms (eg, "trustworthy" for the positive sentiment and "jeopardize" for the negative sentiment). While manually verifying or editing these terms, our public health experts selected 20 key aspects related to COVID-19 vaccination for analysis. The sentiment analysis results for the 20 key aspects revealed negative sentiments related to "vaccine distribution," "side effects," "allergy," "reactions," and "anti-vaxxer," and positive sentiments related to "vaccine campaign," "vaccine candidates," and "immune response." These results indicate that the Twitter users express concerns about the safety of vaccines but still consider vaccines as the option to end the pandemic. In addition, compared to the sentiment of the remaining tweets, the most retweeted or liked tweets showed more positive sentiment overall toward key aspects (P<.001), especially vaccines (P<.001) and vaccination (P=.009). Further investigation of the most retweeted or liked tweets revealed two opposing trends in Twitter users who showed negative sentiments toward vaccines: the "anti-vaxxer" population that used negative sentiments as a means to discourage vaccination and the "Covid Zero" population that used negative sentiments to encourage vaccinations while critiquing the public health response. CONCLUSIONS: Our study examined public sentiments toward COVID-19 vaccination on tweets over an extended period in Canada. Our findings could inform public health agencies to design and implement interventions to promote vaccination.

Identifying Language Features Associated With Needs of Ovarian Cancer Patients and Caregivers Using Social Media.

BACKGROUND: Online health communities (OHCs) can be a source for clinicians to learn the needs of cancer patients and caregivers. Ovarian cancer (OvCa) patients and caregivers deal with a wide range of unmet needs, many of which are expressed in OHCs. An automated need classification model could help clinicians more easily understand and prioritize information available in the OHCs. OBJECTIVE: The aim of this study was to use initial OHC postings to develop an automated model for the classification of OvCa patient and caregiver needs. METHODS: We collected data from the OvCa OHC and analyzed the initial postings of patients and caregivers (n = 853). Two annotators coded each posting with 12 types of needs. Then, we applied the machine learning approach with bag-of-words features to build a model to classify needs. F1 score, an indicator of model accuracy, was used to evaluate the model. RESULTS: The most reported needs were information, social, psychological/emotional, and physical. Thirty-nine percent of postings described information and social needs in the same posting. Our model reported a high level of accuracy for classifying those top needs. Psychological terms were important for classifying psychological/emotional and social needs. Medical terms were important for physical and information needs. CONCLUSIONS: We demonstrate the potential of using OHCs to supplement traditional needs assessment. Further research would incorporate additional information (eg, trajectory, stage) for more sophisticated models. IMPLICATIONS FOR PRACTICE: This study shows the potential of automated classification to leverage OHCs for needs assessment. Our approach can be applied to different types of cancer and enhanced by using domain-specific information.

Classification of Alzheimer's Disease Leveraging Multi-task Machine Learning Analysis of Speech and Eye-Movement Data.

Alzheimer's disease (AD) is a progressive neurodegenerative condition that results in impaired performance in multiple cognitive domains. Preclinical changes in eye movements and language can occur with the disease, and progress alongside worsening cognition. In this article, we present the results from a machine learning analysis of a novel multimodal dataset for AD classification. The cohort includes data from two novel tasks not previously assessed in classification models for AD (pupil fixation and description of a pleasant past experience), as well as two established tasks (picture description and paragraph reading). Our dataset includes language and eye movement data from 79 memory clinic patients with diagnoses of mild-moderate AD, mild cognitive impairment (MCI), or subjective memory complaints (SMC), and 83 older adult controls. The analysis of the individual novel tasks showed similar classification accuracy when compared to established tasks, demonstrating their discriminative ability for memory clinic patients. Fusing the multimodal data across tasks yielded the highest overall AUC of 0.83 ± 0.01, indicating that the data from novel tasks are complementary to established tasks.

Applications of Aspect-based Sentiment Analysis on Psychiatric Clinical Notes to Study Suicide in Youth.

Understanding and identifying the risk factors associated with suicide in youth experiencing mental health concerns is paramount to early intervention. 45% of patients are admitted annually for suicidality at BC Children's Hospital. Natural Language Processing (NLP) approaches have been applied with moderate success to psychiatric clinical notes to predict suicidality. Our objective was to explore whether machine-learning-based sentiment analysis could be informative in such a prediction task. We developed a psychiatry-relevant lexicon and identified specific categories of words, such as thought content and thought process that had significantly different polarity between suicidal and non-suicidal cases. In addition, we demonstrated that the individual words with their associated polarity can be used as features in classification models and carry informative content to differentiate between suicidal and non-suicidal cases. In conclusion, our study reveals that there is much value in applying NLP to psychiatric clinical notes and suicidal prediction.

Tracking COVID-19 Discourse on Twitter in North America: Infodemiology Study Using Topic Modeling and Aspect-Based Sentiment Analysis.

BACKGROUND: Social media is a rich source where we can learn about people's reactions to social issues. As COVID-19 has impacted people's lives, it is essential to capture how people react to public health interventions and understand their concerns. OBJECTIVE: We aim to investigate people's reactions and concerns about COVID-19 in North America, especially in Canada. METHODS: We analyzed COVID-19-related tweets using topic modeling and aspect-based sentiment analysis (ABSA), and interpreted the results with public health experts. To generate insights on the effectiveness of specific public health interventions for COVID-19, we compared timelines of topics discussed with the timing of implementation of interventions, synergistically including information on people's sentiment about COVID-19-related aspects in our analysis. In addition, to further investigate anti-Asian racism, we compared timelines of sentiments for Asians and Canadians. RESULTS: Topic modeling identified 20 topics, and public health experts provided interpretations of the topics based on top-ranked words and representative tweets for each topic. The interpretation and timeline analysis showed that the discovered topics and their trend are highly related to public health promotions and interventions such as physical distancing, border restrictions, handwashing, staying home, and face coverings. After training the data using ABSA with human-in-the-loop, we obtained 545 aspect terms (eg, "vaccines," "economy," and "masks") and 60 opinion terms such as "infectious" (negative) and "professional" (positive), which were used for inference of sentiments of 20 key aspects selected by public health experts. The results showed negative sentiments related to the overall outbreak, misinformation and Asians, and positive sentiments related to physical distancing. CONCLUSIONS: Analyses using natural language processing techniques with domain expert involvement can produce useful information for public health. This study is the first to analyze COVID-19-related tweets in Canada in comparison with tweets in the United States by using topic modeling and human-in-the-loop domain-specific ABSA. This kind of information could help public health agencies to understand public concerns as well as what public health messages are resonating in our populations who use Twitter, which can be helpful for public health agencies when designing a policy for new interventions.